Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch macos x86_64 to macos14 in generate-ci #2246

Closed
wants to merge 1 commit into from

Conversation

masklinn
Copy link

@masklinn masklinn commented Oct 5, 2024

The macos12 images are deprecated and will be decommissioned by december, jobs have started triggering warnings and brownouts are planned.

In my understanding this means the x86_64 wheels will be cross-compiled from the arm64 runners, but I understand cross compilation works fine. The x86_64 cross compilation is slower than the ARM64 compilation, but on my test project it's faster than the macos13 x86_64 build so that doesn't seem like a massive concern I feel?

@masklinn masklinn changed the title Switch macos x86_64 to macos14 Switch macos x86_64 to macos14 in generate-ci Oct 5, 2024
The macos12 images [are deprecated and will be decommissioned by december](actions/runner-images#10721), jobs have started triggering warnings and brownouts are planned.

In my understanding this means the x86_64 wheels will be cross-compiled from the arm64 runners, but I understand cross compilation works fine. The x86_64 cross compilation is slower than the ARM64 compilation, but on my test project it's faster than the macos13 x86_64 build so that doesn't seem like a massive concern I feel?
Copy link
Member

@messense messense left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, note that pytest step also needs adjustment because you can't run tests for x86_64 wheels on aarch64 macOS after this change.

@masklinn
Copy link
Author

masklinn commented Oct 9, 2024

Thanks, note that pytest step also needs adjustment because you can't run tests for x86_64 wheels on aarch64 macOS after this change.

Ah good catch, I assumed either arm64 or x86_64 python would be installed and then that's what would be run, but not so, and it's not even a question of enabling architecture for macos as from its docs setup-python installs a universal binary.

From my testing it looks like arch will correctly select the architecture, so I could set env: ARCHPREFERENCE on the run step then prefix every relevant command (pip install to ensure the correct binary arch is selected, and running the test suite) with arch and it should work correctly, what do you think?

@messense
Copy link
Member

messense commented Oct 10, 2024

That may not work with pypy I think, they don't have universal2 binaries: https://pypy.org/download.html

From https://github.com/actions/python-versions/releases/tag/3.13.0-11228081754 I didn't find any universal2 python distributions.

@messense messense linked an issue Oct 10, 2024 that may be closed by this pull request
2 tasks
@masklinn
Copy link
Author

That may not work with pypy I think, they don't have universal2 binaries: https://pypy.org/download.html

So should probably pass the architecture to setup-python still (same as Windows) in order to retrieve the correct pypy (and graal as they don't seem to provide universal binaries either), then arch should not hurt, worst case scenario it'll trigger an error if we're running the wrong architecture which seems a benefit more than a drawback e.g.

$ arch -x86_64 graalpy-24.1.0-macos-aarch64/bin/graalpy
arch: posix_spawnp: graalpy-24.1.0-macos-aarch64/bin/graalpy: Bad CPU type in executable

From https://github.com/actions/python-versions/releases/tag/3.13.0-11228081754 I didn't find any universal2 python distributions.

The note about universal binaries is in the readme: https://github.com/actions/python-versions/tree/3.13.0-11228081754?tab=readme-ov-file#building-installation-packages

The x86_64 and amd64 assets bundle the exact same thing:

$ sha1sum python-3.13.0-darwin-x64/python-3.13.0-macos11.pkg
1c1e6e6cf6370ff8020fb7b93a8b408f23d65054  python-3.13.0-darwin-x64/python-3.13.0-macos11.pkg
$ shasum python-3.13.0-darwin-arm64/python-3.13.0-macos11.pkg
1c1e6e6cf6370ff8020fb7b93a8b408f23d65054  python-3.13.0-darwin-arm64/python-3.13.0-macos11.pkg

@messense
Copy link
Member

The note about universal binaries is in the readme: https://github.com/actions/python-versions/tree/3.13.0-11228081754?tab=readme-ov-file#building-installation-packages

Right, but note that it's only for Python >= 3.11.

@masklinn
Copy link
Author

Right, but note that it's only for Python >= 3.11.

Sure but generate-ci creates a template for 3.x so that's been >= 3.11 since it was created.

Though if we pass the architecture to setup-python for the benefit of pypy and graal it should get picked up for cpython < 3.11 and retrieve arch-specific payloads, and like pypy and graal the arch is basically just an assertion.

I'll try that out to make sure it works.

@masklinn
Copy link
Author

masklinn commented Oct 11, 2024

Well apparently it doesn't, python < 3.11 x64 seems broken on M1: https://github.com/masklinn/ci-test/actions/runs/11296863554/job/31422564472

I'll have to see with the setup-python people if they know why.

edit: opened actions/setup-python#960

@masklinn
Copy link
Author

Welp, apparently action/setup-python has decided not to fix the non-universal python images, so not quite sure what to do there, I guess I'll close this.?

@messense
Copy link
Member

Close for now, we have switched to macos-13.

@messense messense closed this Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

actions/runner-images deprecating macos-12
2 participants