{2025.06}[2024a] PyTorch 2.9.1#1389
Draft
bedroge wants to merge 18 commits into
Draft
Conversation
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen4 |
|
New job on instance
|
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen4 |
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen4 |
|
New job on instance
|
Collaborator
Author
Errors are quite similar to the ones observed in #1314, many of these: |
Collaborator
Author
|
Updated hooks file with a fix for PyTorch has been ingested (EESSI/software-layer-scripts#172), let's try again. bot: build repo:eessi.io-2025.06-software instance:eessi-bot-aws-eu-south for:arch=x86_64/amd/zen5 |
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
Collaborator
Author
|
The neoverse v1 build ran out of memory: |
Collaborator
Author
|
I've modified bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1 |
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace |
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/generic |
|
New job on instance
|
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1 |
|
New job on instance
|
Collaborator
Author
|
No more memory issues for the neoverse v1 build, but too many failing tests: |
Contributor
That's the major blocker. Anything in the logs?
That's the one with the most failures (so far). Do you have a built without ACL to compare? Can you attach the log to check if it is possibly a single issue over many tests? |
Collaborator
Author
Here's the log of the grace build. I'll have to check if I can still access the detailed log file of a successful build without ACL. |
Collaborator
Author
|
Found it, here's the log of the successful build without ACL (#1389 (comment)). |
Contributor
|
Ok, thanks. I found the origin of 54 new failures which is a single test with variations. Added a patch in the PR
I'm now just skipping the first 2 on AARCH64 as they don't seem to get tested upstream either. |
Collaborator
Author
How would I do that exactly? I tried pip installing that version, extracting the source tarball, and running |
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace |
|
New job on instance
|
Contributor
Exactly like that. The error is a conflict with our hypothesis version or so. I guess you can just apply |
Collaborator
Author
|
Thanks, that solved the issue. Now I get this: |
Contributor
|
Ok, then I consider this a known issue and skip the test. PR updated |
Collaborator
Author
|
Thanks! Let me try again: bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace |
Collaborator
Author
|
Oops, forgot to update the commit, but looks like the bot is down anyway. Will retry later. |
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace |
|
New job on instance
|
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-deucalion for:arch=aarch64/a64fx |
|
New job on instance
|
Collaborator
Author
|
41 test failures now for a64fx: |
Contributor
I would call that a spectacular success... That's literally 99.98% of tests passing, on a platform that we know is notoriously difficult to build stuff on. |
Collaborator
Author
|
I've updated the hook in EESSI/software-layer-scripts#181 to allow for 41 failing tests. |
Collaborator
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1 |
|
New job on instance
|
Contributor
@bedroge Can you attach the log, or at least the names of those failures? But yeah, we can consider this a success |
Collaborator
Author
Sure, here it is. |
Contributor
|
There are a couple weird failures. The 20 And for
That's an internal assert which I guess should never fail. If those failures do not occur without ACL we may consider removing it again. Or at least test the PyPI version to check if they fail too to reason on how serious those might be. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.