Run verify --tests in CI#644
Conversation
Adding verify --tests to CI runs it on all versions. This highlighted 2 missed problems: submitpackage is untested for v26 - Add `UNTESTED` label to the types table. abortprivatebroadcast test did not check the model - Fix the test.
|
I left in the existing |
satsfy
left a comment
satsfy
left a comment
There was a problem hiding this comment.
PR looking good aside from one CI difficulty.
| BITCOIND_DOWNLOAD_DIR: ${{ github.workspace }}/.cache/corepc/bitcoind | ||
| run: cd integration_test && cargo test --features=${{ matrix.version }},download | ||
| run: | | ||
| test_output="${{ runner.temp }}/integration-test-${{ matrix.version }}.out" |
There was a problem hiding this comment.
I did a test CI run on my branch, failing the new integration test with assert!(false). The test does not fail on the integration test; it keeps going to verify, which fails loudly because all subsequent outputs after the failed integration test were not produced and did not fail fast so they are all missing at verify time. The solution is simple:
| test_output="${{ runner.temp }}/integration-test-${{ matrix.version }}.out" | |
| set -o pipefail | |
| test_output="${{ runner.temp }}/integration-test-${{ matrix.version }}.out" |
There was a problem hiding this comment.
Thanks, I added the fix.
For every version integration_tests CI capture the output and run verify --tests with the output generated. Run in the same job as the tests since they are needed to generate the file.
jamillambert
force-pushed
the
0622-verify-tests-CI
branch
from
7ea4357 to
da9f017
Compare
satsfy
left a comment
satsfy
left a comment
There was a problem hiding this comment.
tACK da9f017
Ran the CI again, fails as expected now on the integration tests.
2 participants
Fix the two problems highlighted when adding
verify --teststo CI:submitpackageis untested for v26 - AddUNTESTEDlabel to the types table.abortprivatebroadcasttest did not check the model - Fix the test.Add
verify --teststo CI:For every version
integration_testscapture the output and runverify --testswith the output generated. Run in the same job as the tests since they are needed to generate the file.Closes #639