Skip to content

mutest: improved traceback printing#100

Open
liambrady wants to merge 1 commit into
LabNConsulting:mainfrom
liambrady:liambrady/mutest_print_exception
Open

mutest: improved traceback printing#100
liambrady wants to merge 1 commit into
LabNConsulting:mainfrom
liambrady:liambrady/mutest_print_exception

Conversation

@liambrady

Copy link
Copy Markdown
Contributor

Improves the reporting of tracebacks that originate from user-written scripts. Includes printing the top-level mutest file and a traceback explicitely scoped to only the failing script file.

(Example) Before:

$ sudo -E mutest
 INFO: NUMBER     STAT TARGET   TIME  DESCRIPTION
 INFO: ----------------------------------------------------------------------
 INFO: 
 INFO: 1. Test from file: mutest_0.py
 INFO: 1.1        PASS         0.001  Independent successful mutest
 INFO: 
 INFO: 2. Test from file: mutest_1.py
 INFO: 2.1        PASS         0.001  Step before include
 INFO: 
 INFO: 2.2. Test from file: sub_mutest_1.py
 INFO: 2.2.1      PASS        0.0007  Step before assert failure
ERROR: Unexpected exception executing sub_mutest_12_2: 
Traceback (most recent call last):
  File "/home/lbrady/munet/munet/mutest/userapi.py", line 364, in __exec_script
    result = locals()[f"_{name}"](_ok_result)
  File "<string>", line 5, in _sub_mutest_12_2
AssertionError
 INFO: 
 INFO: 3. Test from file: mutest_2.py
 INFO: 3.1        PASS         0.001  Step before assert failure
ERROR: Unexpected exception executing mutest_23: 
Traceback (most recent call last):
  File "/home/lbrady/munet/munet/mutest/userapi.py", line 364, in __exec_script
    result = locals()[f"_{name}"](_ok_result)
  File "<string>", line 5, in _mutest_23
AssertionError
 INFO: 
 INFO: run stats: 4 steps, 4 pass, 0 fail, 2 abort, 2.44s elapsed
 INFO: ----------------------------------------------------------------------
 INFO:  PASS  1:mutest_0
 WARN:  FAIL  2:mutest_1
 WARN:  FAIL  3:mutest_2
 INFO: ----------------------------------------------------------------------
 INFO: END RUN: 3 test scripts, 1 passed, 2 failed

After:

$ sudo -E mutest
 INFO: NUMBER     STAT TARGET   TIME  DESCRIPTION
 INFO: ----------------------------------------------------------------------
 INFO: 
 INFO: 1. Test from file: mutest_0.py
 INFO: 1.1        PASS         0.001  Independent successful mutest
 INFO: 
 INFO: 2. Test from file: mutest_1.py
 INFO: 2.1        PASS         0.001  Step before include
 INFO: 
 INFO: 2.2. Test from file: sub_mutest_1.py
 INFO: 2.2.1      PASS        0.0008  Step before assert failure
ERROR: Script error executing test /home/lbrady/debug/mutest_exception/mutest_1.py: 
Traceback (most recent call last):
  File "/home/lbrady/munet/debug/mutest_exception/sub_mutest_1.py", line 4, in _sub_mutest_12_2
    assert 0 == 1
AssertionError
 INFO: 
 INFO: 3. Test from file: mutest_2.py
 INFO: 3.1        PASS         0.001  Step before assert failure
ERROR: Script error executing test /home/lbrady/debug/mutest_exception/mutest_2.py: 
Traceback (most recent call last):
  File "/home/lbrady/munet/debug/mutest_exception/mutest_2.py", line 4, in _mutest_23
    assert 0 == 2
AssertionError
 INFO: 
 INFO: run stats: 4 steps, 4 pass, 0 fail, 2 abort, 2.39s elapsed
 INFO: ----------------------------------------------------------------------
 INFO:  PASS  1:mutest_0
 WARN:  FAIL  2:mutest_1
 WARN:  FAIL  3:mutest_2
 INFO: ----------------------------------------------------------------------
 INFO: END RUN: 3 test scripts, 1 passed, 2 failed

@liambrady
liambrady requested a review from choppsv1 July 8, 2026 20:34
@liambrady liambrady added the enhancement New feature or request label Jul 8, 2026
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.61%. Comparing base (e565ff6) to head (e6c7444).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
+ Coverage   60.48%   60.61%   +0.12%     
==========================================
  Files          19       19              
  Lines        5988     5974      -14     
==========================================
- Hits         3622     3621       -1     
+ Misses       2366     2353      -13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@liambrady
liambrady force-pushed the liambrady/mutest_print_exception branch 2 times, most recently from c508db7 to 048559b Compare July 10, 2026 18:28
Comment thread munet/mutest/userapi.py
self.oplogf("include: restored info path: %s", old_path)

if do_cli:
raise CLIOnErrorError()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how is the CLI on error handled since this is being removed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is re-raised on line 706

Improves the reporting of tracebacks that originate from user-written
scripts. Includes printing the top-level mutest file and a traceback
explicitely scoped to only the failing script file.

Error paths manually tested:
* generic exception in main mutest file
* generic exception in include()'ed file
* generic exception in nested include()'ed file
* generic exception in main mutest file after multiple include()'s
* generic exception in external library code
* invalid mutest API usage exception in external library code
* CLIOnErrorError exception (should not be printed)
@liambrady
liambrady force-pushed the liambrady/mutest_print_exception branch from 048559b to e6c7444 Compare July 22, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants