Skip to content

Add TimingContext to measure execution time#1421

Open
dalonsoa wants to merge 3 commits into
mainfrom
527_timeit
Open

Add TimingContext to measure execution time#1421
dalonsoa wants to merge 3 commits into
mainfrom
527_timeit

Conversation

@dalonsoa

@dalonsoa dalonsoa commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Description

Uses a context manager, in a pythonic style, to decorate functions whose execution time we want to measure. The result is provided as an info level log entry. For now, only run_dispatch_for_year and perform_agent_investment are decorated, but other functions could be included in the future.

Having said that, if the decorated functions are nested, the output might be confusing, so if that were a use case, we might need to re-think what we output in the logs for it to be useful.

The output of the simple model now looks like (note the [Timer] entries):

[14:17:35 INFO muse2::cli] Starting MUSE2 v2.1.0
[14:17:35 INFO muse2::cli] Loaded model from /home/dalonsoa/Projects/MUSE2/examples/simple
[14:17:35 INFO muse2::cli] Output folder: muse2_results/simple
[14:17:35 WARN muse2::cli] Output folder will be overwritten
[14:17:35 INFO muse2::simulation] Milestone year: 2020
[14:17:35 INFO muse2::simulation] Running dispatch optimisation...
[14:17:35 INFO muse2::timeit] [Timer] 'run_dispatch_for_year' completed in 7.095ms
[14:17:35 INFO muse2::simulation] Milestone year: 2030
[14:17:35 INFO muse2::simulation] Running agent investment...
[14:17:35 INFO muse2::timeit] [Timer] 'perform_agent_investment' completed in 36.010ms
[14:17:35 INFO muse2::simulation] Running dispatch optimisation...
[14:17:35 INFO muse2::timeit] [Timer] 'run_dispatch_for_year' completed in 5.892ms
[14:17:35 INFO muse2::simulation] Max ironing out iterations (1) reached
[14:17:35 INFO muse2::simulation] Running final dispatch optimisation for year 2030...
[14:17:35 INFO muse2::timeit] [Timer] 'run_dispatch_for_year' completed in 5.755ms
[14:17:35 INFO muse2::simulation] Milestone year: 2040
[14:17:35 INFO muse2::simulation] Running agent investment...
[14:17:35 INFO muse2::timeit] [Timer] 'perform_agent_investment' completed in 59.618ms
[14:17:35 INFO muse2::simulation] Running dispatch optimisation...
[14:17:35 INFO muse2::timeit] [Timer] 'run_dispatch_for_year' completed in 5.669ms
[14:17:35 INFO muse2::simulation] Max ironing out iterations (1) reached
[14:17:35 INFO muse2::simulation] Running final dispatch optimisation for year 2040...
[14:17:35 INFO muse2::timeit] [Timer] 'run_dispatch_for_year' completed in 2.388ms
[14:17:35 INFO muse2::cli] Simulation complete!

Fixes #527

Type of change

  • Bug fix (non-breaking change to fix an issue)
  • New feature (non-breaking change to add functionality)
  • Refactoring (non-breaking, non-functional change to improve maintainability)
  • Optimization (non-breaking change to speed up the code)
  • Breaking change (whatever its nature)
  • Documentation (improve or add documentation)

Key checklist

  • All tests pass: $ cargo test
  • The documentation builds and looks OK: $ cargo doc
  • Update release notes for the latest release if this PR adds a new feature or fixes a bug
    present in the previous release

Further checks

  • Code is commented, particularly in hard-to-understand areas
  • Tests added that prove fix is effective or that feature works

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.82%. Comparing base (9f3e1fe) to head (6c8541a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1421      +/-   ##
==========================================
+ Coverage   86.80%   86.82%   +0.01%     
==========================================
  Files          59       60       +1     
  Lines        8384     8395      +11     
  Branches     8384     8395      +11     
==========================================
+ Hits         7278     7289      +11     
  Misses        790      790              
  Partials      316      316              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tsmbland tsmbland left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice. Very minimal and non-invasive so I'm ok with this. Some people may have reservations about adding another dependency? It looks very minimal so possibly something we could write a tailored version of ourselves.

For now I think the two functions you've chosen are the most important to monitor. Long run we may also want look at investments running time each individual commodity.

Maybe should be debug level, since the log messages are referring to function names that normal users shouldn't have to concern themselves with? (Or make the messages more user facing)

I think this is fine for eyeballing, but would be quite fiddly to do any kind of analytics on this. Is there any way to get extra context into the timer messages (e.g. year)? I guess you can look at previous messages to figure out what it's referring to, but that's a bit fiddly.

Interested in @alexdewar's opinion so I'm going to hold off approving until he can have a look

@dc2917 dc2917 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neat, and all looks good as far as I'm concerned.

As Tom says, I think it might be useful to have more descriptive messages about what exactly ran in a given amount of time, rather than the function names.

Also not sure whether microsecond precision is necessary, but is harmless anyway.

@dalonsoa

Copy link
Copy Markdown
Collaborator Author

There are a couple of different options here, depending on how much customisation we want in the logged information, how many different functions are timed and how intrusive we are willing to be.

Option 1

If we want to be not intrusive - as it is now -, we do not have many functions, and we want full customisation of the information logged, we could just create a custom context per function to time. Eg. InvestmentContext, DispatchContext, etc. We are only using at the moment the after hook, but there is also a before hook which is run just before the function. Both have access to the function name and all its input arguments, and the after hook also to the result, so there's a lot of flexibility. Some of these could use the info log level, if they are high level functions, and other, deeper ones debug, for example, or even save the execution time information to a file. This would be just one step short of modifying the existing code of the functions and put all of this in there.

Option 2

If we want to keep things generic, there's an option I was exploring - with my buddy Claude 😆 - that uses a small macro to customise the message and that is used at the calling point. Something like:

#[context_manager::wrap(TimingContext)]
fn query_database(id: u32) -> String {
    // Do something here
}

// and then at the calling point...
let record = timed!("Querying database", query_database(42));

Resulting in [Timer] Querying database completed in 20.341ms. But I do not really like macros, it is more intrusive and does not offer a huge amount of flexibility besides a custom string.

Option 3

Any other option I feel would require coding our own context_manager solution, which is not a big deal, but I thought it would be best to make sure that we do need that extra step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Output information about time taken for program run

3 participants