Data CSV Dump Attribute

[electronsandstuff]

This PR adds a new attribute data_dump_file to Xopt which will automatically dump Xopt.data to a user specified CSV file while running. This avoids users having to parse the YAML file generated from dump_file and extracting their data. Full list of changes:

• New attribute data_dump_file for a CSV file dumped during .run()
• Rename dump_file to xopt_dump_file to avoid ambiguous name with new attribute.
• Add legacy support for old name with deprecation warnings to update
  • Before validator for existing config files
  • Properties for user code
• Add environment variable expansion in data_dump_file and xopt_dump_file
• Add tests for new property, legacy support, environment variable expansion

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[nikitakuklev]

No objections in general.

• Not a fan of the rename + shim because of how many saves it affects. Would keeping dump_file + adding data_dump_file work?

• In general, .data can contain whatever the evaluators return, including dicts - ok via YAML, but will get dumped as strings in CSV. Is that intended behavior? (probably yes)

Couple Claude-inspired-nits:

Silent behavior change in dump(). It previously raised ValueError when no file was specified; now it silently no-ops (and evaluate_data relies on that by calling it unconditionally).

adding xopt_dump_file to an old dumped config (which always contains dump_file: null) raises a ValidationError, precisely when the user follows the deprecation message's advice. The conflict check at xopt/base.py:267 should only fire when data["dump_file"] is not None.

[electronsandstuff]

because of how many saves it affects

Sorry, could you explain what you mean?

Personally, I think having a .dump_file and .data_dump_file is confusing and I prefer the unambiguous names. With the before validator and properties, I think all user code / save files should keep working as-is. The intent would be to keep it that way for a long time with .xopt_dump_file being the official internal name in documentation to avoid having a .dump_file and .data_dump_file that seem to be the same thing.

In general, .data can contain whatever the evaluators return, including dicts

Yeah, I don't see a good way of getting around this, but this appears on the user side of whatever the evaluator function returns, so keep as-is.