Add qartod and config

[madrichardson]

No description provided.

[madrichardson]

I added qartod.py inside the esdglider folder, and I added qartod-config.yml in esdglider/data. Also, I messed up my branch name because I've been working with Quarto a lot, so the branch should've been qartod-module, but instead I accidentally named it quarto-module.

[smwoodman]

Please remove this file from the pr

[smwoodman]

I use xr.load_dataset(input_file) rather than these three lines

[madrichardson]

Ya that's a better approach, I'll update it.

[smwoodman]

Hi Madi - thanks for this! I've put some comments inline. A couple general ones, happy to discuss any of these:

For log messages, please use the syntax _log.info("QCing %s", var), rather than f-strings. I haven't run it in a while, but this comes from the pre-commit hooks that I (intermittently) use. I definitely need to pick a specific style guide, and have CoPilot go through and make this consistent :)

For me, having extra blank lines, eg between function calls makes the code less easily readable (example below). For esdglider, I'd like to stick to the principles of a) generally keeping code on one line is the line length is <80, and b) not having extra blank lines between lines in the same function.

# Please change 
final_flags = np.maximum.reduce(

    test_results

).astype("int8")

# to either:
final_flags = np.maximum.reduce(test_results).astype("int8")

# or:
final_flags = np.maximum.reduce(
    test_results
).astype("int8")

[madrichardson]

Hi Sam,

I'll make sure to fix all of these!

[madrichardson]

Hi Madi - thanks for this! I've put some comments inline. A couple general ones, happy to discuss any of these:

For log messages, please use the syntax _log.info("QCing %s", var), rather than f-strings. I haven't run it in a while, but this comes from the pre-commit hooks that I (intermittently) use. I definitely need to pick a specific style guide, and have CoPilot go through and make this consistent :)

For me, having extra blank lines, eg between function calls makes the code less easily readable (example below). For esdglider, I'd like to stick to the principles of a) generally keeping code on one line is the line length is <80, and b) not having extra blank lines between lines in the same function.

# Please change 
final_flags = np.maximum.reduce(

    test_results

).astype("int8")

# to either:
final_flags = np.maximum.reduce(test_results).astype("int8")

# or:
final_flags = np.maximum.reduce(
    test_results
).astype("int8")

Do you want me to remove the comments in the code (example below)?

EVALUATE ALL DATA VARIABLES

for var in ds.data_vars:
    # REQUIRE TIME DIMENSION
    if "time" not in ds[var].dims:
        continue
    # SKIP EXISTING QC VARIABLES
    if var.endswith("_qc"):
        continue
    # SKIP EXCLUDED VARIABLES
    #
    # Exclude metadata variables and redundant
    # coordinate variables that should not
    # receive QARTOD testing.
    if var in skip_variables:
        continue
    time_variables.append(var)
# LOG SUMMARY
_log.info(
    "Found %d time variables for QC",
    len(time_variables)
)
return time_variables

[smwoodman]

No, I think the comments are overall good, although I typically don't like them over logs because (unless it's really complicated) I think the log should be self-explanatory. I would format that block something like:

for var in ds.data_vars:
    # REQUIRE TIME DIMENSION
    if "time" not in ds[var].dims:
        continue

    # SKIP EXISTING QC VARIABLES AND EXCLUDED VARIABLES
    # {SMW note: I think fine to add more context here, if you feel it's necessary}
    if var.endswith("_qc") or var in skip_variables:
        continue

    time_variables.append(var)

_log.info(
    "Found %d time variables for QC",
    len(time_variables)
)

return time_variables

Thanks!

[madrichardson]

I updated everything based on your comments. If there's anything else, please let me know!

[smwoodman]

Thank you! This is looking good. I'll try out the fork tomorrow.

One other ask - could you add a notebook (to the 'notebooks' folder) that shows a representative example or two of using this functionality?