Skip to content

F1 Score, Recall, Precision + log loss and rocauc + EvaluatePrequentialDelayedRegression - #12

Open
danielnowakassis wants to merge 3 commits into
adaptive-machine-learning:capymoafrom
danielnowakassis:f1&pr&EvaluatePrequentialDelayed
Open

F1 Score, Recall, Precision + log loss and rocauc + EvaluatePrequentialDelayedRegression#12
danielnowakassis wants to merge 3 commits into
adaptive-machine-learning:capymoafrom
danielnowakassis:f1&pr&EvaluatePrequentialDelayed

Conversation

@danielnowakassis

@danielnowakassis danielnowakassis commented Jul 23, 2026

Copy link
Copy Markdown

Fixes F1 Score Recall and Precision handling:

  • Deals with classes not yet predicted (that might result in ? and should be treated as zero cases)
  • Differentiate macro (per class and shows when per class option is activated) and micro
  • results match sklearn

Adds log loss and roc auc to BasicClassificationPerformanceEvaluator

  • adds log loss (new metric), matches sklearn
  • roc auc is borrowed from Dariusz Brzezinski implementation in MOA.

Adds EvaluatePrequentialDelayedRegression

@danielnowakassis danielnowakassis changed the title F1 Score, Recall, Precision + EvaluatePrequentialDelayedRegression F1 Score, Recall, Precision + log loss and rocauc + EvaluatePrequentialDelayedRegression Jul 23, 2026
@hmgomes
hmgomes self-requested a review August 5, 2026 03:18
@hmgomes

hmgomes commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Thanks @danielnowakassis
I assigned myself to review it, should take 1 to 2 weeks from now due to workload

@danielnowakassis

Copy link
Copy Markdown
Author

Hi @hmgomes.

Ok, thank you.

I'm just fixing some details in my next pr, will do it soon.

My recommendation for priority order:

1 - Metrics and delayed regression (most simple commit)
2 - incremental hyperparameter tuning algorithms (5 new algorithms)
3 - iPFI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants