Add Booster model_type and parquet dataframes to ds_models - #29
Conversation
Serving support for the rank-prediction models: raw xgboost.Booster predict via categorical-enabled DMatrix, and parquet s3_dataframe artifacts for large lookup tables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR extends the ds_models S3-backed model loaders to support additional XGBoost artifact types (native Booster) and parquet-backed pandas DataFrames, enabling serving larger lookup tables and non-sklearn XGBoost models through the coach.
Changes:
- Add
model_type: Boostersupport tos3_xgboostusingxgboost.Booster+DMatrix(..., enable_categorical=True)for prediction. - Add
application/x-parquetsupport tos3_dataframeviapd.read_parqueton S3 object bytes. - Add/extend tests covering Booster prediction round-trip and parquet DataFrame round-trip; update changelog.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pureskillgg_dsdk/ds_models/s3_xgboost.py | Adds XGBoost Booster loading/prediction path and centralizes xgboost import handling. |
| pureskillgg_dsdk/ds_models/s3_xgboost_test.py | Adds a Booster-based test case (incl. categorical feature) and adjusts test scaffolding. |
| pureskillgg_dsdk/ds_models/s3_dataframe.py | Adds parquet loading support for S3 DataFrame artifacts. |
| pureskillgg_dsdk/ds_models/s3_dataframe_test.py | New test validating parquet DataFrame round-trip through s3_dataframe. |
| CHANGELOG.md | Documents the new Booster and parquet capabilities. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if self._model_type == "Booster": | ||
| model = xgboost.Booster() | ||
| else: | ||
| model = xgboost.XGBClassifier() | ||
| model.load_model(bytearray(body)) |
| predictions = ds_model.invoke(frame) | ||
| expected = trained.predict(xgboost.DMatrix(frame, enable_categorical=True)) | ||
| np.testing.assert_array_equal(predictions, expected) |
| The format is based on [Keep a Changelog](https://keepachangelog.com/) | ||
| and this project adheres to [Semantic Versioning](https://semver.org/). | ||
|
|
||
| ## 3.2.0 |
| The format is based on [Keep a Changelog](https://keepachangelog.com/) | ||
| and this project adheres to [Semantic Versioning](https://semver.org/). | ||
|
|
||
| ## 3.2.0 |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Addressed the review: unknown |
Why
Shipping predicted FACEIT level / Premier rating to the scoreboard needs three XGBoost artifacts served by the coach: a 10-class softprob classifier, a censored
survival:aftbooster, and a percentile-target regressor — plus several parquet reference tables (llr grids, headshot-angle lookup, death-danger percentiles) too large for CSV.What
s3_xgboostacceptsmodel_type: Booster: loads withxgboost.Boosterand predicts throughDMatrix(..., enable_categorical=True)— one code path for regressors, AFT boosters, and softprob classifiers, including models with pandas categorical features. ExistingXGBClassifierpath untouched.s3_dataframeacceptsres_type: application/x-parquet.Reactor's models.yaml schema already takes
model_type/res_typeas free strings, so no schema pair needed this time.Tests
🤖 Generated with Claude Code