Implement updates tracking OWID grapher#6
Open
xrendan wants to merge 1581 commits into
Open
Conversation
Adds support for dumbbell charts in search.
This includes:
- Constructing the data table rendered in search
- Making sure the big data value on the right is appropriate
- Should say `Start value -> End value` in time-range mode (same as slope charts)
- Otherwise, it should be hidden (we never show a value for multi-y charts)
The thumbnail PNGs (with imMinimal=0/1) look good as is, I think.
This is a bit difficult to test. I mainly looked at the various endpoints (`.search-result.json`, `values.json`) and added tests.
This PR also includes a refactor that drops the dumbbell's series strategy and replaces it with a custom `mode`. If the series strategy is set to `column`, search expects the series items to refer to columns, not entities. But dumbbell charts always plot entities, so the appropriate series strategy is `entity`, and we should differentiate between time-range and two-column mode separately.
### Follow-up work
* ~Dumbbell elements~
* ~New dumbbell-specific options: value labels, end points~
* ~In-chart legend, aligned with the top-most dots~
* ~Interaction (focus, hover, tooltips)~
* Color (hard-coded for now)
* More entity sorting options
* ~Dumbbell plots in search~ (addressed in this PR)
* ~Dumbbell chart thumbnails~ (addressed in this PR)
Adds three more sorting options for dumbbell plots: by start and end value, and by change. Also refactors the sorting code a bit: Chart types now expose `availableSortKeys`, which is then used in the admin to construct dropdown options, and to narrow the type for the object specifying the sorting options. Marimekkos are excluded from the refactor because their sorting is intermingled with other code, and sooner or later we'll rewrite the Marimekko chart component anyway. ### Follow-up work * ~Dumbbell elements~ * ~New dumbbell-specific options: value labels, end points~ * ~In-chart legend, aligned with the top-most dots~ * ~Interaction (focus, hover, tooltips)~ * Color (hard-coded for now) * ~More entity sorting options~ * ~Dumbbell plots in search~ * ~Dumbbell chart thumbnails~
Allows color customisation for dumbbell charts.
* For two-column mode: Authors can assign an indicator colour via the admin
* For time-range mode: Authors can assign colours in the dumbbell section via the admin. These are mapped to the new `dumbbell.trendColorMap` config field.
* Instead of introducing a dumbbell-specific colour mapping, we could also re-use the top-level colorMap config field. But the `colorMap` is shared between all chart types, so if the dumbbell chart is shown next to a line chart, the line chart colour scheme would also apply to the dumbbell chart, which isn’t ideal
* Instead of introducing a new config field type, we could also re-use the existing colorMap type inside the dumbbell namespace. This feels like overkill though, and it’s also not a great fit because the internal ColorScale needs a color column, which doesn’t exist for dumbbells since there’s no column encoding increase/decrease/noChange. Of course, we could create one on the fly, but that seemed unnecessarily complex
* I also considered simply not allowing customisation, but at the very least, we’d need an “invert colors” config because mapping increase to green and decrease to red doesn’t work when “decrease is good”. So if we have to add one piece of config anyway, we might as well allow customisation
* I also considered not allowing customisation and simply choosing neutral colours for increase/decrease, but Marwa wasn’t a fan
Misc touchups for dumbbell charts after Marwa's feedback
Surface the existing type=Dumbbell embed test view on the admin test index page, alongside the other chart-type links.
Add dumbbell chart link to admin test index
Replace the hard-coded POPULATION_INDICATOR_ID_USED_IN_ADMIN and GDP_PER_CAPITA_INDICATOR_ID_USED_IN_ADMIN constants with catalog-path resolution, so the admin scatter defaults follow data updates instead of pointing at a soon-stale version. - Add getLatestVariableIdsByCatalogPath DB helper + a matching /api/variables/latestByCatalogPath.json endpoint that resolves version-agnostic catalog paths to the latest variable id. - Prefetch the ids in ChartEditorPage (variableIdsByCatalogPath) and read them in EditorBasicTab when applying scatter defaults. CONTINENTS_INDICATOR_ID stays hard-coded: it's also used at Grapher render time via isContinentsVariableId, which has no admin-API access.
…y-catalog-path 🔨 Resolve admin scatter default indicators by catalog path
… path When resolving a catalog path to the latest variable id, join with datasets and filter out archived ones, so admin scatter defaults never point at an indicator from a dataset that's no longer maintained.
The endpoint was registered at /api/variables/latestByCatalogPath.json, which
overlaps the /api/variables/:variableId.json param route — the request was
captured by getVariablesVariableIdJson, which tried expectInt("latestByCatalogPath")
and returned a 400.
Move it to the dot-style /api/variables.latestByCatalogPath.json (matching the
existing /api/variables.usages.json convention) so it can't be matched by the
:variableId route.
…g-path-lookup 🔨 Ignore archived datasets when resolving latest variable by catalog path
* 🐝 🤖 Clean up tsconfig base options Switch `module` from `commonjs` to `preserve` so it matches our `bundler` module resolution and the ESM reality of our toolchain — the emitted JS is never run (tsx and Vite consume the TS source directly), so the previous CommonJS setting was both inert and inconsistent. Drop options that are redundant with the inherited `@tsconfig/node24` base or with current TS defaults: - `sourceMap` — emitted .js.map files are never used; nothing runs or debugs the tsc output. - `alwaysStrict` — implied by the base's `strict: true`. - `forceConsistentCasingInFileNames`, `allowJs` — already the defaults. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Remove explorerJobsWorker examples using bare Node We only deploy it using tsx and supporting Node is not necessary. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Fix styling: it rendered unstyled at the bottom of the page because the SCSS is scoped to .GrapherComponent. Since we use it only on standalone Grapher pages, it can render inside Grapher instead. - Convert it to a plain function component. - Replace imperative toggle with React state. - Ensure all commands run in a MobX action
🐛 Restrict table-tab data download to the selected time range
Signed-off-by: dependabot[bot] <support@github.com>
Replace the entity dropdown's grouped Selected/Suggested/All layout with a single ordered list: any pinned aggregates first (e.g. 'All countries'), then the selected entity, then the user's country and continent, then the remaining entities alphabetically. The user's country and continent are marked with a location icon wherever they appear, including when one of them is selected.
Rename the whole-slideshow delete button/confirm to say 'Delete slideshow' and the single-slide delete action to say 'Delete slide' (with a tooltip/aria-label since it's icon-only), so the two destructive actions are unambiguous at the point of the click. Also strengthen the slideshow delete confirmation copy to make clear the whole deck is being permanently deleted.
Makes the destructive whole-slideshow delete action visually more distinct from the lightweight single-slide delete confirmation, in both the editor footer and the slideshows index page.
Authenticate staging admin via Tailscale Serve headers
Fix staging Serve admin auth through nginx
* 🎉🤖 Add optional license field to chart configs Charts default to CC BY, but a stricter Creative Commons license (e.g. CC BY-NC-ND) can now be set per chart when a data provider requires it. - Add optional "license" enum (all six CC licenses, default cc-by) to the grapher schema (non-breaking) and GrapherInterface, with a CHART_LICENSES name/URL map in @ourworldindata/types - Chart footer shows the configured license and links to its creativecommons.org page; the footer hover tooltip drops the "free to use, share, and adapt" wording for non-CC-BY licenses - Add a License dropdown to the chart editor's Text tab - Data pages: "Reuse this work" renders a provider-specific notice via a shared ChartLicenseNotice component (both old and new layouts) - JSON-LD on grapher and data pages emits the actual license URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * 🐛🤖 Address Codex review: show license without OWID logo, plumb into multi-dim pages - A configured non-default license now overrides the "Powered by ourworldindata.org" footer fallback on charts without the OWID logo, since provider-required license terms must always be shown - Expose license on DataPageDataV2 (from the view's grapher config) and pass it to MetadataSection on multi-dim data pages Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
✨ Order the bespoke entity dropdown by user location
Per mille (‰) is not widely understood, so display the net migration rate as a percentage instead. This is display-only: the stored data and simulation model stay in per-1,000 units, and values are converted (divide by 10) only at the axis, value editor, and formatted labels via a new `displayScale` config field. Display precision is kept separate from the stored per-1,000 precision via `displayDecimals`, so URL serialization and modified-vs-baseline detection are unaffected.
✨ Show net migration rate as % instead of ‰
Staging servers built with the `staging-bake` label run the deploy queue
against a Buildkite pipeline (grapher/etl-automated-staging-environment),
triggering a build for their own branch via the REST API. Those pipelines
are PR-triggered ("Build branches" is off), so an API build for a branch
is rejected with 422 "Branches have been disabled for this pipeline", which
the deploy queue then retries indefinitely (Sentry ADMIN-N4, 17k+ events).
- Send `ignore_pipeline_branch_filters: true` so the deliberate
API-triggered branch build is accepted regardless of the pipeline's
branch trigger/filter settings. Verified against the live pipeline: the
same payload returns 422 without the flag and 201 with it.
- Include the pipeline slug, branch, and an explanation in the thrown
error so this is self-diagnosing next time instead of a bare `Error: 422`.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
For single-y-indicator charts and multi-dim views, index the y-indicator's titleVariant and display it (italic, sentence-cased) as a prefix to the subtitle, disambiguating otherwise identical-looking results. For #5800
✨🤖 (search) show indicator titleVariant in search results
…6641) * 🔨 Rename WID regions: MENA → Middle East and North Africa; & → and Match the ETL regions rename (owid/etl#6314): - 'MENA (WID)' → 'Middle East and North Africa (WID)' - 'South & South-East Asia (WID)' → 'South and South-East Asia (WID)' Updates the hand-maintained name-keyed registries (CustomSchemes ContinentColors color pins, RegionTooltipData customRegionDisplayOrder) and the two region names in the auto-generated regions.data.ts (stopgap — to be reconciled via yarn runRegionsUpdater once etl#6314 is live on the prod catalog). 🤖 Generated with [Claude Code](https://claude.com/claude-code) * 🔨 Regenerate regions.data.ts from prod for WID rename Ran runRegionsUpdater (via tsx) now that the WID region rename is live on the prod catalog. Fixes what the earlier hand-edit missed: adds MENA shortName, and regenerates both slugs (middle-east-and-north-africa-wid, south-and-south-east-asia-wid). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Labeling a PR via the issues API requires pull-requests: write on the GITHUB_TOKEN; issues: write alone yields a 403 (Resource not accessible by integration). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🐛🤖 grant pull-requests write permission to staging-viz labeler job
🐛 (discrete bar) fix bar placement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Links to issues, Figma, Slack, and a technical introduction to the work.
Screenshots / Videos / Diagrams
Add if relevant, i.e. might not be necessary when there are no UI changes.
Testing guidance
Step-by-step instructions on how to test this change
Reminder to annotate the PR diff with design notes, alternatives you considered, and any other helpful context.
Checklist
(delete all that do not apply)
Before merging
If DB migrations exists:
After merging