Track usage and estimate cost for calls

[jnu]

Tracking API usage from responses, and use it to estimate API costs. Both usage and cost estimation must be requested and are not enabled by default on pipeline runs.

These data are not intended to be used in production pipeline runs (since cost is much more accurate and easier to track in the production deployments on Azure), but rather intended to be used to form benchmarks to inform upgrades, particularly model version upgrades.

[sentry]

Bug: _select_unique_price can return a non-deterministic meter when multiple Azure items share the same price, as it returns matches[0] without a stable sort.
_{Severity: LOW}

Suggested Fix

To ensure deterministic results, either modify the OData filter to include isPrimaryMeterRegion eq true when querying the Azure Retail Prices API, which would select the primary meter. Alternatively, if multiple meters must be considered, apply a deterministic sort to the matches list based on a stable key (e.g., meterName) before returning matches[0]. This will guarantee consistent metadata in the cost estimation reports.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: bc2/core/common/azure_pricing.py#L229-L232

Potential issue: The `_select_unique_price` function may exhibit non-deterministic
behavior. When the Azure Retail Prices API returns multiple items with the same price
and unit of measure but different `meterName` values, the function returns the first
element (`matches[0]`) from a list derived from a set. The order is not guaranteed,
leading to inconsistent `meter_name` metadata in the cost estimate report across
different runs for the same operation. While the estimated cost remains correct, the
associated metadata becomes arbitrary, which can cause confusion in reporting and
auditing for benchmarking purposes. The code does not filter for the primary meter
region, which is a common cause for such duplicate-priced items from the API.

_{Did we get this right? 👍 / 👎 to inform future reviews.}