Add Dask Cloud Provider user agent to Nebius SDK

[complynx]

Summary

• prefix Nebius SDK requests with dask-cloudprovider/<version>
• require Nebius SDK 0.2.55 or newer, where user_agent_prefix is supported
• cover the configured prefix with a focused unit test

Testing

• pytest dask_cloudprovider/nebius/tests/test_nebius.py -q
• black --check dask_cloudprovider/nebius/instances.py dask_cloudprovider/nebius/tests/test_nebius.py setup.py
• flake8 dask_cloudprovider/nebius/instances.py dask_cloudprovider/nebius/tests/test_nebius.py setup.py

[complynx]

Failed tests due to pytest 9. Using pytest<9 will fix the tests.

[complynx]

@dbalabka will there be any action regarding this PR?

[dbalabka]

@complynx, I’ll review it sometime this week. Whenever you have a moment, could you please let me know if there are easy steps I can follow to test on Nebius cloud? Thanks so much!

[complynx]

If you have Nebius account and can run an integration test, the check in this comment can be done. If you don't have an account, you can rely on my word that it works and on the unit test included in the PR. At least, the requests are being sent with the correct UA, as I can see it.

You can try using the following (set your project ID):

nebius profile create --parent-id <PROJECT_ID>

git clone https://github.com/dask/dask-cloudprovider.git
cd dask-cloudprovider
git fetch origin pull/486/head:pr-486
git switch pr-486

python -m venv .venv
. .venv/bin/activate

python -m pip install -U pip
python -m pip install -e ".[nebius]" -r requirements_test.txt "pytest<9"

export DASK_CLOUDPROVIDER__NEBIUS__PROJECT_ID="<PROJECT_ID>"
export DASK_CLOUDPROVIDER__NEBIUS__TOKEN="$(nebius iam get-access-token)"

python -m pytest \
  dask_cloudprovider/nebius/tests/test_nebius.py::test_create_cluster \
  --create-external-resources \
  -q -s

After that script, make sure to check that no leftover disks or VMs remained.

To check current disks/vms/IPs:

echo "=== VMs ==="
nebius compute instance list \
  --parent-id "$NB_PROJECT_ID" \
  --all \
  --format table

echo "=== Disks ==="
nebius compute disk list \
  --parent-id "$NB_PROJECT_ID" \
  --all \
  --format table

echo "=== IP allocations ==="
nebius vpc allocation list \
  --parent-id "$NB_PROJECT_ID" \
  --all \
  --format table

Do this prior to the dask invocation and after, to compare. If any unwanted units remain present, use one of the following deletion calls:

nebius compute instance delete --id <VM_ID>
nebius compute disk delete --id <DISK_ID>
nebius vpc allocation delete --id <ALLOCATION_ID>