Skip to content
12 changes: 6 additions & 6 deletions sdk/ai/azure-ai-inference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ To report an issue with the client library, or request additional features, plea
* For Serverless API endpoints or Managed Compute endpoints
* An [Azure subscription](https://azure.microsoft.com/free).
* An [AI Model from the catalog](https://ai.azure.com/explore/models) deployed through Azure AI Foundry.
* The endpoint URL of your model, in of the form `https://<your-host-name>.<your-azure-region>.models.ai.azure.com`, where `your-host-name` is your unique model deployment host name and `your-azure-region` is the Azure region where the model is deployed (e.g. `eastus2`).
* The endpoint URL of your model, in the form `https://<your-host-name>.<your-azure-region>.models.ai.azure.com`, where `your-host-name` is your unique model deployment host name and `your-azure-region` is the Azure region where the model is deployed (e.g. `eastus2`).
* Depending on your authentication preference, you either need an API key to authenticate against the service, or Entra ID credentials.
* For Azure OpenAI (AOAI) service
* An [Azure subscription](https://azure.microsoft.com/free).
* An [OpenAI Model from the catalog](https://oai.azure.com/resource/models) deployed through Azure AI Foundry.
* The endpoint URL of your model, in the form `https://<your-resouce-name>.openai.azure.com/openai/deployments/<your-deployment-name>`, where `your-resource-name` is your globally unique AOAI resource name, and `your-deployment-name` is your AI Model deployment name.
* The endpoint URL of your model, in the form `https://<your-resource-name>.openai.azure.com/openai/deployments/<your-deployment-name>`, where `your-resource-name` is your globally unique AOAI resource name, and `your-deployment-name` is your AI Model deployment name.
* Depending on your authentication preference, you either need an API key to authenticate against the service, or Entra ID credentials.
* An api-version. Latest preview or GA version listed in the `Data plane - inference` row in [the API Specs table](https://aka.ms/azsdk/azure-ai-inference/azure-openai-api-versions). At the time of writing, latest GA version was "2024-06-01".

Expand Down Expand Up @@ -91,7 +91,7 @@ client = ChatCompletionsClient(

# For Azure OpenAI endpoint
client = ChatCompletionsClient(
endpoint=endpoint, # Of the form https://<your-resouce-name>.openai.azure.com/openai/deployments/<your-deployment-name>
endpoint=endpoint, # Of the form https://<your-resource-name>.openai.azure.com/openai/deployments/<your-deployment-name>
credential=AzureKeyCredential(key),
api_version="2024-06-01", # Azure OpenAI api-version. See https://aka.ms/azsdk/azure-ai-inference/azure-openai-api-versions
)
Expand Down Expand Up @@ -197,7 +197,7 @@ Entra ID authentication is also supported by the `load_client` function. Replace

### Get AI model information

If you are using Serverless API or Managed Compute endpoints, you can call the client method `get_model_info` to retrive AI model information. This makes a REST call to the `/info` route on the provided endpoint, as documented in [the REST API reference](https://learn.microsoft.com/azure/ai-studio/reference/reference-model-inference-info). This call will not work for GitHub Models or Azure OpenAI endpoints.
If you are using Serverless API or Managed Compute endpoints, you can call the client method `get_model_info` to retrieve AI model information. This makes a REST call to the `/info` route on the provided endpoint, as documented in [the REST API reference](https://learn.microsoft.com/azure/ai-studio/reference/reference-model-inference-info). This call will not work for GitHub Models or Azure OpenAI endpoints.

<!-- SNIPPET:sample_get_model_info.get_model_info -->

Expand All @@ -211,7 +211,7 @@ print(f"Model type: {model_info.model_type}")

<!-- END SNIPPET -->

AI model information is cached in the client, and futher calls to `get_model_info` will access the cached value and wil not result in a REST API call. Note that if you created the client using `load_client` function, model information will already be cached in the client.
AI model information is cached in the client, and further calls to `get_model_info` will access the cached value and will not result in a REST API call. Note that if you created the client using the `load_client` function, model information will already be cached in the client.

AI model information is displayed (if available) when you `print(client)`.

Expand Down Expand Up @@ -245,7 +245,7 @@ In the following sections you will find simple examples of:
* [Image Embeddings](#image-embeddings-example)

The examples create a synchronous client assuming a Serverless API or Managed Compute endpoint. Modify client
construction code as descirbed in [Key concepts](#key-concepts) to have it work with GitHub Models endpoint or Azure OpenAI
construction code as described in [Key concepts](#key-concepts) to have it work with GitHub Models endpoint or Azure OpenAI
endpoint. Only mandatory input settings are shown for simplicity.

See the [Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/ai/azure-ai-inference/samples) folder for full working samples for synchronous and asynchronous clients.
Expand Down
18 changes: 9 additions & 9 deletions sdk/ai/azure-ai-inference/samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ These are runnable console Python scripts that show how to do chat completion an

Samples with `azure_openai` in their name show how to do chat completions and text embeddings against Azure OpenAI endpoints.

Samples in this folder use the a synchronous clients. Samples in the subfolder `async_samples` use the asynchronous clients. The concepts are similar, you can easily modify any of the synchronous samples to asynchronous.
Samples in this folder use the synchronous clients. Samples in the subfolder `async_samples` use the asynchronous clients. The concepts are similar, so you can easily adapt any of the synchronous samples for asynchronous use.

## Prerequisites

Expand All @@ -32,7 +32,7 @@ See [Prerequisites](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/
```bash
pip install --upgrade azure-ai-inference
```
* If you plan to run the asynchronous client samples, insall the additional package [aiohttp](https://pypi.org/project/aiohttp/):
* If you plan to run the asynchronous client samples, install the additional package [aiohttp](https://pypi.org/project/aiohttp/):
```bash
pip install aiohttp
```
Expand All @@ -41,10 +41,10 @@ See [Prerequisites](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/

To construct any of the clients, you will need to pass in the endpoint URL. If you are using key authentication, you also need to pass in the key associated with your deployed AI model.

* For Serverless API and Managed Compute endpoints, the endpoint URL has the form `https://your-unique-resouce-name.your-azure-region.models.ai.azure.com`, where `your-unique-resource-name` is your globally unique Azure resource name and `your-azure-region` is the Azure region where the model is deployed (e.g. `eastus2`).
* For Serverless API and Managed Compute endpoints, the endpoint URL has the form `https://your-unique-resource-name.your-azure-region.models.ai.azure.com`, where `your-unique-resource-name` is your globally unique Azure resource name and `your-azure-region` is the Azure region where the model is deployed (e.g. `eastus2`).
* For Managed Compute Endpoints, do not include the inference path (e.g. `/score`) in endpoint URL.

* For Azure OpenAI endpoints, the endpoint URL has the form `https://your-unique-resouce-name.openai.azure.com/openai/deployments/your-deployment-name`, where `your-unique-resource-name` is your globally unique Azure OpenAI resource name, and `your-deployment-name` is your AI Model deployment name.
* For Azure OpenAI endpoints, the endpoint URL has the form `https://your-unique-resource-name.openai.azure.com/openai/deployments/your-deployment-name`, where `your-unique-resource-name` is your globally unique Azure OpenAI resource name, and `your-deployment-name` is your AI Model deployment name.

For convenience, and to promote the practice of not hard-coding secrets in your source code, all samples here assume the endpoint URL and key are stored in environment variables. You will need to set these environment variables before running the samples as-is. The environment variables are mentioned in the tables below.

Expand All @@ -55,7 +55,7 @@ Note that the client library does not directly read these environment variable a
| Sample type | Endpoint environment variable name | Key environment variable name |
|----------|----------|----------|
| Chat completions | `AZURE_AI_CHAT_ENDPOINT` | `AZURE_AI_CHAT_KEY` |
| Test embeddings | `AZURE_AI_EMBEDDINGS_ENDPOINT` | `AZURE_AI_EMBEDDINGS_KEY` |
| Text embeddings | `AZURE_AI_EMBEDDINGS_ENDPOINT` | `AZURE_AI_EMBEDDINGS_KEY` |
| Image embeddings | `AZURE_AI_IMAGE_EMBEDDINGS_ENDPOINT` | `AZURE_AI_IMAGE_EMBEDDINGS_KEY` |


Expand Down Expand Up @@ -103,9 +103,9 @@ similarly for the other samples.
| [sample_chat_completions_from_input_dict_with_image_url.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_from_input_dict_with_image_url.py) | One chat completion operation using a synchronous client, with input messages provided as a dictionary (type `MutableMapping[str, Any]`). Includes sending an input image URL. |
| [sample_chat_completions_from_input_prompt_string.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_from_input_prompt_string.py) | One chat completion operation using a synchronous client, with input message template in string format. |
| [sample_chat_completions_from_input_prompty.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_from_input_prompty.py) | One chat completion operation using a synchronous client, with the input in Prompty format from a Prompty file. Prompty website: https://prompty.ai |
| [sample_chat_completions_with_tools.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_with_tools.py) | Shows how do use a tool (function) in chat completions, for an AI model that supports tools |
| [sample_chat_completions_streaming_with_tools.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_streaming_with_tools.py) | Shows how do use a tool (function) in chat completions, with streaming response, for an AI model that supports tools |
| [sample_load_client.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_load_client.py) | Shows how do use the function `load_client` to create the appropriate synchronous client based on the provided endpoint URL. In this example, it creates a synchronous `ChatCompletionsClient`. |
| [sample_chat_completions_with_tools.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_with_tools.py) | Shows how to use a tool (function) in chat completions, for an AI model that supports tools |
| [sample_chat_completions_streaming_with_tools.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_streaming_with_tools.py) | Shows how to use a tool (function) in chat completions, with streaming response, for an AI model that supports tools |
| [sample_load_client.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_load_client.py) | Shows how to use the function `load_client` to create the appropriate synchronous client based on the provided endpoint URL. In this example, it creates a synchronous `ChatCompletionsClient`. |
| [sample_get_model_info.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_get_model_info.py) | Get AI model information using the chat completions client. Similarly can be done with all other clients. |
| [sample_chat_completions_with_model_extras.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_with_model_extras.py) | Chat completions with additional model-specific parameters. |
| [sample_chat_completions_azure_openai.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_azure_openai.py) | Chat completions against Azure OpenAI endpoint. |
Expand Down Expand Up @@ -135,7 +135,7 @@ similarly for the other samples.
|----------------|-------------|
|[sample_chat_completions_streaming_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_chat_completions_streaming_async.py) | One chat completion operation using an asynchronous client and streaming response. |
|[sample_chat_completions_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_chat_completions_async.py) | One chat completion operation using an asynchronous client. |
|[sample_load_client_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_load_client_async.py) | Shows how do use the function `load_client` to create the appropriate asynchronous client based on the provided endpoint URL. In this example, it creates an asynchronous `ChatCompletionsClient`. |
|[sample_load_client_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_load_client_async.py) | Shows how to use the function `load_client` to create the appropriate asynchronous client based on the provided endpoint URL. In this example, it creates an asynchronous `ChatCompletionsClient`. |
|[sample_chat_completions_from_input_bytes_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_chat_completions_from_input_bytes_async.py) | One chat completion operation using an asynchronous client, with input messages provided as `IO[bytes]`. |
|[sample_chat_completions_from_input_dict_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_chat_completions_from_input_dict_async.py) | One chat completion operation using an asynchronous client, with input messages provided as a dictionary (type `MutableMapping[str, Any]`) |
|[sample_chat_completions_streaming_azure_openai_async.py](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/async_samples/sample_chat_completions_streaming_azure_openai_async.py) | One chat completion operation using an asynchronous client and streaming response against an Azure OpenAI endpoint |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
https://aka.ms/azsdk/azure-ai-inference/azure-openai-api-versions
3. Set one or two environment variables, depending on your authentication method:
* AZURE_OPENAI_CHAT_ENDPOINT - Your AOAI endpoint URL, with partial path, in the form
https://<your-unique-resouce-name>.openai.azure.com/openai/deployments/<your-deployment-name>
https://<your-unique-resource-name>.openai.azure.com/openai/deployments/<your-deployment-name>
where `your-unique-resource-name` is your globally unique AOAI resource name,
and `your-deployment-name` is your AI Model deployment name.
For example: https://your-unique-host.openai.azure.com/openai/deployments/gpt-4o
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

For use_azure_openai_endpoint = True, set these two environment variables before running the sample:
1) AZURE_OPENAI_CHAT_ENDPOINT - Your AOAI endpoint URL, with partial path, in the form
https://<your-unique-resouce-name>.openai.azure.com/openai/deployments/<your-deployment-name>
https://<your-unique-resource-name>.openai.azure.com/openai/deployments/<your-deployment-name>
where `your-unique-resource-name` is your globally unique AOAI resource name,
and `your-deployment-name` is your AI Model deployment name.
For example: https://your-unique-host.openai.azure.com/openai/deployments/gpt-4o
Expand Down Expand Up @@ -77,15 +77,15 @@ def get_flight_info(origin_city: str, destination_city: str):
destination_city (str): The destination city.

Returns:
str: The airline name, fight number, date and time of the next flight between the cities, in JSON format.
str: The airline name, flight number, date and time of the next flight between the cities, in JSON format.
"""
if origin_city == "Seattle" and destination_city == "Miami":
return json.dumps(
{"airline": "Delta", "flight_number": "DL123", "flight_date": "May 7th, 2024", "flight_time": "10:00AM"}
)
return json.dumps({"error": "No flights found between the cities"})

# Define a function 'tool' that the model can use to retrieves flight information
# Define a function 'tool' that the model can use to retrieve flight information
flight_info = ChatCompletionsToolDefinition(
function=FunctionDefinition(
name="get_flight_info",
Expand Down Expand Up @@ -120,8 +120,8 @@ def get_flight_info(origin_city: str, destination_city: str):

# Make a streaming chat completions call asking for flight information, while providing a tool to handle the request
messages = [
SystemMessage("You an assistant that helps users find flight information."),
UserMessage("What is the next flights from Seattle to Miami?"),
SystemMessage("You are an assistant that helps users find flight information."),
UserMessage("What is the next flight from Seattle to Miami?"),
]

response = client.complete(messages=messages, tools=[flight_info], stream=True)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
Set these two environment variables before running the sample:

1) AZURE_AI_CHAT_ENDPOINT - Your endpoint URL for model inference, in the form
"https://<your-ai-services-resouce-name>.services.ai.azure.com/models".
"https://<your-ai-services-resource-name>.services.ai.azure.com/models".
This URL is shown in the model card page in your AI Foundry project: Click on "Models + endpoints" tab,
then click on the relevant model name to open the model card page. Copy the URL up to and including the
"/models" route.
Expand Down
Loading
Loading