diff --git a/Instructions/Exercises/AI-03-Multi-stage-reasoning.md b/Instructions/Exercises/AI-03-Multi-stage-reasoning.md index 31723bb..21e1fa9 100644 --- a/Instructions/Exercises/AI-03-Multi-stage-reasoning.md +++ b/Instructions/Exercises/AI-03-Multi-stage-reasoning.md @@ -1,6 +1,6 @@ --- lab: - title: Multi-stage Reasoning with LangChain using Azure Databricks and Azure OpenAI + title: Multi-stage Reasoning with LangChain using Azure Databricks and Microsoft Foundry description: You'll gain hands-on experience building sophisticated AI applications with LangChain by creating vector indexes with embeddings, implementing retriever-based chains for question-answering systems, and combining multiple chains into a multi-stage reasoning system. You'll learn how to chain LangChain components together to handle complex text processing tasks, such as retrieving relevant context from documents and then transforming that output into different formats like social media posts. duration: 30 minutes level: 400 @@ -11,7 +11,7 @@ lab: - Microsoft Foundry --- -# Multi-stage Reasoning with LangChain using Azure Databricks and Azure OpenAI +# Multi-stage Reasoning with LangChain using Azure Databricks and Microsoft Foundry Multi-stage reasoning is a cutting-edge approach in AI that involves breaking down complex problems into smaller, more manageable stages. LangChain, a software framework, facilitates the creation of applications that leverage large language models (LLMs). When integrated with Azure Databricks, LangChain allows for seamless data loading, model wrapping, and the development of sophisticated AI agents. This combination is particularly powerful for handling intricate tasks that require a deep understanding of context and the ability to reason across multiple steps. @@ -23,91 +23,120 @@ This lab will take approximately **30** minutes to complete. You'll need an [Azure subscription](https://azure.microsoft.com/free) in which you have administrative-level access. -## Provision an Azure OpenAI resource +## Create a Microsoft Foundry resource and project -If you don't already have one, provision an Azure OpenAI resource in your Azure subscription. +If you don't already have one, create a Microsoft Foundry resource and project in your Azure subscription. + +> **Note**: Creating a Foundry resource only requires a subscription, resource group, region, and name. No Key Vault or Application Insights resources are needed. 1. Sign into the **Azure portal** at `https://portal.azure.com`. -1. Create an **Azure OpenAI** resource with the following settings: - - **Subscription**: *Select an Azure subscription that has been approved for access to the Azure OpenAI service* +2. Use the following link to open the Foundry resource creation page: `https://portal.azure.com/#create/Microsoft.CognitiveServicesAIFoundry` +3. On the **Create** page, provide the following information on the **Basics** tab: + - **Subscription**: *Select your Azure subscription* - **Resource group**: *Choose or create a resource group* + - **Name**: *A unique name of your choice* - **Region**: *Make a **random** choice from any of the following regions*\* - - Australia East - - Canada East - - East US - - East US 2 - - France Central - - Japan East - North Central US - Sweden Central - - Switzerland North - - UK South - - **Name**: *A unique name of your choice* - - **Pricing tier**: Standard S0 + - **Default project name**: *Leave the pre-filled default or enter a custom project name* +4. Select **Review + create**, then select **Create** and wait for deployment to complete. + +> \* Foundry resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. -> \* Azure OpenAI resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. +5. Once deployment completes, go to the deployed resource. In the left pane, under **Resource Management**, select **Keys and Endpoint**, then on the **Foundry** tab, copy the **API endpoint** — you will use it later in this exercise. -1. Wait for deployment to complete. Then go to the deployed Azure OpenAI resource in the Azure portal. +6. In the **Overview** page, select **Go to Foundry portal** to open your resource in the Foundry portal (or navigate directly to `https://ai.azure.com`). -1. In the left pane, under **Resource Management**, select **Keys and Endpoint**. +7. In **Microsoft Foundry**, select the project within your Foundry resource. A default project is created automatically — select it to open it. If no project exists, create one: + - Select **+ New project** in the left navigation. + - Enter a **Project name** and select **Create project**. + - Wait for the project to be created. + +8. In a new browser tab, return to the **Azure portal** at `https://portal.azure.com` and launch Cloud Shell. Run the following command to get a temporary authorization token for API calls. Copy the `accessToken` value and save it alongside the endpoint you copied in step 5. + + ```bash + az account get-access-token --resource https://cognitiveservices.azure.com + ``` -1. Copy the endpoint and one of the available keys as you will use it later in this exercise. + >**Note**: You only need to copy the `accessToken` field value and **not** the entire JSON output. ## Deploy the required models -Azure provides a web-based portal named **Microsoft Foundry**, that you can use to deploy, manage, and explore models. You'll start your exploration of Azure OpenAI by using Microsoft Foundry to deploy a model. +Microsoft Foundry allows you to deploy, manage, and explore models. > **Note**: As you use Microsoft Foundry, message boxes suggesting tasks for you to perform may be displayed. You can close these and follow the steps in this exercise. -1. In the Azure portal, on the **Overview** page for your Azure OpenAI resource, scroll down to the **Get Started** section and select the button to go to **Microsoft Foundry**. - -1. In Microsoft Foundry, in the pane on the left, select the **Deployments** page and view your existing model deployments. If you don't already have one, create a new deployment of the **gpt-4o** model with the following settings: - - **Deployment name**: *gpt-4o* - - **Deployment type**: Standard - - **Model version**: *Use default version* +1. In **Microsoft Foundry**, on the home page select **View deployments** (or select **Build** in the top navigation bar, then select **Deployments**). + +1. Select **Deploy** > **Deploy a base model**, search for and select **gpt-4.1**, then select **Deploy** > **Custom settings** to configure the deployment with the following settings: + - **Deployment name**: *gpt-4.1* + - **Deployment type**: Global Standard + - **Model version** *(under Model version settings)*: *2025-04-14* + - **Model version upgrade policy** *(under Model version settings)*: Upgrade once new default version becomes available - **Tokens per minute rate limit**: 10K\* - - **Content filter**: Default - - **Enable dynamic quota**: Disabled - -1. Go back to the **Deployments** page and create a new deployment of the **text-embedding-ada-002** model with the following settings: + - **Guardrails**: DefaultV2 + + Then select **Deploy** at the bottom of the page. + +1. Return to the **Deployments** page and create a new deployment of the **text-embedding-ada-002** model with the following settings: - **Deployment name**: *text-embedding-ada-002* - - **Deployment type**: Standard - - **Model version**: *Use default version* + - **Deployment type**: Global Standard - **Tokens per minute rate limit**: 10K\* - - **Content filter**: Default - - **Enable dynamic quota**: Disabled + - **Guardrails**: DefaultV2 + + Then select **Deploy** at the bottom of the page. > \* A rate limit of 10,000 tokens per minute is more than adequate to complete this exercise while leaving capacity for other people using the same subscription. +2. Wait for the deployments to complete. + ## Provision an Azure Databricks workspace > **Tip**: If you already have an Azure Databricks workspace, you can skip this procedure and use your existing workspace. 1. Sign into the **Azure portal** at `https://portal.azure.com`. 1. Create an **Azure Databricks** resource with the following settings: - - **Subscription**: *Select the same Azure subscription that you used to create your Azure OpenAI resource* - - **Resource group**: *The same resource group where you created your Azure OpenAI resource* - - **Region**: *The same region where you created your Azure OpenAI resource* - - **Name**: *A unique name of your choice* - - **Pricing tier**: *Premium* or *Trial* + - **Subscription**: *Select the same Azure subscription that you used to create your Foundry resource* + - **Resource group**: *The same resource group where you created your Foundry resource* + - **Workspace name**: *A unique name of your choice* + - **Region**: *Select any available region* + - **Pricing tier**: Premium (+ Role-based access controls) + - **Workspace type**: Hybrid + - **Managed Resource Group name**: *Leave blank* -1. Select **Review + create** and wait for deployment to complete. Then go to the resource and launch the workspace. +> **Note**: Azure Databricks does not need to be in the same region as your Foundry resource. If cluster creation fails due to quota limits in your chosen region, try deleting the workspace and creating a new one in a different region. -## Create a notebook and install required libraries +1. Select **Review + create**, and once validation succeeds, select **Create**. + +1. When deployment is complete, select **Go to resource**, then select **Launch Workspace** to open your Azure Databricks workspace in a new browser tab. + +## Create a cluster + +Azure Databricks is a distributed processing platform that uses Apache Spark *clusters* to process data in parallel on multiple nodes. Each cluster consists of a driver node to coordinate the work, and worker nodes to perform processing tasks. In this exercise, you'll create a *single-node* cluster to minimize the compute resources used in the lab environment (in which resources may be constrained). In a production environment, you'd typically create a cluster with multiple worker nodes. -1. In the Azure portal, browse to the resource group where the Azure Databricks workspace was created. +> **Tip**: If you already have a cluster with a 17.3 LTS **ML** or higher runtime version in your Azure Databricks workspace, you can use it to complete this exercise and skip this procedure. -1. Select your Azure Databricks Service resource. +1. In the sidebar on the left, select the **(+) New** task, select **More**, and then select **Cluster**. +1. In the **New Cluster** page, create a new cluster with the following settings: + - **Cluster name**: *User Name's* cluster (the default cluster name) + - **Policy**: Unrestricted + - **Machine learning**: Enabled + - **Databricks runtime**: 17.3 LTS + - **Use Photon Acceleration**: Unselected + - **Worker type**: Standard_D4ds_v5 + - **Single node**: Checked + - **Terminate after**: 30 minutes of inactivity +1. Select **Create** -1. In the **Overview** page for your workspace, use the **Launch Workspace** button to open your Azure Databricks workspace in a new browser tab; signing in if prompted. +1. Wait for the cluster to be created. It may take a minute or two. - > **Tip**: As you use the Databricks Workspace portal, various tips and notifications may be displayed. Dismiss these and follow the instructions provided to complete the tasks in this exercise. +> **Note**: If your cluster fails to start, your subscription may have insufficient quota in the region where your Azure Databricks workspace is provisioned. See [CPU core limit prevents cluster creation](https://docs.microsoft.com/azure/databricks/kb/clusters/azure-core-limit) for details. If this happens, you can try deleting your workspace and creating a new one in a different region. -1. In the Databricks workspace, go to the **Workspace** section. +## Create a notebook and install required libraries -1. Select **Create** and then select **Notebook**. +1. In the sidebar on the left, use the **(+) New** link to create a **Notebook**. -1. Name your notebook and select `Python` as the language. Select **Serverless** as the default compute. +1. Name your notebook and select `Python` as the language. In the **Connect** drop-down list, select your cluster if it is not already selected. If the cluster is not running, it may take a minute or so to start. 1. In the first code cell, enter and run the following code to install the necessary libraries: @@ -115,22 +144,27 @@ Azure provides a web-based portal named **Microsoft Foundry**, that you can use %pip install langchain openai langchain_openai langchain-community faiss-cpu ``` + > **Note**: You may see warnings that package versions are not pinned, or that core Python package versions changed. These are advisory only and won't affect the lab — the `%restart_python` command in the next step restarts the Python environment to apply the updates. + 1. After the installation is complete, restart the kernel in a new cell: ```python %restart_python ``` -1. In a new cell, define the authentication parameters that will be used to initialize the OpenAI models, replacing `your_openai_endpoint` and `your_openai_api_key` with the endpoint and key copied earlier from your OpenAI resource: +1. In a new cell, run the following code with the access information you copied earlier to assign persistent environment variables for authentication: ```python - endpoint = "your_openai_endpoint" - key = "your_openai_api_key" + import os + os.environ["AZURE_OPENAI_ENDPOINT"] = "your_endpoint" # e.g. https://yourresource.services.ai.azure.com/ + os.environ["COGNITIVE_SERVICES_TOKEN"] = "your_cognitiveservices_access_token" # from: az account get-access-token --resource https://cognitiveservices.azure.com ``` + + > **Note**: The access token expires after approximately 60 minutes. If you encounter authentication errors during the lab, re-run the Cloud Shell command and update this cell. ## Create a Vector Index and Store Embeddings -A vector index is a specialized data structure that allows for efficient storage and retrieval of high-dimensional vector data, which is crucial for performing fast similarity searches and nearest neighbor queries. Embeddings, on the other hand, are numerical representations of objects that capture their meaning in a vector form, enabling machines to process and understand various types of data, including text and images. +In this lab, you're building a retrieval-augmented generation (RAG) pipeline using LangChain. Before the language model can answer questions, it needs a knowledge base to search. In this section, you convert sample documents into vector embeddings and store them in a FAISS index. Later, the retriever uses this index to find the most relevant documents for a given query, and passes them to the model as context. 1. In a new cell, run the following code to load a sample dataset: @@ -153,19 +187,19 @@ A vector index is a specialized data structure that allows for efficient storage embedding_function = AzureOpenAIEmbeddings( deployment="text-embedding-ada-002", model="text-embedding-ada-002", - azure_endpoint=endpoint, - openai_api_key=key, - openai_api_version="2023-03-15-preview" + azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), + azure_ad_token=os.getenv("COGNITIVE_SERVICES_TOKEN"), + openai_api_version="2025-04-01-preview", chunk_size=1 ) ``` -1. In a new cell, run the following code to create a vector index using the first text sample as a reference for the vector dimension: +1. In a new cell, run the following code to create a vector index using the known dimension for `text-embedding-ada-002` embeddings: ```python import faiss - index = faiss.IndexFlatL2(len(embedding_function.embed_query("Azure Databricks is a fast, easy, and collaborative Apache Spark-based analytics platform."))) + index = faiss.IndexFlatL2(1536) # text-embedding-ada-002 always produces 1536-dimensional vectors ``` ## Build a Retriever-based Chain @@ -175,7 +209,7 @@ A retriever component fetches relevant documents or data based on a query. This 1. In a new cell, run the following code to create a retriever that can search the vector index for the most similar texts. ```python - from langchain.community_vectorstores import FAISS + from langchain_community.vectorstores import FAISS from langchain_core.vectorstores import VectorStoreRetriever vector_store = FAISS.from_documents( @@ -186,20 +220,20 @@ A retriever component fetches relevant documents or data based on a query. This retriever = VectorStoreRetriever(vectorstore=vector_store) ``` -1. In a new cell, run the following code to create a QA system using the retriever and the `gpt-4o` model: +1. In a new cell, run the following code to create a QA system using the retriever and the `gpt-4.1` model: ```python from langchain_openai import AzureChatOpenAI from langchain_core.prompts import ChatPromptTemplate - from langchain_classic.chains.combine_documents.stuff import create_stuff_documents_chain - from langchain_classic.chains import create_retrieval_chain + from langchain_core.output_parsers import StrOutputParser + from operator import itemgetter llm = AzureChatOpenAI( - deployment_name="gpt-4o", - model_name="gpt-4o", - azure_endpoint=endpoint, - api_version="2023-03-15-preview", - openai_api_key=key, + deployment_name="gpt-4.1", + model_name="gpt-4.1", + azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), + api_version="2025-04-01-preview", + azure_ad_token=os.getenv("COGNITIVE_SERVICES_TOKEN"), ) system_prompt = ( @@ -214,9 +248,18 @@ A retriever component fetches relevant documents or data based on a query. This ("human", "{input}") ]) - chain = create_stuff_documents_chain(llm, prompt1) - - qa_chain1 = create_retrieval_chain(retriever, chain) + def format_docs(docs): + return "\n\n".join(doc.page_content for doc in docs) + + qa_chain1 = ( + { + "context": itemgetter("input") | retriever | format_docs, + "input": itemgetter("input") + } + | prompt1 + | llm + | StrOutputParser() + ) ``` 1. In a new cell, run the following code to test the QA system: @@ -254,6 +297,6 @@ Langchain is a versatile tool that allows the combination of multiple chains int ## Clean up -When you're done with your Azure OpenAI resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. +When you're done with your Microsoft Foundry resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. If you've finished exploring Azure Databricks, you can delete the resources you've created to avoid unnecessary Azure costs and free up capacity in your subscription. diff --git a/Instructions/Exercises/AI-06-Responsible-AI.md b/Instructions/Exercises/AI-06-Responsible-AI.md index 03c388f..7b75540 100644 --- a/Instructions/Exercises/AI-06-Responsible-AI.md +++ b/Instructions/Exercises/AI-06-Responsible-AI.md @@ -1,6 +1,6 @@ --- lab: - title: Responsible AI with Large Language Models using Azure Databricks and Azure OpenAI + title: Responsible AI with Large Language Models using Azure Databricks and Microsoft Foundry description: You'll gain hands-on experience testing Large Language Models for bias by creating neutral and gender-loaded input prompts and comparing the outputs to detect potential biases inherited from training data. You'll learn practical techniques for identifying and analyzing bias in AI systems, and understand mitigation approaches like re-sampling, re-weighting, or modifying training data to ensure fairer and more responsible AI deployments. duration: 20 minutes level: 400 @@ -11,9 +11,9 @@ lab: - Microsoft Foundry --- -# Responsible AI with Large Language Models using Azure Databricks and Azure OpenAI +# Responsible AI with Large Language Models using Azure Databricks and Microsoft Foundry -Integrating Large Language Models (LLMs) into Azure Databricks and Azure OpenAI offers a powerful platform for responsible AI development. These sophisticated transformer-based models excel in natural language processing tasks, enabling developers to innovate rapidly while adhering to principles of fairness, reliability, safety, privacy, security, inclusiveness, transparency, and accountability. +Responsible AI development requires actively testing language models for potential biases that can emerge from training data. In this lab, you use Azure Databricks and Microsoft Foundry to probe a large language model for gender bias by crafting and comparing neutral and gender-loaded prompts. By sending both types of prompts to a `gpt-4.1` model and analyzing the responses side by side in a PySpark DataFrame, you can see firsthand how a model's output language shifts based on subtle input differences — a practical technique for identifying bias before deploying AI systems at scale. This lab will take approximately **30** minutes to complete. @@ -23,93 +23,138 @@ This lab will take approximately **30** minutes to complete. You'll need an [Azure subscription](https://azure.microsoft.com/free) in which you have administrative-level access. -## Provision an Azure OpenAI resource +## Create a Microsoft Foundry resource and project -If you don't already have one, provision an Azure OpenAI resource in your Azure subscription. +If you don't already have one, create a Microsoft Foundry resource and project in your Azure subscription. + +> **Note**: Creating a Foundry resource only requires a subscription, resource group, region, and name. No Key Vault or Application Insights resources are needed. 1. Sign into the **Azure portal** at `https://portal.azure.com`. -2. Create an **Azure OpenAI** resource with the following settings: - - **Subscription**: *Select an Azure subscription that has been approved for access to the Azure OpenAI service* +2. Use the following link to open the Foundry resource creation page: `https://portal.azure.com/#create/Microsoft.CognitiveServicesAIFoundry` +3. On the **Create** page, provide the following information on the **Basics** tab: + - **Subscription**: *Select your Azure subscription* - **Resource group**: *Choose or create a resource group* + - **Name**: *A unique name of your choice* - **Region**: *Make a **random** choice from any of the following regions*\* - - East US 2 - North Central US - Sweden Central - - Switzerland West - - **Name**: *A unique name of your choice* - - **Pricing tier**: Standard S0 + - **Default project name**: *Leave the pre-filled default or enter a custom project name* +4. Select **Review + create**, then select **Create** and wait for deployment to complete. + +> \* Foundry resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. -> \* Azure OpenAI resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. +5. Once deployment completes, go to the deployed resource. In the left pane, under **Resource Management**, select **Keys and Endpoint**, then on the **Foundry** tab, copy the **API endpoint** — you will use it later in this exercise. -3. Wait for deployment to complete. Then go to the deployed Azure OpenAI resource in the Azure portal. +6. In the **Overview** page, select **Go to Foundry portal** to open your resource in the Foundry portal (or navigate directly to `https://ai.azure.com`). -4. In the left pane, under **Resource Management**, select **Keys and Endpoint**. +7. In **Microsoft Foundry**, select the project within your Foundry resource. A default project is created automatically — select it to open it. If no project exists, create one: + - Select **+ New project** in the left navigation. + - Enter a **Project name** and select **Create project**. + - Wait for the project to be created. + +8. In a new browser tab, return to the **Azure portal** at `https://portal.azure.com` and launch Cloud Shell. Run the following command to get a temporary authorization token for API calls. Copy the `accessToken` value and save it alongside the endpoint you copied in step 5. + + ```bash + az account get-access-token --resource https://cognitiveservices.azure.com + ``` -5. Copy the endpoint and one of the available keys as you will use it later in this exercise. + >**Note**: You only need to copy the `accessToken` field value and **not** the entire JSON output. ## Deploy the required model -Azure provides a web-based portal named **Microsoft Foundry**, that you can use to deploy, manage, and explore models. You'll start your exploration of Azure OpenAI by using Microsoft Foundry to deploy a model. +Microsoft Foundry allows you to deploy, manage, and explore models. > **Note**: As you use Microsoft Foundry, message boxes suggesting tasks for you to perform may be displayed. You can close these and follow the steps in this exercise. -1. In the Azure portal, on the **Overview** page for your Azure OpenAI resource, scroll down to the **Get Started** section and select the button to go to **Microsoft Foundry**. - -1. In Microsoft Foundry, in the pane on the left, select the **Deployments** page and view your existing model deployments. If you don't already have one, create a new deployment of the **gpt-4o** model with the following settings: - - **Deployment name**: *gpt-4o* - - **Deployment type**: Standard - - **Model version**: *Use default version* +1. In **Microsoft Foundry**, on the home page select **View deployments** (or select **Build** in the top navigation bar, then select **Deployments**). + +1. Select **Deploy** > **Deploy a base model**, search for and select **gpt-4.1**, then select **Deploy** > **Custom settings** to configure the deployment with the following settings: + - **Deployment name**: *gpt-4.1* + - **Deployment type**: Global Standard + - **Model version** *(under Model version settings)*: *2025-04-14* + - **Model version upgrade policy** *(under Model version settings)*: Upgrade once new default version becomes available - **Tokens per minute rate limit**: 10K\* - - **Content filter**: Default - - **Enable dynamic quota**: Disabled - + - **Guardrails**: DefaultV2 + + Then select **Deploy** at the bottom of the page. + > \* A rate limit of 10,000 tokens per minute is more than adequate to complete this exercise while leaving capacity for other people using the same subscription. +2. Wait for the deployment to complete. + ## Provision an Azure Databricks workspace > **Tip**: If you already have an Azure Databricks workspace, you can skip this procedure and use your existing workspace. 1. Sign into the **Azure portal** at `https://portal.azure.com`. 2. Create an **Azure Databricks** resource with the following settings: - - **Subscription**: *Select the same Azure subscription that you used to create your Azure OpenAI resource* - - **Resource group**: *The same resource group where you created your Azure OpenAI resource* - - **Region**: *The same region where you created your Azure OpenAI resource* - - **Name**: *A unique name of your choice* - - **Pricing tier**: *Premium* or *Trial* + - **Subscription**: *Select the same Azure subscription that you used to create your Foundry resource* + - **Resource group**: *The same resource group where you created your Foundry resource* + - **Workspace name**: *A unique name of your choice* + - **Region**: *Select any available region* + - **Pricing tier**: Premium (+ Role-based access controls) + - **Workspace type**: Hybrid + - **Managed Resource Group name**: *Leave blank* -3. Select **Review + create** and wait for deployment to complete. Then go to the resource and launch the workspace. +> **Note**: Azure Databricks does not need to be in the same region as your Foundry resource. If cluster creation fails due to quota limits in your chosen region, try deleting the workspace and creating a new one in a different region. -## Create a notebook +1. Select **Review + create**, and once validation succeeds, select **Create**. -1. In the Azure portal, browse to the resource group where the Azure Databricks workspace was created. +1. When deployment is complete, select **Go to resource**, then select **Launch Workspace** to open your Azure Databricks workspace in a new browser tab. -1. Select your Azure Databricks Service resource. +## Create a cluster -1. In the **Overview** page for your workspace, use the **Launch Workspace** button to open your Azure Databricks workspace in a new browser tab; signing in if prompted. +Azure Databricks is a distributed processing platform that uses Apache Spark *clusters* to process data in parallel on multiple nodes. Each cluster consists of a driver node to coordinate the work, and worker nodes to perform processing tasks. In this exercise, you'll create a *single-node* cluster to minimize the compute resources used in the lab environment (in which resources may be constrained). In a production environment, you'd typically create a cluster with multiple worker nodes. - > **Tip**: As you use the Databricks Workspace portal, various tips and notifications may be displayed. Dismiss these and follow the instructions provided to complete the tasks in this exercise. +> **Tip**: If you already have a cluster with a 17.3 LTS **ML** or higher runtime version in your Azure Databricks workspace, you can use it to complete this exercise and skip this procedure. -1. In the sidebar, use the **(+) New** link to create a **Notebook**. - -1. Name your notebook and in the **Connect** drop-down list, select **Serverless** as the default compute. +1. In the sidebar on the left, select the **(+) New** task, select **More**, and then select **Cluster**. +1. In the **New Cluster** page, create a new cluster with the following settings: + - **Cluster name**: *User Name's* cluster (the default cluster name) + - **Policy**: Unrestricted + - **Machine learning**: Enabled + - **Databricks runtime**: 17.3 LTS + - **Use Photon Acceleration**: Unselected + - **Worker type**: Standard_D4ds_v5 + - **Single node**: Checked + - **Terminate after**: 30 minutes of inactivity +1. Select **Create** + +1. Wait for the cluster to be created. It may take a minute or two. + +> **Note**: If your cluster fails to start, your subscription may have insufficient quota in the region where your Azure Databricks workspace is provisioned. See [CPU core limit prevents cluster creation](https://docs.microsoft.com/azure/databricks/kb/clusters/azure-core-limit) for details. If this happens, you can try deleting your workspace and creating a new one in a different region. + +## Create a notebook + +1. In the sidebar on the left, use the **(+) New** link to create a **Notebook**. + +1. Name your notebook and select `Python` as the language. In the **Connect** drop-down list, select your cluster if it is not already selected. If the cluster is not running, it may take a minute or so to start. 1. In the first code cell, enter and run the following code to install the required libraries: + + ```python + %pip install openai + ``` + + > **Note**: You may see warnings that package versions are not pinned, or that core Python package versions changed. These are advisory only and won't affect the lab — the `%restart_python` command in the next step restarts the Python environment to apply the updates. + +1. After the installation is complete, restart the kernel in a new cell: ```python - %pip install openai - dbutils.library.restartPython() + %restart_python ``` -1. In a new cell, run the following code with the access information you copied at the beginning of this exercise to assign persistent environment variables for authentication when using Azure OpenAI resources: +1. In a new cell, run the following code with the access information you copied earlier to assign persistent environment variables for authentication: ```python import os - os.environ["AZURE_OPENAI_API_KEY"] = "your_openai_api_key" - os.environ["AZURE_OPENAI_ENDPOINT"] = "your_openai_endpoint" - os.environ["AZURE_OPENAI_API_VERSION"] = "your_model_version_example: '2023-03-15-preview'" + os.environ["AZURE_OPENAI_ENDPOINT"] = "your_foundry_endpoint" + os.environ["COGNITIVE_SERVICES_TOKEN"] = "your_cognitiveservices_access_token" # from: az account get-access-token --resource https://cognitiveservices.azure.com ``` + > **Note**: The access token expires after approximately 60 minutes. If you encounter authentication errors during the lab, re-run the Cloud Shell command and update this cell. + 1. In a new cell, run the following code to create two input samples: ```python @@ -133,7 +178,7 @@ These samples will be used to verify if the model has a gender bias inherited fr ## Implement Responsible AI Practices -Responsible AI refers to the ethical and sustainable development, deployment, and use of artificial intelligence systems. It emphasizes the need for AI to operate in a manner that aligns with legal, social, and ethical norms. This includes considerations for fairness, accountability, transparency, privacy, safety, and the overall societal impact of AI technologies. Responsible AI frameworks promote the adoption of guidelines and practices that can mitigate the potential risks and negative consequences associated with AI, while maximizing its positive impacts for individuals and society as a whole. +You now have two sets of prompts — one neutral, one gender-loaded. In this section, you send both sets to `gpt-4.1` and collect the responses. You then load both sets of answers into PySpark DataFrames and display them side by side. By comparing how the model describes, for example, 'a nurse' versus 'a male nurse', you can see whether specifying a gender changes the model's word choice, tone, or assumptions — a direct way to surface bias before it reaches production. 1. In a new cell, run the following code to generate outputs for your sample inputs: @@ -143,8 +188,8 @@ Responsible AI refers to the ethical and sustainable development, deployment, an client = AzureOpenAI( azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), - api_key = os.getenv("AZURE_OPENAI_API_KEY"), - api_version = os.getenv("AZURE_OPENAI_API_VERSION") + azure_ad_token = os.getenv("COGNITIVE_SERVICES_TOKEN"), + api_version = "2025-04-01-preview" ) system_prompt = "You are an advanced language model designed to assist with a variety of tasks. Your responses should be accurate, contextually appropriate, and free from any form of bias." @@ -153,7 +198,7 @@ Responsible AI refers to the ethical and sustainable development, deployment, an for row in neutral_input: completion = client.chat.completions.create( - model="gpt-4o", + model="gpt-4.1", messages=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": row}, @@ -164,7 +209,7 @@ Responsible AI refers to the ethical and sustainable development, deployment, an for row in loaded_input: completion = client.chat.completions.create( - model="gpt-4o", + model="gpt-4.1", messages=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": row}, @@ -192,6 +237,6 @@ If bias is detected, there are mitigation techniques such as re-sampling, re-wei ## Clean up -When you're done with your Azure OpenAI resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. +When you're done with your Microsoft Foundry resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. If you've finished exploring Azure Databricks, you can delete the resources you've created to avoid unnecessary Azure costs and free up capacity in your subscription. diff --git a/Instructions/Exercises/AI-07-LLMOps.md b/Instructions/Exercises/AI-07-LLMOps.md index 674dba9..fd0785f 100644 --- a/Instructions/Exercises/AI-07-LLMOps.md +++ b/Instructions/Exercises/AI-07-LLMOps.md @@ -13,7 +13,7 @@ lab: # Implement LLMOps with Azure Databricks -Azure Databricks provides a unified platform that streamlines the AI lifecycle, from data preparation to model serving and monitoring, optimizing the performance and efficiency of machine learning systems. It supports the development of generative AI applications, leveraging features like Unity Catalog for data governance, MLflow for model tracking, and Mosaic AI Model Serving for deploying LLMs. +LLMOps (Large Language Model Operations) brings structured observability to AI applications by tracking every interaction a model has in production. In this lab, you use Azure Databricks and Microsoft Foundry to connect a `gpt-4.1` model to MLflow's autologging capabilities, capturing inputs, outputs, token usage, and metadata for each run. After executing a prompt, you explore the MLflow Trace UI to see how individual requests are recorded — giving you the foundation to compare runs over time, detect data drift, and keep your LLM applications reliable at scale. This lab will take approximately **30** minutes to complete. @@ -23,93 +23,141 @@ This lab will take approximately **30** minutes to complete. You'll need an [Azure subscription](https://azure.microsoft.com/free) in which you have administrative-level access. -## Provision an Azure OpenAI resource +## Create a Microsoft Foundry resource and project -If you don't already have one, provision an Azure OpenAI resource in your Azure subscription. +If you don't already have one, create a Microsoft Foundry resource and project in your Azure subscription. + +> **Note**: Creating a Foundry resource only requires a subscription, resource group, region, and name. No Key Vault or Application Insights resources are needed. 1. Sign into the **Azure portal** at `https://portal.azure.com`. -2. Create an **Azure OpenAI** resource with the following settings: - - **Subscription**: *Select an Azure subscription that has been approved for access to the Azure OpenAI service* +2. Use the following link to open the Foundry resource creation page: `https://portal.azure.com/#create/Microsoft.CognitiveServicesAIFoundry` +3. On the **Create** page, provide the following information on the **Basics** tab: + - **Subscription**: *Select your Azure subscription* - **Resource group**: *Choose or create a resource group* + - **Name**: *A unique name of your choice* - **Region**: *Make a **random** choice from any of the following regions*\* - - East US 2 - North Central US - Sweden Central - - Switzerland West - - **Name**: *A unique name of your choice* - - **Pricing tier**: Standard S0 + - **Default project name**: *Leave the pre-filled default or enter a custom project name* +4. Select **Review + create**, then select **Create** and wait for deployment to complete. + +> \* Foundry resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. -> \* Azure OpenAI resources are constrained by regional quotas. The listed regions include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit in scenarios where you are sharing a subscription with other users. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. +5. Once deployment completes, go to the deployed resource. In the left pane, under **Resource Management**, select **Keys and Endpoint**, then on the **Foundry** tab, copy the **API endpoint** — you will use it later in this exercise. -3. Wait for deployment to complete. Then go to the deployed Azure OpenAI resource in the Azure portal. +6. In the **Overview** page, select **Go to Foundry portal** to open your resource in the Foundry portal (or navigate directly to `https://ai.azure.com`). -4. In the left pane, under **Resource Management**, select **Keys and Endpoint**. +7. In **Microsoft Foundry**, select the project within your Foundry resource. A default project is created automatically — select it to open it. If no project exists, create one: + - Select **+ New project** in the left navigation. + - Enter a **Project name** and select **Create project**. + - Wait for the project to be created. + +8. In a new browser tab, return to the **Azure portal** at `https://portal.azure.com` and launch Cloud Shell. Run the following command to get a temporary authorization token for API calls. Copy the `accessToken` value and save it alongside the endpoint you copied in step 5. + + ```bash + az account get-access-token --resource https://cognitiveservices.azure.com + ``` -5. Copy the endpoint and one of the available keys as you will use it later in this exercise. + >**Note**: You only need to copy the `accessToken` field value and **not** the entire JSON output. ## Deploy the required model -Azure provides a web-based portal named **Microsoft Foundry**, that you can use to deploy, manage, and explore models. You'll start your exploration of Azure OpenAI by using Microsoft Foundry to deploy a model. +Microsoft Foundry allows you to deploy, manage, and explore models. > **Note**: As you use Microsoft Foundry, message boxes suggesting tasks for you to perform may be displayed. You can close these and follow the steps in this exercise. -1. In the Azure portal, on the **Overview** page for your Azure OpenAI resource, scroll down to the **Get Started** section and select the button to go to **Microsoft Foundry**. - -1. In Microsoft Foundry, in the pane on the left, select the **Deployments** page and view your existing model deployments. If you don't already have one, create a new deployment of the **gpt-4o** model with the following settings: - - **Deployment name**: *gpt-4o* - - **Deployment type**: Standard - - **Model version**: *Use default version* +1. In **Microsoft Foundry**, on the home page select **View deployments** (or select **Build** in the top navigation bar, then select **Deployments**). + +1. Select **Deploy** > **Deploy a base model**, search for and select **gpt-4.1**, then select **Deploy** > **Custom settings** to configure the deployment with the following settings: + - **Deployment name**: *gpt-4.1* + - **Deployment type**: Global Standard + - **Model version** *(under Model version settings)*: *2025-04-14* + - **Model version upgrade policy** *(under Model version settings)*: Upgrade once new default version becomes available - **Tokens per minute rate limit**: 10K\* - - **Content filter**: Default - - **Enable dynamic quota**: Disabled - + - **Guardrails**: DefaultV2 + + Then select **Deploy** at the bottom of the page. + > \* A rate limit of 10,000 tokens per minute is more than adequate to complete this exercise while leaving capacity for other people using the same subscription. +2. Wait for the deployment to complete. + ## Provision an Azure Databricks workspace > **Tip**: If you already have an Azure Databricks workspace, you can skip this procedure and use your existing workspace. 1. Sign into the **Azure portal** at `https://portal.azure.com`. 2. Create an **Azure Databricks** resource with the following settings: - - **Subscription**: *Select the same Azure subscription that you used to create your Azure OpenAI resource* - - **Resource group**: *The same resource group where you created your Azure OpenAI resource* - - **Region**: *The same region where you created your Azure OpenAI resource* - - **Name**: *A unique name of your choice* - - **Pricing tier**: *Premium* or *Trial* + - **Subscription**: *Select the same Azure subscription that you used to create your Foundry resource* + - **Resource group**: *The same resource group where you created your Foundry resource* + - **Workspace name**: *A unique name of your choice* + - **Region**: *Select any available region* + - **Pricing tier**: Premium (+ Role-based access controls) + - **Workspace type**: Hybrid + - **Managed Resource Group name**: *Leave blank* -3. Select **Review + create** and wait for deployment to complete. Then go to the resource and launch the workspace. +> **Note**: Azure Databricks does not need to be in the same region as your Foundry resource. If cluster creation fails due to quota limits in your chosen region, try deleting the workspace and creating a new one in a different region. -## Create a notebook +1. Select **Review + create**, and once validation succeeds, select **Create**. -1. In the Azure portal, browse to the resource group where the Azure Databricks workspace was created. +1. When deployment is complete, select **Go to resource**, then select **Launch Workspace** to open your Azure Databricks workspace in a new browser tab. -1. Select your Azure Databricks Service resource. +## Create a cluster -1. In the **Overview** page for your workspace, use the **Launch Workspace** button to open your Azure Databricks workspace in a new browser tab; signing in if prompted. +Azure Databricks is a distributed processing platform that uses Apache Spark *clusters* to process data in parallel on multiple nodes. Each cluster consists of a driver node to coordinate the work, and worker nodes to perform processing tasks. In this exercise, you'll create a *single-node* cluster to minimize the compute resources used in the lab environment (in which resources may be constrained). In a production environment, you'd typically create a cluster with multiple worker nodes. - > **Tip**: As you use the Databricks Workspace portal, various tips and notifications may be displayed. Dismiss these and follow the instructions provided to complete the tasks in this exercise. +> **Tip**: If you already have a cluster with a 17.3 LTS **ML** or higher runtime version in your Azure Databricks workspace, you can use it to complete this exercise and skip this procedure. -1. In the sidebar, use the **(+) New** link to create a **Notebook**. Select **Serverless** as the default compute. +1. In the sidebar on the left, select the **(+) New** task, select **More**, and then select **Cluster**. +1. In the **New Cluster** page, create a new cluster with the following settings: + - **Cluster name**: *User Name's* cluster (the default cluster name) + - **Policy**: Unrestricted + - **Machine learning**: Enabled + - **Databricks runtime**: 17.3 LTS + - **Use Photon Acceleration**: Unselected + - **Worker type**: Standard_D4ds_v5 + - **Single node**: Checked + - **Terminate after**: 30 minutes of inactivity +1. Select **Create** + +1. Wait for the cluster to be created. It may take a minute or two. + +> **Note**: If your cluster fails to start, your subscription may have insufficient quota in the region where your Azure Databricks workspace is provisioned. See [CPU core limit prevents cluster creation](https://docs.microsoft.com/azure/databricks/kb/clusters/azure-core-limit) for details. If this happens, you can try deleting your workspace and creating a new one in a different region. + +## Create a notebook + +1. In the sidebar on the left, use the **(+) New** link to create a **Notebook**. + +1. Name your notebook and select `Python` as the language. In the **Connect** drop-down list, select your cluster if it is not already selected. If the cluster is not running, it may take a minute or so to start. 1. In the first code cell, enter and run the following code to install the required libraries: + + ```python + %pip install openai mlflow + ``` + + > **Note**: You may see warnings that package versions are not pinned, or that core Python package versions changed. These are advisory only and won't affect the lab — the `%restart_python` command in the next step restarts the Python environment to apply the updates. + +1. After the installation is complete, restart the kernel in a new cell: ```python - %pip install openai mlflow - dbutils.library.restartPython() + %restart_python ``` + ## Log the LLM using MLflow -MLflow’s LLM tracking capabilities allow you to log parameters, metrics, predictions, and artifacts. Parameters include key-value pairs detailing input configurations, while metrics provide quantitative measures of performance. Predictions encompass both the input prompts and the model’s responses, stored as artifacts for easy retrieval. This structured logging helps in maintaining a detailed record of each interaction, facilitating better analysis and optimization of LLMs. +In this section, you wire up MLflow's autologging before making any API calls. Calling `mlflow.openai.autolog()` tells MLflow to intercept every OpenAI SDK call and automatically record the prompt, response, model name, and token counts as a trace. You then wrap a test prompt inside `mlflow.start_run()` so it belongs to a named experiment run, and manually log one extra parameter — the number of completion tokens — to show how you can attach custom metrics alongside the automatic trace data. -1. In a new cell, run the following code with the access information you copied at the beginning of this exercise to assign persistent environment variables for authentication when using Azure OpenAI resources: +1. In a new cell, run the following code with the access information you copied earlier to assign persistent environment variables for authentication: ```python import os - os.environ["AZURE_OPENAI_API_KEY"] = "your_openai_api_key" - os.environ["AZURE_OPENAI_ENDPOINT"] = "your_openai_endpoint" - os.environ["AZURE_OPENAI_API_VERSION"] = "your_model_version_example: '2023-03-15-preview'" + os.environ["AZURE_OPENAI_ENDPOINT"] = "your_foundry_endpoint" + os.environ["COGNITIVE_SERVICES_TOKEN"] = "your_cognitiveservices_access_token" # from: az account get-access-token --resource https://cognitiveservices.azure.com ``` + + > **Note**: The access token expires after approximately 60 minutes. If you encounter authentication errors during the lab, re-run the Cloud Shell command and update this cell. 1. In a new cell, run the following code to initialize your Azure OpenAI client: ```python @@ -118,8 +166,8 @@ MLflow’s LLM tracking capabilities allow you to log parameters, metrics, predi client = AzureOpenAI( azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), - api_key = os.getenv("AZURE_OPENAI_API_KEY"), - api_version = os.getenv("AZURE_OPENAI_API_VERSION") + azure_ad_token = os.getenv("COGNITIVE_SERVICES_TOKEN"), + api_version = "2025-04-01-preview" ) ``` @@ -136,7 +184,7 @@ MLflow’s LLM tracking capabilities allow you to log parameters, metrics, predi with mlflow.start_run(): response = client.chat.completions.create( - model="gpt-4o", + model="gpt-4.1", messages=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": "Tell me a joke about animals."}, @@ -152,14 +200,18 @@ The cell above will start an experiment in your workspace and register the trace ## Monitor the model -After running the last cell, MLflow Trace UI will automatically be displayed together with the cell's output. You can also see it by selecting **Experiments** in the left sidebar, and then opening your notebook's experiment run: +After running the last cell, the trace is recorded automatically. To view it: + +1. In the left sidebar, select **Experiments**. +1. In the list of experiments, select the name of your notebook experiment (for example, *New Notebook 2026-06-25 17:27:57*). +1. In the left panel of the experiment view, under **Observability**, select **Traces**. - ![MLFlow Trace UI](./images/trace-ui.png) +You'll see a table listing each recorded trace with columns for the **Request** (the prompt you sent), **Response** (the model's reply), **Tokens** used, **Execution time**, and **State**. Select any trace to expand it and see the full input/output detail. -The command `mlflow.openai.autolog()` will log the traces of each run by default, but you can also log additional parameters with `mlflow.log_param()` that can later be used to monitor the model. Once you start monitoring the model, you can compare the traces from different runs to detect data drift. Look for significant changes in the input data distributions, model predictions, or performance metrics over time. You can also use statistical tests or visualization tools to aid in this analysis. +With autologging active, every run you add builds a history you can compare. Look at how the prompt, response content, and token counts vary across runs. You can also attach custom metrics using `mlflow.log_param()`. Over time, drift shows up as systematic shifts — responses getting longer or shorter, token usage climbing, or output quality changing — and spotting these trends early lets you retune the model or adjust the prompt before users notice a difference. ## Clean up -When you're done with your Azure OpenAI resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. +When you're done with your Microsoft Foundry resource, remember to delete the deployment or the entire resource in the **Azure portal** at `https://portal.azure.com`. If you've finished exploring Azure Databricks, you can delete the resources you've created to avoid unnecessary Azure costs and free up capacity in your subscription.