Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 170 additions & 22 deletions Allfiles/01-explore-azure-databricks.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,16 @@
"cells": [
{
"cell_type": "markdown",
"id": "82c6d5ed",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "7a601857-78c8-4bab-b2a6-4389ddc11845",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"## Exercise 1: Running Python Code\n",
"\n",
Expand All @@ -17,25 +25,80 @@
"2. Prints a formatted message that summarises the route. For example:\n",
" `Route: City Express | From: Central Station → To: Airport Terminal`\n",
"\n",
"> 💬 **Genie Code tip:** Open Genie Code panel (click the ![assistant-icon](https://raw.githubusercontent.com/MicrosoftLearning/DP-750T00-Implement-Data-Engineering-Solutions-using-Azure-Databricks/refs/heads/main/Allfiles/media/databricks-assistant.svg) icon on the cell) and type:\n",
"> \uD83D\uDCAC **Genie Code tip:** Open Genie Code panel (click the ![assistant-icon](https://raw.githubusercontent.com/MicrosoftLearning/DP-750T00-Implement-Data-Engineering-Solutions-using-Azure-Databricks/refs/heads/main/Allfiles/media/databricks-assistant.svg) icon on the cell) and type:\n",
"> *\"Write Python code to define three route variables and print a formatted summary message\"*\n",
"> Use the suggestion as a starting point, then personalise the values."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef6b149a",
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"implicitDf": true,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "784b33d8-e96a-4ecf-912c-b4c1ea50a4cc",
"showTitle": true,
"tableResultSettingsMap": {},
"title": "Cell 2"
}
},
"outputs": [],
"source": [
"# TODO: Define your route variables and print a formatted summary message"
"%python\n",
"\n",
"route_data =[]\n",
"route_id = ['R001' 'R002', 'R003']\n",
"route_name = ['City Express', 'Harbor Line', 'Metro']\n",
"origin = ['Central Station', 'Downtown', 'Airport Terminal']\n",
"destination = ['Airport Terminal', 'Airport Terminal', 'City Center']\n",
"\n",
"for i in range(len(route_id)):\n",
" route_data.append([route_id[i],route_name[i],origin[i],destination[i]])\n",
"\n",
"route_data\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "374888a6-5fa5-4ad1-a90a-6dab15ad7252",
"showTitle": true,
"tableResultSettingsMap": {},
"title": "Cell 3"
}
},
"outputs": [],
"source": [
"for i in range(len(route_data)):\n",
" print(f\"Route: {route_data[i][1]} | From: {route_data[i][2]} → To: {route_data[i][3]}\")"
]
},
{
"cell_type": "markdown",
"id": "0d2df199",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "55d3b919-ace8-4586-9f93-ae3b06e46e27",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"## Exercise 2: Running SQL Commands\n",
"\n",
Expand All @@ -50,26 +113,46 @@
"\n",
"The result should be a single-row table with at least two columns.\n",
"\n",
"> 💬 **Genie Code tip:** Ask Genie Code:\n",
"> \uD83D\uDCAC **Genie Code tip:** Ask Genie Code:\n",
"> *\"Write a SQL SELECT statement in a Databricks notebook that returns the current timestamp and a string literal as named columns\"*\n",
"> Remember to start your cell with `%sql`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "01a150db",
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"implicitDf": true,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "57a32746-f97d-44ec-9832-fcb1b9f76ebd",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"outputs": [],
"source": [
"# TODO: Use the %sql magic command and write a SQL query\n",
"# Hint: start with %sql on the first line, then write your SELECT statement"
"%sql\n",
"SELECT current_timestamp() AS current_datetime, 'CityMoves Transit' AS system_name\n"
]
},
{
"cell_type": "markdown",
"id": "2a577b30",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "63912b42-24eb-43e2-8b7f-3ad3fc22e35a",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"## Exercise 3: Documenting with Markdown\n",
"\n",
Expand All @@ -83,23 +166,88 @@
"2. A short paragraph (2–3 sentences) describing the purpose of the notebook in the context of CityMoves Transit.\n",
"3. A bulleted list (`-`) of at least three things a reader will find in this notebook.\n",
"\n",
"> 💬 **Genie Code tip:** Select this markdown cell and ask Genie Code:\n",
"> \uD83D\uDCAC **Genie Code tip:** Select this markdown cell and ask Genie Code:\n",
"> *\"Write a Markdown introduction cell for a data engineering notebook about a public transit system called CityMoves Transit\"*\n",
"> Refine the generated text as you see fit."
]
},
{
"cell_type": "markdown",
"id": "974bf85b",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "6d007e6b-f716-477d-9bc1-1c92b36fdd3b",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"# CityMoves Transit — Data Engineering Notebook\n",
"\n",
"This notebook serves as an introductory exploration of the CityMoves Transit data platform built on Azure Databricks. It demonstrates core notebook capabilities that the data engineering team will use daily for route analytics and operational reporting.\n",
"\n",
"- Python code for defining and summarising transit route data\n",
"- SQL queries for retrieving system metadata and timestamps\n",
"- Markdown documentation for clear, shareable notebook structure"
]
},
{
"cell_type": "markdown",
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "f398ad50-590e-48ac-874e-bba7e6217801",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "ee66acac-aec1-451c-a96b-c4498421ece1",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": []
}
],
"metadata": {
"application/vnd.databricks.v1+notebook": {
"computePreferences": null,
"dashboards": [],
"environmentMetadata": {
"base_environment": "",
"environment_version": "5"
},
"inputWidgetPreferences": null,
"language": "python",
"notebookMetadata": {
"mostRecentlyExecutedCommandWithImplicitDF": {
"commandId": 3202781778147245,
"dataframes": [
"_sqldf"
]
},
"pythonIndentUnit": 4
},
"notebookName": "01-explore-azure-databricks (1)",
"widgets": {}
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
"nbformat_minor": 0
}
Loading