Submitting a U-SQL Job to Azure Data Lake Analytics
Lab Steps
Ready for the real environment experience?
Description
Big data and analytics in Azure can help you gain insights into your data and deliver better experiences. This Lab demonstrates using two of Azure's big data services:
- Azure Data Lake Store for storing massive amounts of structured, semi-structured, and unstructured data, and
- Azure Data Lake Analytics for massively parallel analysis of data stored in Data Lake Store.
- Azure Data Lake Analytics will be retired on February 29th, 2024. Please see https://aka.ms/adlaqa for more details.
You will use Data Lake Analytics' U-SQL big data query language to transform website search logs stored in a Data Lake Store and discover which searches take longer than a threshold to complete.
Lab Objectives
Upon completion of this Lab you will be able to:
- Store data in Azure Data Lake Stores
- Submit jobs to Azure Data Lake Analytics
- Understand the trade-off between cost and performance in Azure Data Lake Analytics
- Explain basic U-SQL queries
Lab Prerequisites
You should be familiar with:
- Basic SQL knowledge is helpful but not strictly required
- Completion of the Introduction to Azure Data Lake Store and Analytics course is recommended
Lab Environment
Before completing the Lab instructions, the environment will look as follows:
After completing the Lab instructions, the environment should look similar to:
Updates
December 9th, 2022 - Updated the instructions and screenshots to reflect the latest UI and added validation check
September 16th, 2022 - Resolved Azure Data Lake Analytics issue
Logan has been involved in software development and research since 2007 and has been in the cloud since 2012. He is an AWS Certified DevOps Engineer - Professional, AWS Certified Solutions Architect - Professional, Microsoft Certified Azure Solutions Architect Expert, MCSE: Cloud Platform and Infrastructure, Google Cloud Certified Associate Cloud Engineer, Certified Kubernetes Security Specialist (CKS), Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), and Certified OpenStack Administrator (COA). He earned his Ph.D. studying design automation and enjoys all things tech.