Azure Data Factory For Data Engineers - Project on Covid19 (Udemy.com)
Real world project for Data Engineers using Azure Data Factory, SQL, Data Lake, Databricks, HDInsight, CI/CD [DP203]
Created by: Ramesh Retnasamy
Last updated March 2026
Our take
Based on the ratings of 18,072 students, a sample of their written reviews and the syllabus, as the course stood in March 2026. No course pays to be reviewed.
This is a project course, not a feature tour. Ramesh Retnasamy, a working senior data engineer, builds one pipeline across 12.7 hours and 145 lectures: pulling Covid-19 case and hospital data through Azure Data Factory, transforming it with Data Flows, Databricks, and HDInsight, loading it into Azure SQL, then wiring up monitoring, Power BI reporting, and CI/CD through Azure DevOps. The syllabus leans toward people who already know some IT or cloud basics and want to see a production-style pipeline assembled end to end.
Reviewers consistently call the explanations clear and well organized, and at least one student singles out the CI/CD section as unusually thorough for a course at this level. The hands-on structure gets credit for making concepts stick rather than just being watched. The complaints cluster in two places: a few beginners say the pace and delivery lost them early on, and at least one longtime student flagged that the Covid dataset and the Databricks-to-storage connection could use a refresh to reflect newer tooling like Unity Catalog.
At 4.6 stars across over 18,000 ratings, the course has stayed reliable for years, and the March 2026 update plus the CI/CD sections added in late 2022 show it gets maintained rather than left to rot. For anyone comparing Azure Data Factory options on Udemy, this one earns its bestseller badge on the strength of its DevOps and monitoring sections. Worth it if a real pipeline, not just slides, is what you're after.
Pros
- CI/CD setup in Azure DevOps singled out as excellent by a reviewer
- One real Covid-19 project ties Data Factory, Databricks, and HDInsight together
- Includes monitoring and Power BI reporting on top of the pipeline build
- Updated as recently as March 2026, with CI/CD content added in 2022
Cons
- A few reviewers say the delivery is hard to follow for true beginners
- Covid dataset and Databricks connection method are getting dated
- No coding exercises or practice tests, just lecture and follow-along
Great presentation, very organized, has made it very easy to follow. Has good amount of hands-on which makes the subject stick for longer periods.
Listed as all levels, but reviewers say it suits people with some IT or cloud background, not total newcomers.
What you will learn
- You will learn how to build a real-world data pipeline in Azure Data Factory (ADF).
- You will acquire good Data Engineering skills in Azure using Azure Data Factory (ADF), Azure Data Lake Storage Gen2, Azure SQL Database and Azure Monitor
- You will learn how to ingest data from sources such as HTTP and Azure Blob Storage into Azure Data Lake Gen2 using Azure Data Factory (ADF)
- You will learn how to transform data using Data Flows in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to transform data using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to transform data using Azure HDInsight Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to load transformed data from Azure Data Lake Storage Gen2 to Azure SQL Database using Azure Data Factory (ADF)
- You will learn extensively about Triggers in Azure Data Factory (ADF) and how to use them to schedule the data pipelines.
- You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor and Log Analytics with a real-world project.
- You will learn how to build production ready pipelines and good practices and naming standards
- You will learn the topics required on Azure Data Factory to pass the Azure Data Engineer Associate Certification Exam DP203
- You will learn about how to create CI/CD pipelines in Azure Devops to release ADF pipelines to higher environments (Testing/ Production)
Course content
17 sections · 145 lectures · 13 hours of video 1 quiz, 20 articles
- 1Introduction 2 free previews3 lectures · 8 min
- 2Overviews 3 free previews5 lectures · 23 min
- 3Environment Set-up 1 free preview9 lectures · 34 min
- 4Data Ingestion from Azure Blob 3 free previews12 lectures · 1.2 hours
- 5Data Ingestion From HTTP 1 free preview12 lectures · 1.1 hours
- 6Data Flows - Cases & Deaths Data Transformation 2 free previews12 lectures · 1.3 hours
- 7Data Flows - Hospital Admissions Data Transformation 1 free preview15 lectures · 1.2 hours
- 8Prepare Data for HDInsight & Databricks1 lecture · 8 min
- 9HDInsight Activity 1 free preview9 lectures · 40 min
- 10Databricks Activity 1 free preview9 lectures · 44 min
- 11Copy Data to Azure SQL 1 free preview5 lectures · 23 min
- 12Making Pipelines Production Ready4 lectures · 30 min
- 13Monitoring 1 free preview12 lectures · 59 min
- 14Power BI Reports 1 free preview4 lectures · 1 quiz · 16 min
- 15Continuous Integration / Continuous Delivery (CI/CD) 2 free previews22 lectures · 2.4 hours
- 16CI/CD Scenario - Data Lake Access9 lectures · 47 min
- 17Conclusion2 lectures · 1 min
Who it is for
The instructor says it suits
- University students looking for a career in Data Engineering
- IT developers working on other disciplines trying to move to Data Engineering
- Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Technologies
- Data Architects looking to gain an understanding about Azure Data Engineering stack
- Data Scientists who want extend their knowledge into data engineering
What you need before you start
- Basic understanding about cloud computing will be useful, but not necessary.
- Experience in Azure is not required, I will take you through everything necessary to learn this course and build the project
- An Azure Account is required, If you don't have one we will create a free account in the course
How it compares
Against the other 70 rated Udemy courses in Cloud Computing on CourseDuck:
- Rating4.6 out of 5higher than 64% of them
- Reviews18,072#3 of 71 by number of reviews
- Length12.7 hourslonger than 78% of them
- Last updatedMarch 2026more recently than 81% of them
Course Description
Major updates to the course since the launch
October 2023 - Updates related to UI changes to Storage Browser & Azure Data Factory. Renaming of Azure Active Directory to Microsoft Entra ID & Default settings changes to Devops Organisation
January 2023 - Updates to section 3 (Environment Set-up) to reflect the change to the User Interface. Re-recorded 5 lessons.
November 2022 - Addition of sections 15 & 16 focusing on Continuous Integration & Continuous Delivery (CI/CD)
Welcome!
I am looking forward to helping you with learning one of the in-demand data engineering tools in the cloud, Azure Data Factory (ADF)! This course has been taught with implementing a data engineering solution using Azure Data Factory (ADF) for a real world problem of reporting Covid-19 trends and prediction of the spread of this virus.
This is like no other course in Udemy for Azure Data Factory or Data Engineering Technologies. Once you have completed the course including all the assignments, I strongly believe that you will be in a position to start a real world data engineering project on your own and also proficient on Azure Data Factory (ADF).
I have also included lessons on the storage solutions such as Azure Data Lake Storage, Azure Blob Storage, Azure SQL Database etc. Also, there are lessons on Azure HDInsight and Azure Databricks. I have even included lessons on building reports using Power BI on the data processed by the Azure Data Factory data pipelines. I have considered the machine learning models to be out of scope. You can use this data to build your own models and predict the spread.
The course follows a logical progression of real world project implementation with technical concepts being explained and the data pipelines in Azure Data Factory (ADF) being built at the same time. Even-though this course is not specifically designed to teach you the skills required for passing the Azure Data Engineer Associate Certification exam DP203, it can greatly help you get most of the necessary skills required for the exam.
I value your time as much as I do mine. So, I have designed this course to be fast-paced and to the point. Also, the course has been taught with simple English and no jargons. I start the course from basis and by the end of the course you will be proficient in the technologies used.
Currently the course teaches you the following
Azure Data Factory
Building a solution architecture for a data engineering solution using Azure Data Engineering technologies such as Azure Data Factory (ADF), Azure Data Lake Gen2, Azure Blob Storage, Azure SQL Database, Azure Databricks, Azure HDInsight and Microsoft PowerBI.
Integrating data from HTTP clients, Azure Blob Storage and Azure Data Lake Gen2 using Azure Data Factory.
Branching and Chaining activities in Azure Data Factory (ADF) Pipelines using control flow activities such as Get Metadata. If Condition, ForEach, Delete, Validation etc.
Using Parameters and Variables in Pipelines, Datasets and LinkedServices to create a metadata driven pipelines in Azure Data Factory (ADF)
Debugging the data pipelines and resolving issues.
Scheduling pipelines using triggers such as Event Trigger, Schedule Trigger and Tumbling Window Trigger in Azure Data Factory (ADF)
Creating Mapping Data Flows to create transformation logic. The course covers all of the transformation steps such as Source, Filter, Select, Pivot, Lookup, Conditional Split, Derived Column, Aggregate, Join and Sink transformation.
Debugging data flows, investigating issues, fixing failures etc
Implementing Azure Data Factory pipelines to invoke Mapping Data Flows and executing them.
Creating ADF pipelines to execute HDInsight activities and carry out data transformations.
Creating ADF pipelines to execute Databricks Notebook activities to carry out transformations.
Creating dependency between pipelines to orchestrate the data flow
Creating dependency between triggers to orchestrate the data flow
Monitoring data pipelines, creating alerts, reporting of metrics from the Azure Data Factory Monitor.
Monitoring of Data Factory pipelines using Azure Monitor and setting diagnostic setting to be forwarded to Azure Storage Account or Log Analytics Workspace.
Creating Log Analytics workspace, creating workbooks and charts from log analytics on the Azure Data Factory pipelines
Implementing the Azure Data Factory Analytics monitoring tool and how to extend the capability further.
Azure Storage Solutions
Creating Azure Storage Account, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account
Creating Azure Data Lake Gen2, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account
Creating Azure SQL Database, Pricing Tiers, Creating Admin User, Creating Tables, Loading Data and Querying the database.
Azure HDInsight & Databricks
Creating HDInsight Clusters, Interacting with the UI, Using Ambari, Creating Hive tables, Invoking HDInsight activities from Azure Data Factory
Creating Azure Databricks Workspace, Creating Databricks clusters, Mounting storage accounts, Creating Databricks notebooks, performing transformations using Databricks notebooks, Invoking Databricks notebooks from Azure Data Factory.
Azure Devops (CI/CD)
Creating Azure Devops Environment and configuring Azure Devops Git Repository
CI/ CD process for releasing Azure Data Factory artefacts to higher environments
Creating build and release pipelines in Azure Devops to release code to higher environments (Test/ Prod)
Configuring/ Parameterise CI/CD pipelines to release ADF pipelines that access Azure Data Lake Storage.
Instructor Details
- 4.6 Rating
18,072 Reviews
Ramesh Retnasamy
Hello! I am a full-time Senior Data Engineer and Architect with over 25 years of experience delivering large scale data projects across industries including technology, gaming, finance, retail, and government.
I have worked extensively on cloud platforms such as Azure and AWS, as well as on-premises systems. I hold multiple certifications, including Data Engineering credentials from Microsoft Azure and Databricks.
Teaching is a passion of mine, and I take great pride in the success of my students. My approach is different from traditional IT training. I focus on real world projects that not only explain the concepts but also help them stick through hands-on application. I provide guidance on best practices and help you build solutions that are ready for real production environments.
I value your time as much as mine, so I keep my courses focused, practical, and free of jargon. All lessons are taught in simple English. By the end of my courses, you will have the skills and confidence to begin working on real projects, along with a strong foundation to continue learning and growing in your career.
More courses by Ramesh Retnasamy
Azure Databricks & Spark for Data Engineers:Hands-on Project (2026)
4.7 (28,183 Reviews)
Provider: Udemy
Time: 31.8h
$17.99
Original priceDatabricks Certified Data Engineer Associate - Ultimate Prep (2026)
4.7 (5,064 Reviews)
Provider: Udemy
Time: 21.9h
$11.99
Original priceAzure Synapse Analytics For Data Engineers -Hands On Project (2025)
4.4 (4,267 Reviews)
Provider: Udemy
Time: 13.5h
$11.99
Original priceDatabricks Certified Data Engineer Associate- Practice Exams (2026)
4.5 (1,639 Reviews)
Provider: Udemy
$11.99
Original priceComprehensive SQL Course For Data Professionals (2025)
4.6 (1,105 Reviews)
Provider: Udemy
Time: 13.7h
$11.99
Original price
More Cloud Computing courses
Build and Deploy Responsive Websites on AWS using HTML & CSS (2023)
4.8 (113 Reviews)
Provider: Udemy
Time: 13.4h
$11.99
Original priceMicrosoft Azure: From Zero to Hero - The Complete Guide (2026)
4.7 (23,269 Reviews)
Provider: Udemy
Time: 21.2h
$17.99
Original priceGCP Professional Cloud Architect: Google Cloud Certification (2026)
4.7 (15,148 Reviews)
Provider: Udemy
Time: 33.4h
$19.99
Original priceGCP for AWS Professionals - Learn Google Cloud [2021]
4.7 (733 Reviews)
Provider: Udemy
Time: 7.5h
$11.99
Original priceCloud Computing for Beginners - Database Technologies (2025)
4.7 (685 Reviews)
Provider: Udemy
Time: 2.4h
$10.99
Original priceDevOps Beginner Prerequisites | Cloud Computing Pre-reqs (2025)
4.7 (390 Reviews)
Provider: Udemy
Time: 14h
$11.99
Original price
Reviews
By Allan BIVEGHE on 9/3/2026
Pour l'instant ça va mais j'ai perdu le film sur le role de ADF car trop d'informations à la fois pour une première en tant que debutant
By Bhushan Oulkar on 8/7/2026
Fanstactic Explanation I admire bought two from the same person . Sir really felt very happy about yeour teaching but small request please include Delta live tables,Declarative pipeline and streaming data explanation in databricks cource please
By Veda Gaonkar on 7/17/2026
Great presentation, very organized, has made it very easy to follow. Has good amount of hands-on which makes the subject stick for longer periods. I found it helpful, would recommend it.
By Steven Ing on 6/1/2026
Good foundational course on Azure Data Factory. Improvements to the course would be to update the course to use the latest ECDC Covid 19 data and to update the Azure-to-Databricks connection to use Unity Catalog.
By Raghuram Bodicharla on 4/21/2026
Explanation about Architecture is good. Along with Architecture explanation I am able to corelate to real time project experience
By Varun Goyal on 3/17/2026
Nice exhaustive course for a beginner but very well explained by the instructor. Though sometimes it was too fast to digest.
By Abdulkarim Habib on 1/18/2026
Good explanation and on hands course, i feel comfortable in understanding all the concepts and able to execute all exercises.
By Alexander Watts-Barnett on 12/28/2025
Like all of Ramesh's (this instructor's) courses this Azure Data Factory course is very well organized and thorough, providing an easy-to-follow structure with plenty of detailed explanations to help the learner understand the process. Audio, picture quality and structure are great and the course materials are very useful. The reason I took this course was because I was put on a project at work that required knowledge of Azure Data Factory and in many respects the actual project I was working on was very similar to the project we build in this course, so it was a huge help. If I had to suggest improvements I'd say that more attention could be spent on parameterization and conditional pipelines. Also in my case I was never able to get Microsoft to sign off on my parallelism so I was not able to do some of the CI/CD exercises. Luckily I had experience deploying pipelines through Git Hub so I could simply watch the lectures and follow the process mentally. I think it might be useful if the GIT Hub method for version control and CI/CD were also demonstrated as it requires no parallelism requirement from Microsoft yet follows a similar structure (though I'm not sure how likely one is to see that in a production environment) -- but these are just suggestions to make to add to an already excellent course. Great experience overall.
By Jose Antonio Delgado Guerra on 12/14/2025
El curos cubre los aspectos mas relevantes de ciclo de vide de aplicaciones en ADF asi como varios escenarios que son comunes en el ambiente real, creo que cubre todos los aspectos y deja al lector con elementos suficientes para ser prodcutivo en estas herramientas.
By Julian Addington-Barker on 11/25/2025
I thought that this was a really good comprehensive course to learn Azure Data Factory. My only criticism was the CI/CD section which was quite long and for me not as interesting as the rest of the course but, I understand that this is a necessary aspect of Data Engineering, even so this is still a very useful and informative section of the course.
Quality Score
No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.
Overall Score : 92 / 100








![GCP for AWS Professionals - Learn Google Cloud [2021]](/programming/cloud-computing/images/3917678_80c0.jpg?v=1790068180)


