Skip to content
UdemyHadoopPaid courseAll LevelsCertificate

Mastering Databricks & Apache spark -Build ETL data pipeline (Udemy.com)

Learn fundamental concept about databricks and process big data by building your first data pipeline on Azure

Created by: Priyank Singh

Last updated August 2021

icon
What you will learn

  • Databricks
  • Build your first data pipeline to process CSV, JSON, XML
  • Orchestrate data pipeline on Azure data factory
  • Spin up spark cluster
  • Delta tables
  • Concept of time travel and vacuum on delta tables
  • Apache Spark SQL
  • Filtering Dataframe
  • Renaming, drop, Select, Cast
  • Aggregation operations SUM, AVERAGE, MAX, MIN
  • Rank, Row Number, Dense Rank
  • Building dashboards
  • Build Complete project
  • Build End to End data pipeline

How it compares

Against the other 150 rated Udemy courses in Hadoop on CourseDuck:

  • Rating4.2 out of 5higher than 34% of them
  • Reviews437#73 of 151 by number of reviews
  • Length4.4 hoursshorter than 66% of them
  • Last updatedAugust 2021earlier than 59% of them

icon
Course Description

Welcome to the course on Mastering Databricks & Apache spark -Build ETL data pipeline

Databricks combines the best of data warehouses and data lakes into a lakehouse architecture. In this course we will be learning how to perform various operations in Scala, Python and Spark SQL. This will help every student in building solutions which will create value and mindset to build batch process in any of the language. This course will help in writing same commands in different language and based on your client needs we can adopt and deliver world class solution. We will be building end to end solution in azure databricks.


Key Learning Points

  • We will be building our own cluster which will process our data and with one click operation we will load different sources data to Azure SQL and Delta tables

  • After that we will be leveraging databricks notebook to prepare dashboard to answer business questions

  • Based on the needs we will be deploying infrastructure on Azure cloud

  • These scenarios will give student 360 degree exposure on cloud platform and how to step up various resources

  • All activities are performed in Azure Databricks


Fundamentals

  • Databricks

  • Delta tables

  • Concept of versions and vacuum on delta tables

  • Apache Spark SQL

  • Filtering Dataframe

  • Renaming, drop, Select, Cast

  • Aggregation operations SUM, AVERAGE, MAX, MIN

  • Rank, Row Number, Dense Rank

  • Building dashboards

  • Analytics

This course is suitable for Data engineers, BI architect, Data Analyst, ETL developer, BI Manager

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Priyank Singh

Technically sophisticated and business savvy who can play with data having 10+ years of experience and documented history of bridging technical and management acumen in turning around of information technology. Priyank has spent more than 10 years in consulting for building data related processes. Worked with various banking, logistic and insurance companies to process heterogenous amount of big data. Continuously bridging the gaps and deriving values from building data process


icon
More Hadoop courses

$11.99

$12.99

$14.99

$94.99

$10.99

$39.99

icon
Reviews

4.2

437 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

By Daniel Sepp on 3/24/2023

The course proved to be immensely enriching, perfectly aligning with my interests and effectively augmenting my technical prowess and comprehension of computational intelligence. I deeply appreciate the professor's outstanding teaching approach, which fostered an engaging and nurturing learning atmosphere—truly a treasured experience. Gaining hands-on exposure to Databricks and Azure was proved to be an eye-opening and enlightening experience, reinforcing the course's indisputable value. I'm already applying the acquired knowledge to implement solutions at work, I remain grateful for the unparalleled guidance offered throughout this transformative educational journey. Professor Priyank Singh, warmest greetings from Brazil.

By Rens S on 3/18/2023

The course gives a good practical step-by-step example of a simple ETL workflow with databricks in Azure. Really good for beginners that already have some knowledge on Spark and databricks and want to see it in action. However, if you are looking for a course to understand databricks/Spark/Azure concepts, theory or architecture then this is definitely not the course for you. Also, it was unfortunate that most of the code was written in Scala instead of Python. This was not mentioned in the course description

By Vijay More on 10/2/2022

Definitely not a mastering course, good for beginners. Bit more explanation on certain topics were expected. I was also doing python learning course on udemy in parallel to this course , trainer was more vocal and keeps you more interested in the course there, that was missing here. Could have been better!

By Tung Dinh _ on 7/4/2022

Advantage: get familiar to databrick, nothing else. Very basic Spark. Disadvantage: no detailed explanation, minimal conversation. Not all the code is converted to Python, instructor is familiar to Scala. To my best knowledge, this course is only suitable for playing around, not professional code. Absolutely no mastering at all.

By Ranjita on 9/19/2021

It has been a good starting course for anyone like me who are new to ETL Pipeline in Azure. Following are the parameters based on which review is based: 1. Course structure and content: Minimum theory and maximum practical work. Liked the ETL end-to-end pipeline the most!! 2. Instructor: Good step by step explanations. Easy to follow course work. Looking forward to the advanced level of this course from him.

By Ranjitcdev on 7/24/2021

This course gives a good initial understanding of Azure databricks working. The example shown is good enough to grasp the basic concepts of spark and azure databricks. This is a good course for any one who is about to start some project in Azure databricks and quickly want to get hold of basics.

By Jhanvi verma on 4/17/2021

Thank you for this fundamental yet detailed course on Databricks. This was my introduction to this software and the content included was totally appropriate and helpful for my basic understanding of the same. This course depicts the capabilities of this tool in a smooth way, the interactive notebooks and workspaces, highly optimized processing of data definitely motivates me to explore more. The step-by-step demonstration for setting up the resources, databases, containers, processing data using all the supported languages, loading onto delta tables and Azure SQL, developing simple dashboard gives a clear overview of the lifecycle of data analytics project development using Databricks. This course was perfectly paced for a fresher like me. Looking forward for many such courses from the Instructor. It was a great learning experience.

By Donald Jones on 4/11/2021

I am really excited to use this skill to make an impact in my future projects. Simple dataset always helps me in understanding the concept in much better way. Simple and precise are the two words that I have for this course. I really don’t like lengthy course which are 10 hrs or 15 hrs. This course checks all criteria’s that I have and totally worth the money. I like the passion that author have to teach. This course does talk about building something end to end.

By Chadi Diab on 4/6/2021

This course can be a lot better. The instructor at times presented the materials poorly and some times it felt very robotic. The content can be enriched with more useful stuff. I didn't have much trouble because I have some databricks experience and this was more like a refreshing course for me. Overall it's not bad but it definitely can be a lot better...

By Shivankar Singh on 3/21/2021

I'm transitioning into a tech career and have found this course helpful. Part of me wants to jump ahead to the other courses in the specialization but I have been learning more about the basics which is good. Good pace in explaining different topics and nice choice of colors while presenting. I will encourage author to create more advanced materials for us. I really like the part of processing same dataset in different language. Keep teaching.

Showing all 10 reviews on CourseDuck

Read all 437 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 84 / 100