Data Engineering Master Course: Spark/Hadoop/Kafka/MongoDB (Udemy.com)
Full Hands on course to become Big Data Engineer: Spark/Kafka/Hadoop/Flume/Hive/Sqoop/MongoDB. Data Engineering course.
Created by: Navdeep Kaur
Last updated May 2025
What you will learn
- Hadoop Ecosystem, Sqoop, Flume, Hive
- Expertise on writing code with Apache Spark
- Learn Kafka Fundamentals and using Kafka Connectors
- Learn writing queries and client in MongoDB
- Learn Data Engineering technologies
How it compares
Against the other 150 rated Udemy courses in Hadoop on CourseDuck:
- Rating4.5 out of 5higher than 68% of them
- Reviews2,219#32 of 151 by number of reviews
- Length12.2 hourslonger than 76% of them
- Last updatedMay 2025more recently than 69% of them
Course Description
In this course, you will start by learning what is hadoop distributed file system and most common hadoop commands required to work with Hadoop File system.
Then you will be introduced to Sqoop Import
Understand lifecycle of sqoop command.
Use sqoop import command to migrate data from Mysql to HDFS.
Use sqoop import command to migrate data from Mysql to Hive.
Use various file formats, compressions, file delimeter,where clause and queries while importing the data.
Understand split-by and boundary queries.
Use incremental mode to migrate the data from Mysql to HDFS.
Further, you will learn Sqoop Export to migrate data.
What is sqoop export
Using sqoop export, migrate data from HDFS to Mysql.
Using sqoop export, migrate data from Hive to Mysql.
Further, you will learn about Apache Flume
Understand Flume Architecture.
Using flume, Ingest data from Twitter and save to HDFS.
Using flume, Ingest data from netcat and save to HDFS.
Using flume, Ingest data from exec and show on console.
Describe flume interceptors and see examples of using interceptors.
Flume multiple agents
Flume Consolidation.
In the next section, we will learn about Apache Hive
Hive Intro
External & Managed Tables
Working with Different Files - Parquet,Avro
Compressions
Hive Analysis
Hive String Functions
Hive Date Functions
Partitioning
Bucketing
You will learn about Apache Spark
Spark Intro
Cluster Overview
RDD
DAG/Stages/Tasks
Actions & Transformations
Transformation & Action Examples
Spark Data frames
Spark Data frames - working with diff File Formats & Compression
Dataframes API's
Spark SQL
Dataframe Examples
Spark with Cassandra Integration
Running Spark on Intellij IDE
Running Spark on EMR
You will learn about Apache Kafka
Kafka Architecture
Partitions and offsets
Kafka Producers and Consumers
Kafka SerDEs
Kafka Messages
Kafka Connector
Ingesting Data using Kafka Connector
You will learn about MongoDB
MongoDB Usecases
CRUD Operations
MongoDB Operators
Working with Arrays
MongoDB with Spark
Data Engineering Interview Preparation
Sqoop Interview Questions
Hive Interview Questions
Spark Interview Questions
Data Engineering common questions
Data Engineering Real project questions.
Instructor Details
- 4.5 Rating
2,219 Reviews
Navdeep Kaur
Navdeep is one of the renowned Premium Instructor at Udemy. Navdeep has 14 years of industry experience in different technologies and domains. With 9+ courses and 60,000+ students and rating of 4.5*, she is one of the leading instructors in the field of Big Data & Cloud.
Happy Learning!
More courses by Navdeep Kaur
Apache Beam | Google Data Flow (Python) | Hands on course (2024)
4.3 (1,110 Reviews)
Provider: Udemy
Time: 3.2h
$10.99
Big Data Analyst -using Sqoop and Advance Hive (CCA159) (2021)
4.3 (361 Reviews)
Provider: Udemy
Time: 5h
$54.99
CCA 175 Exam preparation using Scala & Practice Tests [2020]
4.2 (316 Reviews)
Provider: Udemy
Time: 3.9h
$59.99
Master Apache Spark (Scala) for Data Engineers (2024)
4.4 (127 Reviews)
Provider: Udemy
Time: 4.7h
$64.99
More Hadoop courses
Spark Streaming - Stream Processing in Lakehouse - PySpark (2024)
4.9 (2,251 Reviews)
Provider: Udemy
Time: 22.4h
$11.99
Apache Kafka Series - KSQL on ksqlDB for Stream Processing ! (2026)
4.9 (2,161 Reviews)
Provider: Udemy
Time: 4h
$12.99
Apache Kafka Series - Kafka Streams for Data Processing (2026)
4.8 (6,826 Reviews)
Provider: Udemy
Time: 4.8h
$14.99
Apache Kafka Series - Confluent Schema Registry & REST Proxy (2026)
4.8 (3,794 Reviews)
Provider: Udemy
Time: 4.5h
$94.99
Apache Kafka - Real-time Stream Processing (Master Class) (2023)
4.8 (2,925 Reviews)
Provider: Udemy
Time: 11h
$10.99
Apache Spark 3 & Big Data Essentials in Scala (2024)
4.8 (2,302 Reviews)
Provider: Udemy
Time: 7.3h
$39.99
Reviews
By Zhongliang Zhu on 8/17/2024
This tutor make very clear and detailed explain on the theory part. I love the corresponding chapters very much. It helps me make a good understanding on the fundamental of Spark on Yarn clusters. It helps me understand shuffle, stage and tasks as well. Thanks for the tutor to make this great course. It's very suitable for the beginners to master the data develop skills.
By Anthony Faulise on 9/25/2023
Basically good. Sqoop is deprecated, which may not be obvious to a beginner and should be disclosed in the course description. Wasted a bunch of hours learning a package that is unlikely to be used in practice and that i can no longer even download and install.
By SHIVENDRA SINHA on 3/27/2022
Setups on google cloud has changed now and not as shown in video, that needs a fix as that has completely messed up, Its little risky and challenging as its using credit points and amount. It may charge back as card information is also stored. Not sure, how to remove and delete. This is serious issue, rest is good.
By Rhosung Park on 10/6/2021
Makes it very easy for the beginners. Gives comprehensive understanding from the concept of big data to practices of each big data components. Would recommend.
By Bhanuprakash V. C. on 6/9/2021
Navdeep Kaur, you are a great teacher. You explained almost all of the topics with clarity. I can confidently say now that I know Big data and its processing. Happy to enroll to your course. Thank you.
By PARESH KUMAR Nayak on 4/28/2021
Thank you so much for explaining in detail on various concepts of Hadoop, Scoop, Flume, Hive and Spark. Also thanks a lot for demonstrating hands on exercises. Appreciate it !!
By Ashutosh kawle on 4/11/2021
Thanks Navdeep, I would really appreciate the way you had carried the explanation for each topic , they are very detailed and for newbie like me were easy to grasp. The materials provided made the course helps to relook the things when needed.
By Adrian Avila on 9/23/2020
Good content but it have some errors of typing that if you don't follow step by step, you din't know where happened the error and more than learn one tool you finished searching errors, also it need more edition of video because some videos appears that only was spitted into 2 videos. In general, is a good course...
By Anonymized User on 9/16/2020
I really liked the contents though it can be improved. A beginner with no idea on hadoop/spark/hive or scoop can have a basic idea after going through this course.
By Anonymized User on 7/28/2020
I am new to Big Data Technologies. The videos seem like chopped a bit. Sometimes a video feels like two merged videos. Some videos contain direct usage of commands and APIs and is not good for someone who does not have good idea of Spark..
Quality Score
No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.
Overall Score : 90 / 100


![CCA 175 Exam preparation using Scala & Practice Tests [2020]](/programming/hadoop/images/2112128_f2b9_15.jpg?v=1789443937)







