Skip to content
UdemyHadoopPaid courseAll LevelsCertificate

Data Engineering Master Course: Spark/Hadoop/Kafka/MongoDB (Udemy.com)

Full Hands on course to become Big Data Engineer: Spark/Kafka/Hadoop/Flume/Hive/Sqoop/MongoDB. Data Engineering course.

Created by: Navdeep Kaur

Last updated May 2025

icon
What you will learn

  • Hadoop Ecosystem, Sqoop, Flume, Hive
  • Expertise on writing code with Apache Spark
  • Learn Kafka Fundamentals and using Kafka Connectors
  • Learn writing queries and client in MongoDB
  • Learn Data Engineering technologies

How it compares

Against the other 150 rated Udemy courses in Hadoop on CourseDuck:

  • Rating4.5 out of 5higher than 68% of them
  • Reviews2,219#32 of 151 by number of reviews
  • Length12.2 hourslonger than 76% of them
  • Last updatedMay 2025more recently than 69% of them

icon
Course Description

In this course, you will start by learning what is hadoop distributed file system and most common hadoop commands required to work with Hadoop File system.


Then you will be introduced to Sqoop Import

  • Understand lifecycle of sqoop command.

  • Use sqoop import command to migrate data from Mysql to HDFS.

  • Use sqoop import command to migrate data from Mysql to Hive.

  • Use various file formats, compressions, file delimeter,where clause and queries while importing the data.

  • Understand split-by and boundary queries.

  • Use incremental mode to migrate the data from Mysql to HDFS.


Further, you will learn Sqoop Export to migrate data.

  • What is sqoop export

  • Using sqoop export, migrate data from HDFS to Mysql.

  • Using sqoop export, migrate data from Hive to Mysql.



Further, you will learn about Apache Flume

  • Understand Flume Architecture.

  • Using flume, Ingest data from Twitter and save to HDFS.

  • Using flume, Ingest data from netcat and save to HDFS.

  • Using flume, Ingest data from exec and show on console.

  • Describe flume interceptors and see examples of using interceptors.

  • Flume multiple agents

  • Flume Consolidation.


In the next section, we will learn about Apache Hive

  • Hive Intro

  • External & Managed Tables

  • Working with Different Files - Parquet,Avro

  • Compressions

  • Hive Analysis

  • Hive String Functions

  • Hive Date Functions

  • Partitioning

  • Bucketing


You will learn about Apache Spark

  • Spark Intro

  • Cluster Overview

  • RDD

  • DAG/Stages/Tasks

  • Actions & Transformations

  • Transformation & Action Examples

  • Spark Data frames

  • Spark Data frames - working with diff File Formats & Compression

  • Dataframes API's

  • Spark SQL

  • Dataframe Examples

  • Spark with Cassandra Integration

  • Running Spark on Intellij IDE

  • Running Spark on EMR


You will learn about Apache Kafka

  • Kafka Architecture

  • Partitions and offsets

  • Kafka Producers and Consumers

  • Kafka SerDEs

  • Kafka Messages

  • Kafka Connector

  • Ingesting Data using Kafka Connector

You will learn about MongoDB

  • MongoDB Usecases

  • CRUD Operations

  • MongoDB Operators

  • Working with Arrays

  • MongoDB with Spark


Data Engineering Interview Preparation

  • Sqoop Interview Questions

  • Hive Interview Questions

  • Spark Interview Questions

  • Data Engineering common questions

  • Data Engineering Real project questions.




icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Navdeep Kaur

Navdeep is one of the renowned Premium Instructor at Udemy. Navdeep has 14 years of industry experience in different technologies and domains. With 9+ courses and 60,000+ students and rating of 4.5*, she is one of the leading instructors in the field of Big Data & Cloud.

Happy Learning!

icon
More courses by Navdeep Kaur

$10.99

$54.99

$59.99

$64.99

icon
More Hadoop courses

$11.99

$12.99

$14.99

$94.99

$10.99

$39.99

icon
Reviews

4.5

2,219 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

By Zhongliang Zhu on 8/17/2024

This tutor make very clear and detailed explain on the theory part. I love the corresponding chapters very much. It helps me make a good understanding on the fundamental of Spark on Yarn clusters. It helps me understand shuffle, stage and tasks as well. Thanks for the tutor to make this great course. It's very suitable for the beginners to master the data develop skills.

By Anthony Faulise on 9/25/2023

Basically good. Sqoop is deprecated, which may not be obvious to a beginner and should be disclosed in the course description. Wasted a bunch of hours learning a package that is unlikely to be used in practice and that i can no longer even download and install.

By SHIVENDRA SINHA on 3/27/2022

Setups on google cloud has changed now and not as shown in video, that needs a fix as that has completely messed up, Its little risky and challenging as its using credit points and amount. It may charge back as card information is also stored. Not sure, how to remove and delete. This is serious issue, rest is good.

By Rhosung Park on 10/6/2021

Makes it very easy for the beginners. Gives comprehensive understanding from the concept of big data to practices of each big data components. Would recommend.

By Bhanuprakash V. C. on 6/9/2021

Navdeep Kaur, you are a great teacher. You explained almost all of the topics with clarity. I can confidently say now that I know Big data and its processing. Happy to enroll to your course. Thank you.

By PARESH KUMAR Nayak on 4/28/2021

Thank you so much for explaining in detail on various concepts of Hadoop, Scoop, Flume, Hive and Spark. Also thanks a lot for demonstrating hands on exercises. Appreciate it !!

By Ashutosh kawle on 4/11/2021

Thanks Navdeep, I would really appreciate the way you had carried the explanation for each topic , they are very detailed and for newbie like me were easy to grasp. The materials provided made the course helps to relook the things when needed.

By Adrian Avila on 9/23/2020

Good content but it have some errors of typing that if you don't follow step by step, you din't know where happened the error and more than learn one tool you finished searching errors, also it need more edition of video because some videos appears that only was spitted into 2 videos. In general, is a good course...

By Anonymized User on 9/16/2020

I really liked the contents though it can be improved. A beginner with no idea on hadoop/spark/hive or scoop can have a basic idea after going through this course.

By Anonymized User on 7/28/2020

I am new to Big Data Technologies. The videos seem like chopped a bit. Sometimes a video feels like two merged videos. Some videos contain direct usage of commands and APIs and is not good for someone who does not have good idea of Spark..

Showing all 10 reviews on CourseDuck

Read all 2,219 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 90 / 100