PYSPARK End to End Developer Course (Spark with Python) (Udemy.com)
Learn PySpark end to end features and functionalities. Course also includes a Python course and HDFS Commands Course.
Created by: Sibaram Nanda
Last updated April 2023
What you will learn
- Complete Development Functionalities and Features of PySpark
- Spark Cluster Execution Architecture
- Spark SQL Architecture
- Spark Performance and Optimization
- Python Course
- HDFS Course
How it compares
Against the other 150 rated Udemy courses in Hadoop on CourseDuck:
- Rating4.4 out of 5higher than 54% of them
- Reviews1,531#45 of 151 by number of reviews
- Length29.1 hourslonger than 95% of them
- Last updatedApril 2023earlier than 50% of them
Course Description
Introduction to Spark.
HDFS Commands
Python Course.
Why Spark was developed.
What is Spark and its features.
Spark Main Components.
Introduction to Spark.
HDFS Commands
Introduction to SparkSession
RDD Fundamentals
What is RDD
RDD Properties
When to use RDD
RDD Problems
Create RDD
Different Ways to Create RDDs
RDD Operations
Transformations - Low Level
Transformations - Join Types
Actions - Total Aggregations
Shuffle and Combiner
Transformations - Key Aggregations
Transformations - Sorting
Transformations - Ranking
Transformations - Set
Transformations - Sampling
Transformations - Partition
Transformations - Repartition
Transformations - Repartition and Sort
Transformations - Coalesce
Transformations - Repartition Vs Coalesce
Extraction
Spark Cluster Execution Architecture_Full Architecture
Spark Cluster Execution Architecture_YARN As Spark Cluster Manager
Spark Cluster Execution Architecture_JVMs across Clusters
Spark Cluster Execution Architecture- Commonly Used Terms in Execution Framework
Spark Cluster Execution Architecture - Narrow and Wide Transformations
Spark Cluster Execution Architecture - DAG Scheduler
Spark Cluster Execution Architecture - Task Scheduler
RDD Persistence
Spark Shared Variables
SparkSQL Architecture
Detailed SparkSession Features
DataFrame Fundamentals
Datatypes
DataFrame Rows
DataFrame Columns
DataFrame ETL
DataFrame ETL_Introduction to Transformations and Extraction
DataFrame ETL_DataFrame APIs Introduction Extraction
DataFrame ETL_DataFrame APIs Selection
DataFrame ETL_DataFrame APIs Filter or Where
DataFrame ETL_DataFrame APIs Sorting
DataFrame ETL_DataFrame APIs Set
DataFrame ETL_DataFrame APIs Join
DataFrame ETL_DataFrame APIs Aggregations
DataFrame ETL_DataFrame APIs GroupBy
DataFrame ETL_DataFrame APIs Windows
DataFrame ETL_DataFrame Built-in Functions Introduction
Performance and Optimization
Instructor Details
- 4.4 Rating
1,531 Reviews
Sibaram Nanda
I have been teaching Spark and Other IT Courses like Snowflake, AWS, relational databases for over 10 Years now. Professionally, I work as a Senior Data Engineer with 14 plus years of experience in various data engineering functions like ETL Solutions, cloud solutions and data-warehousing solutions. I love teaching in a very easy and structured manner. Teaching is my hobby and passion.
More Hadoop courses
Spark Streaming - Stream Processing in Lakehouse - PySpark (2024)
4.9 (2,251 Reviews)
Provider: Udemy
Time: 22.4h
$11.99
Apache Kafka Series - KSQL on ksqlDB for Stream Processing ! (2026)
4.9 (2,161 Reviews)
Provider: Udemy
Time: 4h
$12.99
Apache Kafka Series - Kafka Streams for Data Processing (2026)
4.8 (6,826 Reviews)
Provider: Udemy
Time: 4.8h
$14.99
Apache Kafka Series - Confluent Schema Registry & REST Proxy (2026)
4.8 (3,794 Reviews)
Provider: Udemy
Time: 4.5h
$94.99
Apache Kafka - Real-time Stream Processing (Master Class) (2023)
4.8 (2,925 Reviews)
Provider: Udemy
Time: 11h
$10.99
Apache Spark 3 & Big Data Essentials in Scala (2024)
4.8 (2,302 Reviews)
Provider: Udemy
Time: 7.3h
$39.99
Reviews
By Dinesh Babu Balan on 5/7/2025
While each chapter starting there is more noise and more sound while showing on Aurora Academy title. Trainer explanation voice is too low. so I am using full volume and additional speaker to listen what trainer is speaking after that also low sound Remaining is good.
By Naga Bhushan Reddy on 3/22/2025
good with examples. improvement suggestion : without using HDFS from Linux node, can be shown the examples using complete local set up in windows PC. OR guidance can be given on how to perform those examples in local set up.
By Eswar Gourabathina on 3/6/2025
Very good course to start learning Pyspark. There is a lot of new technology (AWS,DATABRICKS)available in today's world to execute the spark codes. But the technology used in the course is a bit outdated. But this is a very good start for Pyspark and covers all the fundamentals. Once we know all this concepts, we can then learn how to use these Pyspark code in various DE tools that we have.
By Antonio Petreski on 2/7/2025
I think the content of the course is very well structured and easy to understand once you get into it. It starts with some basic and fundamental knowledge and as you progress it gets more and more interesting and complex. The only reason why i haven't given it five starts is because of the background noises like clicks, squeaks etc, which can affect the focus for many people like myself, that can be easily distracted. And the lack of exercises. It would've been a lot better if there were couple of exercises that we can do on our own, to practice the coding part of the course. With that being said, for me this was a great course and i definitively gained a lot of knowledge about Python/Spark and how everything works.
By Veron Reci on 1/21/2025
The chime in the beginning is too loud. No set-up instructions for macOS. The instructor doesn't respond on Q&A, the videos seem to be cut too short sometimes, also can't understand the instructor sometimes as he doesn't pronounce some words correctly.
By Saksham Saxena on 11/25/2023
Awesome course! It is really very help full for beginner as well as others. Suggestion: should add the spark streaming in this course and one project like real-time ingestion records (End-to-End pipeline).
By K Neelakanta on 10/15/2023
Teaching is good but please, update installations videos with updated versions, as it is taking more time between your installation version and current version
By Bob Brandt on 6/19/2023
Speaker is very quiet.. so I have to turn volume WAY up to hear him. Then.. at the beginning of the next video when the music comes on, it BLASTS me because my volume is already turned way up in order to hear instructor. So I have to yank my headphones off so I'm not blasted.. then keep it loud in order to hear instructor. This gets old and sometimes I forget and get blasted!
By Pawan Singh Vohra on 6/7/2023
Amazing course, much recommended to gain in-depth knowledge, after searching for many courses this course is the best I would say.
By Ankit Kumar on 5/30/2023
Need some more clearity on setup Hadoop ecosystem in windows. In video you did every practicel(hadoop) in Linux(cmd prompt). Apart from this every thing is fine.
Quality Score
No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.
Overall Score : 88 / 100






