Skip to content
Write a review
UdemyBestseller on UdemyPythonPaid courseIntermediate levelCertificate

Spark and Python for Big Data with PySpark (Udemy.com)

Learn how to use Spark with Python, including Spark Streaming, Machine Learning, Spark 2.0 DataFrames and more!

4.6
26,667 reviews

Created by: Jose Portilla

Last updated May 2020

Our take

Based on the ratings of 26,667 students, a sample of their written reviews and the syllabus, as the course stood in May 2020. No course pays to be reviewed.

Jose Portilla's PySpark course is a 10.6 hour tour built around Spark 2.0 DataFrames and MLlib. After setup chapters for Databricks, VirtualBox and AWS, plus a Python crash course, it moves through DataFrame basics and then regression, trees, clustering, recommenders, NLP and Spark Streaming. It has 67 lectures, with mock consulting projects instead of quizzes. In practice it is a data science course that happens to run on Spark, aimed at people who already code.

Students like the teaching a lot. Reviewers call it clear, direct and easy to follow, and one says it gave them confidence with MLlib. The complaints are consistent. The course was last updated in May 2020, and recent reviewers say the Python install steps are stale, Databricks no longer works as shown and some links are dead. Others wanted real data engineering, such as RDDs, cluster management or preparing large data, and found more Python and theory than big data.

Out of about 26,700 ratings, most are four or five stars, so the 4.6 average is earned, and the low scores cluster around age. One reviewer notes Spark itself is mostly the same and the current docs fill the gaps. That makes it a reasonable way to learn the MLlib workflow, as long as you expect to patch the setup yourself. Anyone aiming at Spark 3 or 4, or a data engineering job, should look for something newer.

Best forPython users who want a quick, practical tour of Spark DataFrames and MLlib models before moving on to newer tooling.
Skip it ifYou want current Spark 3 or 4, Databricks as it works today, or real data engineering skills beyond the basics.

Pros

  • Clear, direct teaching that reviewers find easy to follow
  • Walks through MLlib models: regression, trees, k-means, recommenders and NLP
  • Mock consulting projects and a DataFrame exercise give you something to practice on
  • Spark Streaming section and a Python crash course for people new to the language

Cons

  • Last updated May 2020; Databricks steps and Python install are outdated
  • Light on data engineering, with reviewers missing RDDs and large-data prep
  • No quizzes or coding exercises, and some reviewers want more hands-on work
Crisp and clear, condensed, to the point. Delivered essential information in direct and efficient manner.
Harsh Mishra, a student

Labeled intermediate, but it suits anyone who can already program in any language, Python preferred, and needs Spark.

icon
What you will learn

  • Use Python and Spark together to analyze Big Data
  • Learn how to use the new Spark 2.0 DataFrame Syntax
  • Work on Consulting Projects that mimic real world situations!
  • Classify Customer Churn with Logisitic Regression
  • Use Spark with Random Forests for Classification
  • Learn how to use Spark's Gradient Boosted Trees
  • Use Spark's MLlib to create Powerful Machine Learning Models
  • Learn about the DataBricks Platform!
  • Get set up on Amazon Web Services EC2 for Big Data Analysis
  • Learn how to use AWS Elastic MapReduce Service!
  • Learn how to leverage the power of Linux with a Spark Environment!
  • Create a Spam filter using Spark and Natural Language Processing!
  • Use Spark Streaming to Analyze Tweets in Real Time!

Course content

18 sections · 67 lectures · 11 hours of video 4 articles

  1. 1Introduction to Course 1 free preview4 lectures · 30 min
  2. 2Setting up Python with Spark2 lectures · 6 min
  3. 3Databricks Setup2 lectures · 12 min
  4. 4Local VirtualBox Set-up3 lectures · 31 min
  5. 5AWS EC2 PySpark Set-up4 lectures · 39 min
  6. 6AWS EMR Cluster Setup1 lecture · 17 min
  7. 7Python Crash Course7 lectures · 59 min
  8. 8Spark DataFrame Basics 1 free preview7 lectures · 1.1 hours
  9. 9Spark DataFrame Project Exercise2 lectures · 20 min
  10. 10Introduction to Machine Learning with MLlib2 lectures · 19 min

Who it is for

The instructor says it suits

  • Someone who knows Python and would like to learn how to use it for Big Data
  • Someone who is very familiar with another programming language and needs to learn Spark

What you need before you start

  • General Programming Skills in any Language (Preferrably Python)
  • 20 GB of free space on your local computer (or alternatively a strong internet connection for AWS)

How it compares

Against the other 388 rated Udemy courses in Python on CourseDuck:

  • Rating4.6 out of 5higher than 70% of them
  • Reviews26,667#16 of 389 by number of reviews
  • Length10.6 hourslonger than 51% of them
  • Last updatedMay 2020earlier than 76% of them

icon
Course Description

Learn the latest Big Data Technology - Spark! And learn to use it with one of the most popular programming languages, Python!

One of the most valuable technology skills is the ability to analyze huge data sets, and this course is specifically designed to bring you up to speed on one of the best technologies for this task, Apache Spark! The top technology companies like Google, Facebook, Netflix, Airbnb, Amazon, NASA, and more are all using Spark to solve their big data problems!

Spark can perform up to 100x faster than Hadoop MapReduce, which has caused an explosion in demand for this skill! Because the Spark 2.0 DataFrame framework is so new, you now have the ability to quickly become one of the most knowledgeable people in the job market!

This course will teach the basics with a crash course in Python, continuing on to learning how to use Spark DataFrames with the latest Spark 2.0 syntax! Once we've done that we'll go through how to use the MLlib Machine Library with the DataFrame syntax and Spark. All along the way you'll have exercises and Mock Consulting Projects that put you right into a real world situation where you need to use your new skills to solve a real problem!

We also cover the latest Spark Technologies, like Spark SQL, Spark Streaming, and advanced models like Gradient Boosted Trees! After you complete this course you will feel comfortable putting Spark and PySpark on your resume! This course also has a full 30 day money back guarantee and comes with a LinkedIn Certificate of Completion!

If you're ready to jump into the world of Python, Spark, and Big Data, this is the course for you!
Read More Intermediate Intermediate Level product length iconTime: 10.6h

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Get the Udemy discount

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Jose Portilla

Jose Marcial Portilla has a BS and MS in Mechanical Engineering from Santa Clara University and years of experience as a professional instructor and trainer for Data Science, Machine Learning and Python Programming. He has publications and patents in various fields such as microfluidics, materials science, and data science. Over the course of his career he has developed a skill set in analyzing data and he hopes to use his experience in teaching and data science to help other people learn the power of programming, the ability to analyze data, and the skills needed to present the data in clear and beautiful visualizations. Currently he works as the Head of Data Science for Pierian Training and provides in-person data science and python programming training courses to employees working at top companies, including General Electric, Cigna, SalesForce, Starbucks, McKinsey and many more. Feel free to check out the website link to find out more information about training offerings.

Read More

icon
More courses by Jose Portilla

$199.99

$159.99

$124.99

$159.99

$27.99

$109.99

All courses by Jose Portilla

icon
Reviews

4.6

26,667 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

Write a review

By Teodor Todorov on 3/3/2026

Very good explanation of the setup steps. Altough I would not reccomend this course in 2026 as it is pretty outdated from Databricks point of view. Spark is mostly the same and you always have the recent documentation for reference so in this aspect is fine.

By Fabio Priuli on 1/14/2021

I like a lot Josè Portilla's courses and this one is of high quality like the other ones I have completed. I haven't loved the actual activities too much, and in some cases the exploration of the dataset could have been more in-depth, so to lead to more insightful models, but it's a minor complaint which does not change the positive score.

By Francisco Garcia on 11/14/2020

Really really great course!

By Sheila Khumalo on 11/7/2020

So far so good

By Akshaya Prabha on 11/7/2020

Great course. Awesome instructor.

By Graham Jackson on 11/7/2020

The course content is pitched at the right level for me.

By Ghizlane BOUSKRI on 11/7/2020

The most generous instructor. Love this course so far !

By Bruno Delgado on 11/7/2020

The course doesn't show how to use Spark for real world scenarios. It's just a big overview but doesn't really show how to use it as an ETL.

By Cristian on 10/25/2020

honestly? Its a lot of info, but I can't wait the moment, when we see the firts code

By Vitor Magalhaes on 10/24/2020

Awesome course. It takes more than the time there but totally worth. Stop, pause, try and go on. So far probably the second or third best python and ML course and the best spark course in the market.
Setup section takes more than three hours, but you get a lot from there.

By Palaniyappan Thirugnanam on 10/22/2020

I have attended many Spark training, and was keep searching for the perfect course for PySpark to lader into next learning phase, as I have some knowledge already, I could say this course is awesome and in very short time instructor covers plenty of key knowledge for which I was keep learning with hours of hours course meterial and didn't continue. Thanks to Stephane and this is really excellent course with combination of pySpark + AWS .

By Seyfullah Oguz on 10/21/2020

So far, it seems to be a very good choice for my needs.

Showing 12 of 355 reviews

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 92 / 100

Spark and Python for Big Data with PySpark
Visit course on Udemy Write a review
  • Video Duration: icon 10.6h
  • Price: $27.99
  • Provider: icon Udemy
  • Updated: icon May 2020
  • Level: Intermediate Intermediate
  • Language: icon English
  • Certificate: icon Yes
  • Lectures:67 in 18 sections
  • Students:153,100
  • Subtitles:19 languages
  • Udemy badge:Bestseller
30-day money-back guarantee on most paid courses

Udemy Coupon Price Tracker

Udemy price history

For any issues or feature requests please email CourseDuck's Co-Founder at mkuhlman12@gmail.com