Skip to content
UdemyHadoopPaid courseAll LevelsCertificate

PySpark Essentials for Data Scientists (Big Data + Python) (Udemy.com)

Learn how to wrangle Big Data for Machine Learning using Python in PySpark taught by an industry expert!

Created by: Layla AI

Last updated May 2022

icon
What you will learn

  • Use Python with Big Data on a distributed framework (Apache Spark)
  • Work with REAL datasets on realistic consulting projects
  • How to streaming LIVE data from Twitter using Spark Structured Streaming
  • Learn how to create a "Pandora Like" app that classifies songs into genres using machine learning
  • Flag suspicious job postings using Natural Language Processing
  • Use machine learning to predict optimal cement strength and the factors that affect it
  • Classify Christmas cooking recipes using Topic Modeling (LDA)
  • Customer Segmentation using Gaussian Mixture Modeling (Clustering)
  • Use cluster analysis to develop a strategy designed to increase college graduation rates for under-priveleged populations
  • How to use the k-means clustering algorithm to define a marketing outreach strategy
  • Integrate a UI to monitor your model training and development process with MLflow

How it compares

Against the other 150 rated Udemy courses in Hadoop on CourseDuck:

  • Rating4.4 out of 5higher than 54% of them
  • Reviews833#58 of 151 by number of reviews
  • Length17.3 hourslonger than 86% of them
  • Last updatedMay 2022earlier than 56% of them

icon
Course Description

This course is for data scientists (or aspiring data scientists) who want to get PRACTICAL training in PySpark (Python for Apache Spark) using REAL WORLD datasets and APPLICABLE coding knowledge that you’ll use everyday as a data scientist! By enrolling in this course, you’ll gain access to over 100 lectures, hundreds of example problems and quizzes and over 100,000 lines of code!

I’m going to provide the essentials for what you need to know to be an expert in Pyspark by the end of this course, that I’ve designed based on my EXTENSIVE experience consulting as a data scientist for clients like the IRS, the US Department of Labor and United States Veterans Affairs.

I’ve structured the lectures and coding exercises for real world application, so you can understand how PySpark is actually used on the job. We are also going to dive into my custom functions that I wrote MYSELF to get you up and running in the MLlib API fast and make getting started building machine learning models a breeze! We will also touch on MLflow which will help us manage and track our model training and evaluation process in a custom user interface that will make you even more competitive on the job market!

Each section will have a concept review lecture as well as code along activities structured problem sets for you to work through to help you put what you have learned into action, as well as the solutions to each problem in case you get stuck. Additionally, real world consulting projects have been provided in every section with AUTHENTIC datasets to help you think through how to apply each of the concepts we have covered.

Lastly, I’ve written up some condensed review notebooks and handouts of all the course content to make it super easy for you to reference later on. This will be super helpful once you land your first job programming in PySpark!

I can’t wait to see you in the lectures! And I really hope you enjoy the course! I’ll see you in the first lecture!

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Layla AI

Layla AI is quickly becoming one of Udemy's leading female instructors in the data science realm. She began her career as a data scientist in 2012 while earning her masters degree in Quantitative Analytics and has been a federal consultant since 2016 for clients like the IRS, Veterans Affairs and Department of Labor.

Her skills are most predominantly in predictive modeling, artificial intelligence, natural language processing, topic model, trend analysis, frequent pattern mining, machine-learning, deep-learning, cluster analysis and began teaching in 2020.

Her primary programming language is Python but she also has extensive experience with non-object oriented languages like SAS and SQL.

Most notably however, she is a passionate teacher who loves to share her knowledge with the world!

icon
More Hadoop courses

$11.99

$12.99

$14.99

$94.99

$10.99

$39.99

icon
Reviews

4.4

833 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

By Jorge Castillo on 4/8/2022

I have been trying to learn pyspark, but most of the material makes you believe that there is something mysterious about it, until I started with this course where the comparison between python and pyspark shows the why and makes the mystery to become something simple to understand. Great course.

By Thomas Souliotis on 9/30/2021

I found it hard as a beginner to follow the course. Many things regarding the workflow and approach were not explained but rather just being coded. Some times it felt like just showing code. There is a lot of supporting material provided but i firmly believe that some time of the course showing code could be replaced by providing explanations. (i.e lot of spark documentation links but not a single video going on it and showing how to look). Still very good course!!!

By Arvin Flores on 6/21/2021

Good: I like the teaching, feel homey also the Q&A is pretty active so far if you're really a beginner in Python, you should watch a few videos first about Python and Pandas in freecodecamp then go back here, you'll probably have a smoother journey, if you already have that programming mindset before you try this course. Also don't update to Java 8 291. It doesn't go well with the latest Pyspark version as of June 22 2021, so probably stay at 281.

By Maria Papadaki on 3/27/2021

I have completed the Dataframe Essentials sections that are needed for my job at the present moment. The course is fantastic and because of it, I got quickly familiar with the concepts needed to do my job and adjust in no time to Apache Spark framework. Soon, I will also watch the MLlib sections and update the rating accordingly.

By Adrian Musiał on 3/1/2021

The course is incredibly comprehensive and after finishing most of it, I feel like I can use PySpark really well. I especially enjoyed the projects - they were interesting and quite challenging. The instructor is not only knowledgeable, but also likable and unpretentious. Overall, a great course.

By Bharat Ruparel on 2/10/2021

I just finished going through most of this course. I knew I was on a good start with this course when I could install and work with PySpark without any issues following the video. I think Layla has put in a lot of work in this course which clearly shows. Well done. One note of caution though. This is not a beginner's course unless you are well motivated and resourceful. I have experience with Both Spark and Machine Learning (Sci-kit Learn, NLTK), so I was fine. People who are rank beginners will do well to supplement this course with other courses. This course is well suited for intermediate to advanced students.

By Purnima Sharma on 1/22/2021

This course by Laila is the most exhaustive course that covers all aspects of machine Learning in PySpark. The course is very well organized: starting from fundamentals to advance concepts and then coding exercises. I learnt a lot in this course. The projects are very well designed and practical. She has provided functions which are very useful and can be used as such in your company’s projects. The course got a bit fast paced during the ML part and lots of concepts were covered in short videos. I had to look for additional resources to understand them completely. That happens with courses even in the college. Another thing worth mentioning is that she was very prompt in responding to my questions. Overall, it is a great course, and I would recommend it to anyone who wants to learn machine learning using PySpark.

By Daryl MItchell on 1/6/2021

This has been a wonderful class. The PySpark piece has been excellent and allowed me to review my Data Science group's DataBricks notebooks, and clean them up significantly to become nightly "production" runs. The material is complete, concise and very well presented. I am skipping over the ML stuff for now as I don't have a need for this at the moment but look forward to getting back to them as soon as possible. Thanks!

By Insightably on 10/20/2020

The course starts off extremely well and contains a great collection of useful tips, tricks, and functions. The lecture quality decays rather rapidly and by the time one is onto the machine learning portion of the course one would be better served by standing over the shoulder of a data scientist and watch them code instead. Tonnes of unintuitive pyspark syntax is glossed over with no attempt at explanation, typos are plentiful and the encoding choices are quite frankly just wrong in places which would lead to some fairly devastating losses in industry. If you are happy to pay to have kaggle solutions verbalised for you with no explanation then this is the course for you, else if you came here to learn PySpark, you will probably be hard pressed to learn any of the machine learning portion.

By Adam Walker on 6/25/2020

The instructor is very easy to listen to. I also like the design of the course. It's much more practical than other courses that just go through each algorithm individually and don't show how to compare them. Actual data scientists usually compare several algorithms against each other and that is exactly how the instructor approaches the ML portion of the course. You can tell she (YES SHE! - also great) has a lot of experience and designed the course very thoughtfully. Hand clap for this one! I hope she adds more lectures in the future about using MLflow... that part was awesome!

Showing all 10 reviews on CourseDuck

Read all 833 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 88 / 100