Taming Big Data with Apache Spark and Python - Hands On! (Udemy.com)

Apache Spark tutorial with 20+ hands-on examples of analyzing large data sets on your desktop or on Hadoop with Python!

Created by: Sundog Education by Frank Kane

Produced in 2021

icon
What you will learn

  • Use DataFrames and Structured Streaming in Spark 3
  • Frame big data analysis problems as Spark problems
  • Use Amazon's Elastic MapReduce service to run your job on a cluster with Hadoop YARN
  • Install and run Apache Spark on a desktop computer or on a cluster
  • Use Spark's Resilient Distributed Datasets to process and analyze large data sets across many CPU's
  • Implement iterative algorithms such as breadth-first-search using Spark
  • Use the MLLib machine learning library to answer common data mining questions
  • Understand how Spark SQL lets you work with structured data
  • Understand how Spark Streaming lets your process continuous streams of data in real time
  • Tune and troubleshoot large jobs running on a cluster
  • Share information between nodes on a Spark cluster using broadcast variables and accumulators
  • Understand how the GraphX library helps with network analysis problems

icon
Quality Score

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 86 / 100

icon
Course Description

New!
Updated forSpark 3, more hands-on exercises, and a stronger focus on DataFrames and Structured Streaming.
Big data" analysis is a hot and highly valuable skill and this course will teach you the hottest technology in big data: Apache Spark. Employers including Amazon, EBay, NASA JPL, and Yahoo all use Spark to quickly extract meaning from massive data sets across a fault-tolerant Hadoop cluster. You'll learn those same techniques, using your own Windows system right at home. It's easier than you might think.
Learn and master the art of framing data analysis problems as Spark problems through over 20 hands-on examples, and then scale them up to run on cloud computing services in this course. You'll be learning from an ex-engineer and senior manager fromAmazon and IMDb.
Learn the concepts of Spark's DataFrames and Resilient Distributed DatastoresDevelop and run Spark jobs quickly using PythonTranslate complex analysis problems into iterative or multi-stage Spark scriptsScale up to larger data sets using Amazon's Elastic MapReduce serviceUnderstand how Hadoop YARN distributes Spark across computing clustersLearn about other Spark technologies, like Spark SQL, Spark Streaming, and GraphXBy the end of this course, you'll be running code that analyzes gigabytes worth of information in the cloud in a matter of minutes.
This course uses the familiar Python programming language; if you'd rather use Scala to get the best performance out of Spark, see my "ApacheSpark withScala - Hands On withBigData" course instead.
We'll have some fun along the way. You'll get warmed up with some simple examples of using Spark to analyze movie ratings data and text in a book. Once you've got the basics under your belt, we'll move to some more complex and interesting tasks. We'll use a million movie ratings to find movies that are similar to each other, and you might even discover some new movies you might like in the process! We'll analyze a social graph of superheroes, and learn who the most popular" superhero is and develop a system to find degrees of separation" between superheroes. Are all Marvel superheroes within a few degrees of being connected to The Incredible Hulk? You'll find the answer.
This course is very hands-on; you'll spend most of your time following along with the instructor as we write, analyze, and run real code together both on your own system, and in the cloud using Amazon's Elastic MapReduce service. 7 hours of video content is included, with over 20 real examples of increasing complexity you can build, run and study yourself. Move through them at your own pace, on your own schedule. The course wraps up with an overview of other Spark-based technologies, including Spark SQL, Spark Streaming, and GraphX.
Wrangling big data with ApacheSpark is an important skill in today's technical world. Enroll now!" I studied "Taming Big Data with Apache Spark and Python" with Frank Kane, and helped me build a great platform for Big Data as a Service for my company. I recommend the course! " - Cleuton Sampaio De Melo Jr.
Who this course is for:
People with some software development background who want to learn the hottest technology in big data analysis will want to check this out. This course focuses on Spark from a software development standpoint; we introduce some machine learning and data mining concepts along the way, but that's not the focus. If you want to learn how to use Spark to carve up huge datasets and extract meaning from them, then this course is for you.
If you've never written a computer program or a script before, this course isn't for you - yet. I suggest starting with a Python course first, if programming is new to you.
If your software development job involves, or will involve, processing large amounts of data, you need to know about Spark.
If you're training for a new career in data science or big data, Spark is an important part of it.

icon
Instructor Details

Sundog Education by Frank Kane

Sundog Education's mission is to make highly valuable career skills in big data, data science, and machine learning accessible to everyone in the world. Our consortium of expert instructors shares our knowledge in these emerging fields with you, at prices anyone can afford.
Sundog Education is led by Frank Kane and owned by Frank's company, Sundog Software LLC.Frank spent 9 years at Amazon and IMDb, developing and managing the technology that automatically delivers product and movie recommendations to hundreds of millions of customers, all the time. Frank holds 17 issued patents in the fields of distributed computing, data mining, and machine learning. In 2012, Frank left to start his own successful company, Sundog Software, which focuses on virtual reality environment technology, and teaching others about big data analysis.
Due to our volume of students we are unable to respond to private messages; please post your questions within the Q&A of your course. Thanks for understanding.

icon
More courses by Sundog Education by Frank Kane

AWS Certified Data Analytics Specialty - Hands On!

$11.99

Apache Spark with Scala - Hands On with Big Data!

$11.99

AWS Certified Machine Learning Specialty 2021 - Hands On!

$11.99

Streaming Big Data with Spark Streaming and Scala - Hands On

$11.99

Build a Serverless App with AWS Lambda - Hands On!

$11.99

Machine Learning, Data Science and Deep Learning with Python

$11.99

icon
More hadoop courses

Apache Kafka Series - Kafka Cluster Setup & Administration

$11.99

Apache Kafka Series - Confluent Schema Registry & REST Proxy

$11.99

Apache Spark for Java Developers

$11.99

Apache Kafka - Real-time Stream Processing (Master Class)

$11.99

Apache Kafka Series - Kafka Monitoring & Operations

$11.99

Apache Kafka for absolute beginners

$11.99

icon
Reviews

4.3

311 total reviews

5 star 4 star 3 star 2 star 1 star
% Complete
% Complete
% Complete
% Complete
% Complete

By Diego Mello on 11/7/2020

Very explanatory

By Saso Stepanov on 11/7/2020

Excellent course, useful examples

By Claire Peng on 11/7/2020

Just walk through half of the videos, most of them are just read code out loud without teaching organized or structured knowledge.

By Prag Tyagi on 11/7/2020

Although all concepts are explained with ease but there are lot of things which are missed which are very important . PFB these points :-
i)No description on the type of joins in Dataframes
ii)Not much real world examples
iii)Spark integration with Kafka which is the most used scenario is missing
iv)Static configuration to run a spark job on cluster i.e no of cores and no of cores per executor , no of executor is missing .
v)Link which is being provided to install spark on linux is outdated and doesn't contain enough information with respect to running newer versions of spark on linux

By Parag Jain on 11/7/2020

Excellent course for Beginners and even intermediates. The advances codes in this course are useful to get a feel of the real world use cases and complex lines of codes.

By Jose Mari Escalante on 11/7/2020

Frank makes learning Apache Spark simple and easy. It has a good amount of hands-on activities and exercises to get started. Recommended for beginners.

By Simeon on 11/1/2020

Gave me a solid understanding of what Spark is and how to use it in a business setting! The examples covered were sufficient to get started with Spark.

By Joel Braganza on 10/27/2020

Yes, the slides could have included some pictoral representations showing the structure of
RDDs in context of datasets and how they are spread accross clusters to make us understand concepts well enough. Though the explanation is fine, the better slides could've made it better understandable.

By Cyrus Tata on 10/20/2020

The course covers a number of topics without rushing through any of them. It is well supported and questions are answered reasonably quickly.

By Romina Sarmiento on 10/16/2020

Fun

By Pushpak Bansal on 10/13/2020

good hands on practice with clarity over the concepts.

By Siyanda Matika on 10/9/2020

An excellent refresher on spark