Introduction to Spark SQL and DataFrames

( 0 Reviews )

Created by: Dan Sullivan

Produced in 2019

Quality Score

Content Quality

Video Quality

Qualified Instructor

Course Pace

Course Depth & Coverage

Overall Score : 0 / 100

Course Description

Explore DataFrames, a widely used data structure in Apache Spark. DataFrames allow Spark developers to perform common data operations, such as filtering and aggregation, as well as advanced data analysis on large collections of distributed data. With the addition of Spark SQL, developers have access to an even more popular and powerful query language than the built-in DataFrames API. In this course, instructor Dan Sullivan shows how to perform basic operations- "loading, filtering, and aggregating data in DataFrames- "with the API and SQL, as well as more advanced techniques that are easily performed in SQL. In this section of the course, Dan explains how to join data, eliminate duplicates, and deal with null or NA values. The lessons conclude with three in-depth examples of using DataFrames for data science: exploratory data analysis, time series analysis, and machine learning.

Intermediate Level

Time: 1h 53m

Instructor Details

Dan Sullivan

0.0 Rating
0 Reviews

Dan Sullivan, PhD, is an enterprise architect and big data expert.

Dan specializes in data architecture, analytics, data mining, statistics, data modeling, big data, and cloud computing. In addition, he holds a PhD in genetics, bioinformatics, and computational biology. Dan works regularly with Spark, Oracle, NoSQL, MongoDB, Redis, R, and Python. He has extensive writing experience in topics including cloud computing, big data, Hadoop, and security.