Skip to content
Write a review
UdemyHadoopPaid courseAll LevelsCertificate

Flume and Sqoop for Ingesting Big Data (Udemy.com)

Import data to HDFS, HBase and Hive from a variety of sources , including Twitter and MySQL

4.2
172 reviews

Created by: Loony Corn

Last updated October 2016

Our take

Based on the ratings of 172 students, a sample of their written reviews and the syllabus, as the course stood in October 2016. No course pays to be reviewed.

This is a short course from Loonycorn, the Janani Ravi and Vitthal Srinivasan team, on getting data into Hadoop. It runs about 2.3 hours across 17 lectures. Flume gets most of the time: 11 lectures on agents, events, bucketing, channel selectors and interceptors, with Twitter, HTTP and a spooling directory feeding HDFS and HBase. Sqoop gets four lectures on MySQL imports and incremental jobs. It assumes the learner already knows HDFS and has it installed.

Reviewers like the practical examples and the plain explanations. One calls it a good quick start for a beginner, and another says Flume was explained nicely. The complaints are consistent. Sqoop feels thin next to Flume, and some wanted architecture, troubleshooting and production advice, plus ORC, Parquet or Avro formats and Sqoop export. One reviewer gave a single star, calling it shallow and not hands-on. Another found the background music distracting. Still, 141 of 172 ratings are four or five stars.

The course was last updated in October 2016, so check tool versions before following along. Treat it as an afternoon primer, not a reference. The standalone installs it shows skip the production questions, such as failover and scaling. For a first look at how the pieces fit together, it does the job. Anyone doing real cluster work should plan on more material afterward.

Best forEngineers with a working HDFS setup who want a quick first look at Flume agents and a basic Sqoop import.
Skip it ifYou need production tuning, ORC or Parquet formats, Sqoop export, or failover advice for real clusters.

Pros

  • Flume gets 11 lectures covering agents, events, bucketing, channel selectors and interceptors
  • Reviewers like the practical use cases and examples that are easy to follow along
  • Covers Twitter, HTTP, spooling directory and MySQL sources, with HDFS, HBase and Hive sinks
  • At 2.3 hours it is quick to finish and clear for a first pass

Cons

  • Sqoop gets only four lectures, and reviewers want it explained as fully as Flume
  • Skips production topics like failover, scaling, troubleshooting and tuning
  • Last updated in 2016, with no quizzes or exercises, and HDFS knowledge is assumed
Got a good basic regarding for both Flume and Sqoop. The examples are easy to practice with full explanation. Really nice course to get start with both technologies.
Khalid Kizhakkethil, a student

Listed for all levels, but it assumes working HDFS and basic HBase and Hive shell skills.

icon
What you will learn

  • Use Flume to ingest data to HDFS and HBase
  • Use Sqoop to import data from MySQL to HDFS and Hive
  • Ingest data from a variety of sources including HTTP, Twitter and MySQL

Course content

4 sections · 17 lectures · 2.3 hours of video

  1. 1You, This Course and Us 1 free preview1 lecture · 2 min
  2. 2Why do we need Flume and Sqoop? 1 free preview1 lecture · 18 min
  3. 3Flume 1 free preview11 lectures · 1.6 hours
  4. 4Sqoop4 lectures · 22 min

Who it is for

The instructor says it suits

  • Yep! Engineers building an application with HDFS/HBase/Hive as the data store
  • Yep! Engineers who want to port data from legacy data stores to HDFS

What you need before you start

  • Knowledge of HDFS is a prerequisite for the course
  • HBase and Hive examples assume basic understanding of HBase and Hive shells
  • HDFS is required to run most of the examples, so you'll need to have a working installation of HDFS

How it compares

Against the other 150 rated Udemy courses in Hadoop on CourseDuck:

  • Rating4.2 out of 5higher than 34% of them
  • Reviews172#95 of 151 by number of reviews
  • Length2.3 hoursshorter than 91% of them
  • Last updatedOctober 2016earlier than 94% of them

icon
Course Description

Taught by a team which includes 2 Stanford-educated, ex-Googlers. This team has decades of practical experience in working with Java and with billions of rows of data. 

Use Flume and Sqoop to import data to HDFS, HBase and Hive from a variety of sources, including Twitter and MySQL

Let’s parse that.

Import data : Flume and Sqoop play a special role in the Hadoop ecosystem. They transport data from sources like local file systems, HTTP, MySQL and Twitter which hold/produce data to data stores like HDFS, HBase and Hive. Both tools come with built-in functionality and abstract away users from the complexity of transporting data between these systems. 

Flume: Flume Agents can transport data produced by a streaming application to data stores like HDFS and HBase. 

Sqoop: Use Sqoop to bulk import data from traditional RDBMS to Hadoop storage architectures like HDFS or Hive. 

What's Covered:

Practical implementations for a variety of sources and data stores ..

  • Sources : Twitter, MySQL, Spooling Directory, HTTP
  • Sinks : HDFS, HBase, Hive

.. Flume features : 

Flume Agents, Flume Events, Event bucketing, Channel selectors, Interceptors

.. Sqoop features : 

Sqoop import from MySQL, Incremental imports using Sqoop Jobs
Read More All Levels All Levels product length iconTime: 2.3h

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Get the Udemy discount

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Loony Corn

Loonycorn is us, Janani Ravi and Vitthal Srinivasan. Between us, we have studied at Stanford, been admitted to IIM Ahmedabad and have spent years  working in tech, in the Bay Area, New York, Singapore and Bangalore.

Janani: 7 years at Google (New York, Singapore); Studied at Stanford; also worked at Flipkart and Microsoft

Vitthal: Also Google (Singapore) and studied at Stanford; Flipkart, Credit Suisse and INSEAD too

We think we might have hit upon a neat way of teaching complicated tech courses in a funny, practical, engaging way, which is why we are so excited to be here on Udemy!

We hope you will try our offerings, and think you'll like them :-)

Read More

icon
More courses by Loony Corn

$94.99

$59.99

$14.99

$79.99

$54.99

$13.99

All courses by Loony Corn

icon
More Hadoop courses

$11.99

$12.99

$14.99

$94.99

$10.99

$39.99

All Hadoop courses

icon
Reviews

4.2

172 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

Write a review

By Pardeep Kumar on 12/6/2020

Flume was explained nicely with examples. Sqoop could be explained in more details. Overall the learning experience was nice.

By Amin Karami on 5/13/2020

Very simple and shallow lesson. Sqoop and Flume have very important options and configurations that have not covered here. The most commands explained in theory that we can find them easily in Internet. This is not hands-on. Only wasting my time on this lesson.

By Pravin Rangnekar on 1/18/2019

Excellent way of explaining without busy slides.

By Shaik Maricar on 11/23/2018

Very precise and also detailed explanation when required.. I liked it very much and appreciate the practical use case examples.

By Khalid Kizhakkethil on 9/16/2018

Got a good basic regarding for both Flume and Sqoop. The examples are easy to practice with full explanation. Really nice course to get start with both technologies.

By Subhadip Mitra on 7/17/2018

Short crisp course. Suggestion: Discuss the internals/architecture of Sqoop/Flume; and Real World troubleshooting/optimizations.

By Yu Neeq on 1/12/2018

I love all loony corn courses. Other than having to bear hearing the bragging about IIT, Stanford, Google and Flipkart everytime ;D (just kidding :) ), all their courses including this one are always practical, useful and simple to understand. This course may not be a extensive and definitive guide about flume or sqoop, but a good quick start for a beginner that allows you to grasp what the software is about and demonstrates how simply you can start using it.

By Lokanathan Maruthi Prithivirajan on 11/28/2017

Well explained, I'm starting to like the sessions more... just that the back ground music is sometimes distracting... Overall I'm going to subscribe to more Loonycorn classes

By Denise Rogers on 12/5/2016

good introduction. need more of an architecture overview. great use cases to illustrates the topics being presented.

By Manish Gaur on 7/31/2016

Good info but below are missing Would like to see support for different file formats like ORC, Parquet, AVRO etc. as the syntax differs for formats like ORC etc as we need to use HCatalog. Whether we can change engine from MR to Tez or even Spark for Sqoop jobs? No tutorials on Export!? Special Sqoop connectors for Oracle and other databases?

Showing all 10 reviews on CourseDuck

Read all 172 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 84 / 100

Flume and Sqoop for Ingesting Big Data
Visit course on Udemy Write a review
  • Video Duration: icon 2.3h
  • Price: $19.99
  • Provider: icon Udemy
  • Updated: icon October 2016
  • Level: All Levels All Levels
  • Language: icon English
  • Certificate: icon Yes
  • Lectures:17 in 4 sections
  • Students:3,440
  • Subtitles:1 language
30-day money-back guarantee on most paid courses

Udemy Coupon Price Tracker

Udemy price history

For any issues or feature requests please email CourseDuck's Co-Founder at mkuhlman12@gmail.com