Web Scraping in Nodejs (Udemy.com)
Learn web scraping in Nodejs by example projects with real websites! Craiglist, iMDB, AirBnB and more!
Created by: Stefan Hyltoft
Produced in 2021
What you will learn
- Be able to scrape jobs from a page on Craigslist
- Learn how to use Request
- Learn how to use NightmareJS
- Learn how to use Puppeteer
- Learn how to scrape elements without any identifiable classes or id's
- Learn how to save scraping data to CSV
- Learn how to save scraping data to MongoDb
- Learn how to scrape Facebook using only Request!
- Learn how you can reverse engineer sites and find hidden API's!
- Learn different technologies used for scraping, and when it's best to use them
- Learn how to scrape sites using authentication
Quality Score
Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/
Overall Score : 98 / 100
Course Description
In this course you will learn how to scrape a websites, with practical examples on real websites using Nodejs Request, Cheerio, NightmareJs and Puppeteer. You will be using the newest JavaScript ES7 syntax with async/await.
You will learn how to scrape a Craigslist website for software engineering jobs, using Nodejs Request and Cheerio. You will be using the newest JavaScript ES7 syntax with async/await.
You will then learn how to scrape more advanced websites that require JavaScript such as iMDB and AirBnB using NighmareJs and Puppeteer.
I'm gong to also show you with a practical real-life website, how you can even avoid wasting time on creating a web scraper in the first place, by reverse engineering websites and finding their hidden API's!
Learn how to avoid being blocked from websites when developing out your scraper, by building out the scraper in a test-driven way with mocked html, rather than hitting the website every time as you're debugging and developing it. You'll also learn what you can do if you're blocked and your alternatives to get your scraper up and running regardless!
You will also learn how to scrape on a server with a bad connection, or even if you have a bad connection.
You'll even learn how to save your results to a CSV file and MongoDB!
How do you build a scraper that scrapes every 1 hour (or other interval), and deploy it do a cloud host like Heroku or Google Cloud? Let me show you, quick and easy!
How do you scrape a site requiring passwords? I'm going to show you that too with a real website (Craigslist)!
How do you serve your scraping results in a REST API with Nodejs Express? And how can we build a React frontend that's showing the results? You'll learn that too, in the quickest and simplest way possible!
Plus, a section covering how to make a basic GraphQL API is included in the course.
As a last cherry on the top, I have a section containing a secret backdoor showing you how to scrape Facebook using only Request!
If you have issues regarding a site you're trying to scrape yourself, it's totally okay to reach out to me for some help. I'd be happy to point you in the right direction! Whatever issues my students are facing, I use that to expand on my course! Who this course is for:
You will learn how to scrape a Craigslist website for software engineering jobs, using Nodejs Request and Cheerio. You will be using the newest JavaScript ES7 syntax with async/await.
You will then learn how to scrape more advanced websites that require JavaScript such as iMDB and AirBnB using NighmareJs and Puppeteer.
I'm gong to also show you with a practical real-life website, how you can even avoid wasting time on creating a web scraper in the first place, by reverse engineering websites and finding their hidden API's!
Learn how to avoid being blocked from websites when developing out your scraper, by building out the scraper in a test-driven way with mocked html, rather than hitting the website every time as you're debugging and developing it. You'll also learn what you can do if you're blocked and your alternatives to get your scraper up and running regardless!
You will also learn how to scrape on a server with a bad connection, or even if you have a bad connection.
You'll even learn how to save your results to a CSV file and MongoDB!
How do you build a scraper that scrapes every 1 hour (or other interval), and deploy it do a cloud host like Heroku or Google Cloud? Let me show you, quick and easy!
How do you scrape a site requiring passwords? I'm going to show you that too with a real website (Craigslist)!
How do you serve your scraping results in a REST API with Nodejs Express? And how can we build a React frontend that's showing the results? You'll learn that too, in the quickest and simplest way possible!
Plus, a section covering how to make a basic GraphQL API is included in the course.
As a last cherry on the top, I have a section containing a secret backdoor showing you how to scrape Facebook using only Request!
If you have issues regarding a site you're trying to scrape yourself, it's totally okay to reach out to me for some help. I'd be happy to point you in the right direction! Whatever issues my students are facing, I use that to expand on my course! Who this course is for:
- Anyone who wants to learn how to scrape web sites using Nodejs!
Instructor Details
- 4.9 Rating
36 Reviews
Stefan Hyltoft
Stefan has been building software since elementary school starting out with Visual Basic 6.0. Since then he has been dabbling with Python(Pygame), PHP & MySQL, and Java during university. Since he discovered the JavaScript world he found an intense interest in web development especially using ReactJs/React Native and using NodeJs for the backend.
More courses by Stefan Hyltoft
More node.js courses
The Complete Web Developer in 2021: Zero to Mastery (2021)
4.8 (461 Reviews)
Provider: Udemy
Time: 35.5h
$11.99
Node.js, Express, MongoDB & More: The Complete Bootcamp 2021 (2021)
4.8 (338 Reviews)
- Provider: Udemy
- Time: 42h
$11.99
Node.js API Masterclass With Express & MongoDB (2019)
4.8 (311 Reviews)
- Provider: Udemy
- Time: 12h
$11.99
Just Express (with a bunch of node and http). In detail. (2021)
4.8 (256 Reviews)
- Provider: Udemy
- Time: 12.5h
$11.99
Reviews
- Video Duration:
9.5h - Price: 11.99 Track Price 11.99 Track Price
- Provider: Udemy
- Year:
2021 - Language:
English, English [Auto-generated] - Certificate:
Yes
30-Day Money-Back Guarantee
Udemy Coupon Price Tracker
