Skip to content
UdemyWeb DevelopmentPaid courseIntermediate levelCertificate

Scrape Websites using PhantomJS and CasperJS (Udemy.com)

Become a better JavaScript Developer and Learn Front-End Testing. We'll use javascript, lodash and jquery for scraping.

Created by: Patrick Schroeder

Last updated September 2017

icon
What you will learn

  • Know how to use JavaScript for Data Mining
  • Be able to Capture, Download and Save Website Data
  • Understand how to use CasperJS and PhantomJS
  • Apply What You've Learned to Front-end Testing
  • Create Your Own Scripts for Scraping Data
  • Have a Better Understanding of Functional Programming
  • Fully Understand JavaScript and jQuery Selectors

icon
Course Description

In this course you will learn how to scrape data from web pages using CasperJS.

This course consists of 5 example projects to help you fully understand the powers of the headless browser using the CasperJS API.

What You Will Learn

You will gain a thorough understanding of advanced web scraping concepts and also gain an insight into how to use the CasperJS for Testing DOM manipulation and UI interaction.


What to Expect

  • We'll begin with an overview of how both PhantomJS and CasperJS works along with how to install these frameworks.
  • Next, we'll discuss what our workflow will look like and the options we can pass into a Casper object.
  • Then we'll dive into the meat of this course by working through 5 projects.


The Projects Will Cover

  • How to scrape websites that are rendered with JavaScript instead of standard HTML
  • How to wait for AJAX loaded data to appear before scraping elements
  • How to submit forms both for Authorization and when making searches
  • How to define navigation Steps - like logging into a site, clicking a button and following links
  • How to write and save specified data in tables then output as an .html file or as JSON.
  • And how to take screenshots both of full web pages and specific containers


What is PhantomJS?

PhantomJS is a Full Web Stack that employs a headless browser. Phantom gives us the power to perform many interesting actions on a web page, such as: performing page manipulation, simulating user interaction and the ability to dynamically capture and save website data.


What is CasperJS?

CasperJS is a stand-alone framework built on top Phantom and is compatible with most operating systems. The focus of this course will be on the Casper API and we'll be using this API to write all our web scraping scripts.


What You Should Know

You should already know JavaScript basics. Including what a callback function is. It will help if you know some jQuery. We use lodash in of our examples but only as a replacement for the built-in Map method that's part of the native Javascript API.

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Patrick Schroeder

Software engineer and entrepreneur.  I specialize in full-stack javascript development.  My favorite technologies are Angular and Node.js.  I'm currently working on serverless apps using Firebase Firestore and Google cloud functions running on Node.js.  I enjoy creating high quality courses that clearly explain complex topics.  All courses are captured in 1080p high resolution.  Please message me if you have any questions while taking any of my courses here on Udemy.

icon
More courses by Patrick Schroeder

Free

$54.99

$39.99

icon
More Web Development courses

$19.99

$89.99

$89.99

$79.99

$10.99

$84.99

icon
Reviews

4.2

244 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

By Dave Kidd on 9/4/2018

When adding the path to phantomjs executable file, it would have been nice to have known that each path needs to be separated with a semi colon. Also, the command touch doesn't work on my machine. Also, when installed the node package, most people probably wouldn't have node installed, luckily I did and know how to install node, but assuming that they'll be able to use npm is going to confuse a lot of users. The main issue I'm seeing is things aren't explained in as much detail as they need to be.

By Anonymized User on 12/9/2017

Great quality tutorial, can breeze through it and get a grasp of a load of fundamentals quickly, enough for me to understand and apply to my own use case easily. PhantomJS coupled with CasperJS makes scraping, or browser testing really easy to work with, it's easy to write very minimal, clean code, even for front end coders like me, or even non-coders by using the jQuery API. Now, writing powerful server side code is precisely as easy as writing any other jQuery. Awesome tech, awesome tutorial. Money well spent to rapidly skill up and progress my app project in hours. Thanks! I also quite like the payment structure of Udemy, pay-per-course and access them forever. Ideal for those who may only be able to commit less time to learning, where a higher fixed monthly cost maybe more expensive - unless you know you can get through several courses per month, even then, I would be surprised to find higher quality, or better value courses elsewhere.

By Oliver West on 3/13/2017

Great course, a little on the fast side when I'm trying to type the code instead of just copying or following along with the code files with the course, and there afterwards transitioning between pages like from IDE to the browser etc. Im sure there is a slow down speed button but a decrease in the speed of audio can be a little disturbing. Awesome course will purchase others in the future.

By Stephen Behnfeldt on 2/25/2017

Informative. Moves very fast in some parts, doesn't spend a great deal of time explaining some details. I was actually looking for more general phantom and casper tutorials (for headless web testing), but I still got a fair amount of info from this.

By Baptiste Fleury on 2/4/2017

Le cours est assez complet et très formateur pour les débutants. Par contre: il est assez énervant de voir que les fichiers js fournis, ne correspondent pas toujours à ceux de la vidéo (du coup si l'on veut comparer le code que l'on a noté en suivant la vidéo et celui du formateur ca n'est pas toujours possible). Il manque à mon avis un tuto d'un site sécurisé en https. Globalement, la formation est bonne. Le formateur explique ses actions (même si on aurait aimé un peu plus). Le niveau augmente au fur et à mesure. Je recommande pour les débutants.

By Jake Quade on 1/9/2017

Great visuals – very good at assisting learning. My only feedback for Patrick is that, rather than copying the files and making modifications, you actually re-write the files in every lesson, with prompts for students to follow along. I feel this practice of repeating the crucial steps is more important than simply copying and knowing which components to change. Aside from this, I really enjoyed the course – thank you! @jake_quade

By Nikolay Kolev on 11/27/2016

Somebody below commented that the presenter is not bringing anything outside of the documentation for the framework. Well that might be true however the presenter will save you TONS of time because he summarizes that documentation. As a newcomer to CasperJS I had no idea where to start and what to do. I bought this course because I wanted somebody to walk me through the basics and then show me more advanced things. Step by step. Now I can easily refer to the documentation if I need something in addition to these lectures. The tutor's explanations are extremely on point, super summarized yet very easy to follow and understand. I thank you, sir.

By Bensterenson on 10/8/2016

This course was lackluster. Some of the source code posted alongside the videos do not work. Additionally, the instructor selected extremely dull use cases to explore, making the course boring. He didn't stray too much from the basic functionality shown in the docs, so I recommend anyone considering buying the course to checkout the docs instead. This course has potential to be interesting, I hope that some more effort is put into it. Cheers

By Eduardo Marchuet on 8/14/2016

In terms of quality of videos and explanations is great but I missed a tutorial on how to install Python given that it is a prerrequisite for Casperjs and also a heads up for those of us who know plain old javascript but are not familiar with jquery.

By South Pointe Technologies on 5/23/2016

This course is a great quick start to front-end testing and website scraping. Good instructions and pace. I skipped through lots of the JS stuff I already knew to get to the meat and potatoes, but this is excellent for a beginner.

Showing all 10 reviews on CourseDuck

Read all 244 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 84 / 100