Scrape Websites using PhantomJS and CasperJS (Udemy.com)
Become a better JavaScript Developer and Learn Front-End Testing. We'll use javascript, lodash and jquery for scraping.
Created by: Patrick Schroeder
Last updated September 2017
What you will learn
- Know how to use JavaScript for Data Mining
- Be able to Capture, Download and Save Website Data
- Understand how to use CasperJS and PhantomJS
- Apply What You've Learned to Front-end Testing
- Create Your Own Scripts for Scraping Data
- Have a Better Understanding of Functional Programming
- Fully Understand JavaScript and jQuery Selectors
Course Description
In this course you will learn how to scrape data from web pages using CasperJS.
This course consists of 5 example projects to help you fully understand the powers of the headless browser using the CasperJS API.
What You Will Learn
You will gain a thorough understanding of advanced web scraping concepts and also gain an insight into how to use the CasperJS for Testing DOM manipulation and UI interaction.
What to Expect
- We'll begin with an overview of how both PhantomJS and CasperJS works along with how to install these frameworks.
- Next, we'll discuss what our workflow will look like and the options we can pass into a Casper object.
- Then we'll dive into the meat of this course by working through 5 projects.
The Projects Will Cover
- How to scrape websites that are rendered with JavaScript instead of standard HTML
- How to wait for AJAX loaded data to appear before scraping elements
- How to submit forms both for Authorization and when making searches
- How to define navigation Steps - like logging into a site, clicking a button and following links
- How to write and save specified data in tables then output as an .html file or as JSON.
- And how to take screenshots both of full web pages and specific containers
What is PhantomJS?
PhantomJS is a Full Web Stack that employs a headless browser. Phantom gives us the power to perform many interesting actions on a web page, such as: performing page manipulation, simulating user interaction and the ability to dynamically capture and save website data.
What is CasperJS?
CasperJS is a stand-alone framework built on top Phantom and is compatible with most operating systems. The focus of this course will be on the Casper API and we'll be using this API to write all our web scraping scripts.
What You Should Know
You should already know JavaScript basics. Including what a callback function is. It will help if you know some jQuery. We use lodash in of our examples but only as a replacement for the built-in Map method that's part of the native Javascript API.
Instructor Details
- 4.2 Rating
244 Reviews
Patrick Schroeder
Software engineer and entrepreneur. I specialize in full-stack javascript development. My favorite technologies are Angular and Node.js. I'm currently working on serverless apps using Firebase Firestore and Google cloud functions running on Node.js. I enjoy creating high quality courses that clearly explain complex topics. All courses are captured in 1080p high resolution. Please message me if you have any questions while taking any of my courses here on Udemy.
More courses by Patrick Schroeder
MongooseJS Essentials - Learn MongoDB for Node.js (2017)
4.6 (3,200 Reviews)
Provider: Udemy
Time: 42 mins
Free
Angular 2+ with Typescript - Essential Training (2017)
4.4 (1,786 Reviews)
Provider: Udemy
Time: 3h
$54.99
More Web Development courses
Python Fullstack Mastery: Build Modern Web Apps with Django (2025)
5.0 (165 Reviews)
Provider: Udemy
Time: 35h
$19.99
NgRx Signal store for Angular - The missing guide (2026)
4.9 (436 Reviews)
Provider: Udemy
Time: 11.3h
$89.99
Vue 3: Create a Mobile & Desktop App (with Quasar 2 & Pinia) (2024)
4.9 (177 Reviews)
Provider: Udemy
Time: 7.6h
$79.99
Build a URL Shortener App with ASP.NET: Step by Step (2026)
4.9 (100 Reviews)
Provider: Udemy
Time: 9.6h
$10.99
The Complete Elixir and Phoenix Bootcamp (2021)
4.8 (7,773 Reviews)
Provider: Udemy
Time: 17.4h
$84.99
Reviews
By Dave Kidd on 9/4/2018
When adding the path to phantomjs executable file, it would have been nice to have known that each path needs to be separated with a semi colon. Also, the command touch doesn't work on my machine. Also, when installed the node package, most people probably wouldn't have node installed, luckily I did and know how to install node, but assuming that they'll be able to use npm is going to confuse a lot of users. The main issue I'm seeing is things aren't explained in as much detail as they need to be.
By Anonymized User on 12/9/2017
Great quality tutorial, can breeze through it and get a grasp of a load of fundamentals quickly, enough for me to understand and apply to my own use case easily. PhantomJS coupled with CasperJS makes scraping, or browser testing really easy to work with, it's easy to write very minimal, clean code, even for front end coders like me, or even non-coders by using the jQuery API. Now, writing powerful server side code is precisely as easy as writing any other jQuery. Awesome tech, awesome tutorial. Money well spent to rapidly skill up and progress my app project in hours. Thanks! I also quite like the payment structure of Udemy, pay-per-course and access them forever. Ideal for those who may only be able to commit less time to learning, where a higher fixed monthly cost maybe more expensive - unless you know you can get through several courses per month, even then, I would be surprised to find higher quality, or better value courses elsewhere.
By Oliver West on 3/13/2017
Great course, a little on the fast side when I'm trying to type the code instead of just copying or following along with the code files with the course, and there afterwards transitioning between pages like from IDE to the browser etc. Im sure there is a slow down speed button but a decrease in the speed of audio can be a little disturbing. Awesome course will purchase others in the future.
By Stephen Behnfeldt on 2/25/2017
Informative. Moves very fast in some parts, doesn't spend a great deal of time explaining some details. I was actually looking for more general phantom and casper tutorials (for headless web testing), but I still got a fair amount of info from this.
By Baptiste Fleury on 2/4/2017
Le cours est assez complet et très formateur pour les débutants. Par contre: il est assez énervant de voir que les fichiers js fournis, ne correspondent pas toujours à ceux de la vidéo (du coup si l'on veut comparer le code que l'on a noté en suivant la vidéo et celui du formateur ca n'est pas toujours possible). Il manque à mon avis un tuto d'un site sécurisé en https. Globalement, la formation est bonne. Le formateur explique ses actions (même si on aurait aimé un peu plus). Le niveau augmente au fur et à mesure. Je recommande pour les débutants.
By Jake Quade on 1/9/2017
Great visuals – very good at assisting learning. My only feedback for Patrick is that, rather than copying the files and making modifications, you actually re-write the files in every lesson, with prompts for students to follow along. I feel this practice of repeating the crucial steps is more important than simply copying and knowing which components to change. Aside from this, I really enjoyed the course – thank you! @jake_quade
By Nikolay Kolev on 11/27/2016
Somebody below commented that the presenter is not bringing anything outside of the documentation for the framework. Well that might be true however the presenter will save you TONS of time because he summarizes that documentation. As a newcomer to CasperJS I had no idea where to start and what to do. I bought this course because I wanted somebody to walk me through the basics and then show me more advanced things. Step by step. Now I can easily refer to the documentation if I need something in addition to these lectures. The tutor's explanations are extremely on point, super summarized yet very easy to follow and understand. I thank you, sir.
By Bensterenson on 10/8/2016
This course was lackluster. Some of the source code posted alongside the videos do not work. Additionally, the instructor selected extremely dull use cases to explore, making the course boring. He didn't stray too much from the basic functionality shown in the docs, so I recommend anyone considering buying the course to checkout the docs instead. This course has potential to be interesting, I hope that some more effort is put into it. Cheers
By Eduardo Marchuet on 8/14/2016
In terms of quality of videos and explanations is great but I missed a tutorial on how to install Python given that it is a prerrequisite for Casperjs and also a heads up for those of us who know plain old javascript but are not familiar with jquery.
By South Pointe Technologies on 5/23/2016
This course is a great quick start to front-end testing and website scraping. Good instructions and pace. I skipped through lots of the JS stuff I already knew to get to the meat and potatoes, but this is excellent for a beginner.
Quality Score
No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.
Overall Score : 84 / 100









