Skip to content
Write a review
UdemyBestseller on UdemyIT CertificationsPaid courseAll LevelsCertificate

AI Agents, RAG & LLM Evals for Beginners: DeepEval & RAGAS (Udemy.com)

2026 - Path to AI QA Engineer to test LLMs and AI Apps using DeepEval, RAGAs and HF Evaluate with Local LLMs like Ollama

4.3
877 reviews

Created by: Karthik KK

Last updated June 2026

Our take

Based on the ratings of 877 students, a sample of their written reviews and the syllabus, as the course stood in June 2026. No course pays to be reviewed.

Karthik KK, a long-time automation tester, built this for QA people moving into AI work. It covers DeepEval and RAGAs, running local models with Ollama, RAG testing, agent tool calls, PyTest and Hugging Face Evaluate. The syllabus shows 127 lectures and about 14.7 hours, and the last update was June 2026, so the tooling is fairly fresh. There are ten quizzes but no coding exercises. The listed audience is QA engineers, AI QA test engineers, business analysts and AI engineers.

Happy reviewers say it is easy to follow and quick to apply. Two say they use it at work, and one Selenium and Playwright tester says it helped with interview prep. A few are repeat buyers of Karthik's courses. The complaints are consistent, though. Low-star reviewers say he reads code without explaining why it is written, examples stay close to the documentation, and some code needs fixing for newer library versions. One four-star reviewer says setup and basic Python take too long.

Of 877 ratings, 456 are five stars and 29 are one or two stars, so most people are satisfied. At 8,185 students it is a newer course, published in March 2025. Treat it as a solid first pass at the tools, and expect to build bigger test datasets on your own afterward. If you already test software and know a little Python, it is a fair pick. Total newcomers to programming may find the explanations thin, going by the low-star reviews.

Best forQA engineers from a Selenium or Playwright background who want to start testing RAG apps and AI agents with DeepEval.
Skip it ifYou want slow explanations of Python and LLM basics, or large real-world test datasets.

Pros

  • Covers DeepEval, RAGAs and Hugging Face Evaluate in one place, plus PyTest
  • Runs on free local models through Ollama, so practice needs no API bill
  • RAG and agent tool call testing, and reviewers report using DeepEval and RAGAs at work
  • Refreshed in 2026, with updated source code and PyTest material

Cons

  • Reviewers say code is read out with little explanation of why it is written
  • Examples stay close to the docs, with no large synthetic dataset or real-world comparison
  • Setup and basic Python detours eat time, and some code breaks on newer tool versions
Such an incredible resource on testing AI apps, easy to follow and apply at work.
Tarik Basa, a student

Suits testers with basic Python, Java or JavaScript. Total programming newcomers may find the explanations too thin.

icon
What you will learn

  • Understand the purpose of Testing LLM and LLM based Application
  • Understand DeepEval and RAGAs in detail from complete ground up
  • Understand different metrics and evaluations to evaluate LLMs and LLM based app using DeepEval and RAGAs
  • Understand the advanced concepts of DeepEval and RAGAs
  • Testing RAG based application using DeepEval and RAGAs
  • Testing AI Agents using DeepEval to understand how tool callings can be tested

Course content

17 sections · 127 lectures · 15 hours of video 10 quizzes, 10 articles

  1. 1What You'll Learn: AI Testing, LLM Evaluation & the Tools That Power It 2 free previews6 lectures · 1 quiz · 1.1 hours
  2. 2Run Any LLM on Your Own Machine for Free with Ollama6 lectures · 1 quiz · 26 min
  3. 3Download the Complete Course Source Code1 lecture
  4. 4Set Up Your Testing Environment for LLM Evaluation (DeepEval + RAGAs)5 lectures · 21 min
  5. 5DeepEval from the Ground Up: Core Concepts Every AI Tester Must Know14 lectures · 1 quiz · 1.8 hours
  6. 6Test a Real Local LLM End-to-End: Generate Outputs & Build Evaluation Datasets7 lectures · 1 quiz · 47 min
  7. 7How to Test a RAG App with DeepEval: From Setup to Evaluation6 lectures · 1 quiz · 32 min
  8. 8Advanced RAG Testing: Multi-Input Datasets, Goldens & Confident AI Integration 1 free preview8 lectures · 1 quiz · 30 min
  9. 9How to Test AI Agents & Tool Calls Using Local LLMs and DeepEval8 lectures · 1 quiz · 41 min
  10. 10PyTest + DeepEval: Write Clean, Professional AI Eval Test with Zero Boilerplate14 lectures · 1.7 hours

Who it is for

The instructor says it suits

  • QA Engineers
  • AI QA Test Engineers
  • Business Analyst
  • AI Engineers

What you need before you start

  • Basics of working with LLM like using ChatGPT
  • Basics of any programing language like Java or Javascript
  • Basics of python will be a plus

How it compares

Against the other 848 rated Udemy courses in IT Certifications on CourseDuck:

  • Rating4.3 out of 5higher than 18% of them
  • Reviews877#309 of 849 by number of reviews
  • Length14.7 hourslonger than 72% of them
  • Last updatedJune 2026more recently than 70% of them

icon
Course Description

Testing AI & LLM App with DeepEval, RAGAs & more using Ollama and Local Large Language Models (LLMs)

Master the essential skills for testing and evaluating AI applications, particularly Large Language Models (LLMs). This hands-on course equips QA, AI QA, Developers, data scientists, and AI practitioners with cutting-edge techniques to assess AI performance, identify biases, and ensure robust application development.



Topics Covered:

  • Section 1: Foundations of AI Application Testing (Introduction to LLM testing, AI application types, evaluation metrics, LLM evaluation libraries).

  • Section 2: Local LLM Deployment with Ollama (Local LLM deployment, AI models, running LLMs locally, Ollama implementation, GUI/CLI, setting up Ollama as API).

  • Section 3: Environment Setup (Jupyter Notebook for tests, setting up Confident AI).

  • Section 4: DeepEval Basics (Traditional LLM testing, first DeepEval code for AnswerRelevance, Context Precision, evaluating in Confident AI, testing with local LLM, understanding LLMTestCases and Goldens).

  • Section 5: Advanced LLM Evaluation (LangChain for LLMs, evaluating Answer Relevancy, Context Precision, bias detection, custom criteria with GEval, advanced bias testing).

  • Section 6: RAG Testing with DeepEval (Introduction to RAG, understanding RAG apps, demo, creating GEval for RAG, testing for conciseness & completeness).

  • Section 7: Advanced RAG Testing with DeepEval (Creating multiple test data, Goldens in Confident AI, actual output and retrieval context, LLMTestCases from dataset, running evaluation for RAG).

  • Section 8: Testing AI Agents and Tool Callings (Understanding AI Agents, working with agents, testing agents with and without actual systems, testing with multiple datasets).

  • Section 9: Evaluating LLMs using RAGAS (Introduction to RAGAS, Context Recall, Noise Sensitivity, MultiTurnSample, general purpose metrics for summaries and harmfulness).

  • Section 10: Testing RAG applications with RAGAS (Introduction and setup, creating retrievers and vector stores, MultiTurnSample dataset for RAG, evaluating RAG with RAGAS).
Read More All Levels All Levels product length iconTime: 14.7h

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Get the Udemy discount

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Karthik KK

I am Karthik K.K. I do consulting, blogging and researching on various different tools and technologies which inspire my interest. I have been into Software Automation Testing for over 20 years now and my interest in learning new stuffs which enables automation has never been compromised.

Due to my passion in automation testing I got fully exposed to tools and languages like


AI and Machine Learning: Built an Tool for Playwright MCP Server, LangChain, DeepEval, RAGAs, HuggingFace, Fine-tuning LLMs

Desktop & Web : Cypress, Selenium, Specflow,Puppeteer,Plawright, UFT, VS Coded UI, Ranorex ,Test Complete and Cucumber
Mobile : Appium, Robotium,Calabash, Espresso, Selendroid
Languages : C#, Java, Ruby, Powershell, Javascript, VBScript 
Cloud Tools : Azure, Dockers
Misc Tools: Mockito, Rest-assured, Bound-Box,FluentAutomation
Interested in: MVC, Entity framework, LINQ, Fakes, Distributed deployment, SAAS, Event-Driven System


Read More

icon
More courses by Karthik KK

$11.99

$19.99

$19.99

Free

$10.99

$10.99

All courses by Karthik KK

icon
More IT Certifications courses

$59.99

$10.99

$10.99

$19.99

$11.99

$19.99

All IT Certifications courses

icon
Reviews

4.3

877 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

Write a review

By Asimo on 7/5/2026

I am preparing for interview and this course is already helping me a lot. These days so many companies looking for AI agent building and testing experience , this course already helped me in understanding the concept. I am from Selenium/Playwright testing background, I can see how much this course is already helping me.,

By Yu jing on 5/28/2026

I was searching for a practical DeepEval course for weeks as my company is implementing AI based apps and this was exactly what I needed. The examples using local models were a huge bonus.

By Rahul Jain on 5/26/2026

One of the few courses that actually explains LLM evaluation properly instead of just theory. The DeepEval and RAG sections were extremely practical and easy to follow.

By Ankit Saini on 12/16/2025

I am not sure how I will be testing google adk agent I am confused like if deepeval will require a LLM then how it will test another LLM which is integrated in my project and the project agents are non-conversational. Agents are extracting data like images, texts, summaries, and heading or titles from web. I am not sure how I am gonna test those yet?

By Winston Chow on 11/25/2025

Overall, Karthik provided an awesome in-depth detailed course for Testing AI & LLM applications with DeepEval and RAGAS, and using local LLMs with Ollama. I know that he has provided and re-did sections of the lectures to cover the updates, however, I still ran into issues as the deepeval and langchain libraries continues to update, and needed to spend time to debug and update the code. Still I learned a lot and this is a great starter course for any QA engineer wishing to learn how to test AI, RAG-LLMs, AI Agents.

By Leonardo Miro on 9/24/2025

Karthik does a fine job introducing and showing DeepEval and RAGAs at a programmatic level. The last chapters are really challenging depending on your programming skills. A correct roadmap for testers interested in AI.

By Felix Schlegel (Auticon Swiss AG) on 9/17/2025

The course is ok to get a quick overview over several libraries. However, it is quite superficial, the example are not much more than what you find in the documentation. I am missing a real world scenario where you would create a large synthetic test dataset using tools like Ragas, then compare the performance of different RAG applications, not just passing one or two manual test cases.

By Mahammad Ghouse Guduru on 8/14/2025

Overall good course to have a basic understanding of DeepEval, RAGAS and Hugging face evaluation framework. playing it 1.25 speed would be good. lot of repetition and covers basic metrics only. Overall good course for starting on evaluation.

By Venthan Gowtham Sivakumar on 7/22/2025

Great effort and research, Karthick, in bringing us niche details on AI evaluation and tools. You make complex topics simple with clear, impactful content. Thanks for your hard work — keep it up, and continue inspiring us on this AI journey

By Martin on 4/21/2025

Perfect and Take no look anywhere, this course covers all of Testing of LLMS and AI models details. if you are getting started from basic and wanted to go beyond basic to advanced. Thanks for the course.

Showing all 10 reviews on CourseDuck

Read all 877 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 86 / 100

AI Agents, RAG & LLM Evals for Beginners: DeepEval & RAGAS
Visit course on Udemy Write a review
  • Video Duration: icon 14.7h
  • Price: $19.99
  • Provider: icon Udemy
  • Updated: icon June 2026
  • Level: All Levels All Levels
  • Language: icon English
  • Certificate: icon Yes
  • Lectures:127 in 17 sections
  • Practice:10 quizzes
  • Students:8,185
  • Subtitles:9 languages
  • Udemy badge:Bestseller
30-day money-back guarantee on most paid courses

Udemy Coupon Price Tracker

Udemy price history

For any issues or feature requests please email CourseDuck's Co-Founder at mkuhlman12@gmail.com