Skip to content
UdemyAWSPaid courseIntermediate levelCertificate

Master AWS Lambda Functions for Data Engineers using Python (Udemy.com)

Build Lambda Functions using Python, Lambda Triggers, Deploy using layers and Docker, Validate using Glue and Athena

Created by: Durga Viswanatha Raju Gadiraju

Last updated December 2024

icon
What you will learn

  • Setup required tools on Windows to develop the code for ETL Data Pipelines using Python and AWS Services
  • Setup Project or Development Environment to develop applications using Python and AWS Services
  • Getting Started with AWS by creating account in AWS and also configure AWS CLI as well as Review Data Sets used for the project
  • Develop Core Logic to Ingest Data from source to AWS s3 using Python boto3
  • Getting Started with AWS Lambda Functions using Python 3 Run-time Environment
  • Refactor the application, build zip file to deploy as AWS Lambda Function
  • Create AWS Lambda Function using Zip file and Validate
  • Troubleshoot issues related to AWS Lambda Functions using AWS Cloudwatch
  • Build custom docker image for the application and push to AWS ECR
  • Create AWS Lambda Function using the custom docker image in AWS ECR
  • Develop Applications using AWS Lambda Functions by adding Python Modules as Layers

How it compares

Against the other 235 rated Udemy courses in AWS on CourseDuck:

  • Rating4.3 out of 5higher than 23% of them
  • Reviews268#174 of 236 by number of reviews
  • Length13.3 hourslonger than 65% of them
  • Last updatedDecember 2024more recently than 51% of them

icon
Course Description

Do you want to learn AWS Lambda Functions by building an end-to-end data pipeline using Python as Programming Language and other key AWS Services such as Boto3, S3, Dynamodb, ECR, Cloudwatch, Glue Catalog, Athena, etc? Here is one course using which you will learn AWS Lambda Functions by implementing an end-to-end pipeline by using all the services mentioned.

As part of this course, you will learn how to develop and deploy lambda functions using the zip files, custom docker images as well as layers. Also, you will understand how to trigger lambda functions from Eventsbridge as well as Step Functions.

  • Set up required tools on Windows to develop the code for ETL Data Pipelines using Python and AWS Services. You will take care of setting up Ubuntu using wsl, Docker Desktop, and Visual Studio Code along with Remote Development Extension Kit so that you can develop Python-based applications using AWS Services.

  • Setup Project or Development Environment to develop applications using Python and AWS Services on Windows and Mac.

  • Getting Started with AWS by creating an account in AWS and also configuring AWS CLI as well as Review Data Sets used for the project

  • Develop Core Logic to Ingest Data from source to AWS s3 using Python boto3. The application will be built using Boto3 to interact with AWS Services, Pandas for date arithmetic, and requests to get the files from the source via REST API.

  • Getting Started with AWS Lambda Functions using Python 3.9 Run-time Environment

  • Refactor the application, and build a zip file to deploy as AWS Lambda Function. The application logic includes capturing bookmarks as well as Job Run details in Dynamodb. You will also get an overview of Dynamodb and how to interact with Dynamodb to manage Bookmark as well as Job Run details.

  • Create AWS Lambda Function using a Zip file, deploy using AWS Console and Validate.

  • Troubleshoot issues related to AWS Lambda Functions using AWS Cloudwatch

  • Build a custom docker image for the application and push it to AWS ECR

  • Create AWS Lambda Function using the custom docker image in AWS ECR and then validate.

  • Get an understanding of AWS s3 Event Notifications or s3-based triggers on Lambda Function.

  • Develop another Python application to transform the data and also write the data in the form of Parquet to s3. The application will be built using Pandas by converting 10,000 records at a time to Parquet.

  • Build orchestrated pipeline using AWS s3 Event Notifications between the two Lambda Functions.

  • Schedule the first lambda function using AWS EventsBridge and then validate.

  • Finally, create an AWS Glue Catalog table on the s3 location which has parquet files, and validate by running SQL Queries using AWS Athena.

  • After going through the complete life cycle of Deploying and Scheduling Lambda Function and also validating the data by using Glue Catalog and AWS Athena, you will also understand how to use Layers for Lambda Function.

Here are the key takeaways from this training:

  • Develop Python Applications and Deploy as Lambda Functions by using a Zip-based bundle as well as a custom docker image.

  • Monitor and troubleshoot the issues by going through Cloudwatch logs.

  • The entire application code used for the demo along with the notebook used to come up with core logic.

  • Ability to build solutions using multiple AWS Services such as Boto3, S3, Dynamodb, ECR, Cloudwatch, Glue Catalog, Athena, etc

icon
Udemy Discount

The discount is applied through our link. Open the course from here and Udemy's current promotional price is applied at checkout on most courses, no code to type.

Some courses are excluded from Udemy's promotions. If the price does not drop, clear your browser cookies and use the button again.

icon
Instructor Details

Durga Viswanatha Raju Gadiraju

20+ years of experience in executing complex projects using a vast array of technologies including Big Data and the Cloud.

ITVersity, Inc. - is a US-based organization that provides quality training for IT professionals and we have a track record of training hundreds of thousands of professionals globally.

Building an IT career for people with required tools such as high-quality material, labs, live support, etc to upskill and cross-skill is paramount for our organization.

At this time our training offerings are focused on the following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Datawarehousing, Databases

icon
More courses by Durga Viswanatha Raju Gadiraju

$119.99

$119.99

$74.99

$10.99

$14.99

$10.99

icon
More AWS courses

$99.99

$49.99

$99.99

$124.99

$22.99

$84.99

icon
Reviews

4.3

268 ratings on Udemy

Select a bar to show only those reviews.Select the bar again to show every rating.

By Jose Bueno on 4/20/2025

The course is good, but the project he is building is 2 big. I would have chosen a smaller one.

By Ravi Naidu on 3/11/2024

Good Explanation of various concepts in detail, I will be looking for his course on automating this stuff using something like AWS CDK - instead of working on AWS management console

By Tom Frizzell on 11/11/2023

The course is fine but I have to put the talk back speed to .75 as the presenter speaks very rapidly with an accent.

By Oya kesgin on 9/29/2023

I followed his videos on AWS and actually it is prepared so well. Lots of useful information is given. However, My lambda function is not worked but I assume my version of python wasn't matching with AWS's one. Happy with overall but I would like to know more if we get the data from outside database, how it would be?

By Ravi Oruganti on 4/22/2023

This cource is excellent for beginers .crisp and clear .easy to follow hands on .

By MooseData Kamil Musiałowski Plantowa 5/1 on 1/13/2023

Review: TLDR - compared to top notch Udemy courses this one falls in 3 star ranking due to high level of instructions repetition, lack of 'why' explanations and missing details on Windows environment setup. On the other hand, instructions are so detailed (idiot-proof) that you are able to finish all tasks and course is informative on various way of interacting with AWS Lambda, including various deployment features, also relative to Udemy AWS courses this is great, hence 4 stars. Pros: Course is informative, more 'hands on' than theory (apart from application layout you will see very few presentation slides or animations, but pure coding) and gives a zero to hero approach, including environment setup, which was great for me coming from DB background. I think the layout is ok- first insight on tech stack we are going to use and then how to connect the dots on your local env, finally applying this to AWS Lambda. Cons: At the beginning of the course you are given details on setup of local environment, but one thing is missing - how to use Ubuntu WSL in Visual Studio Code (with AWS credentials setup), which may cause plenty of issues later on as you attempt to execute linux syntax commands in a Visual Studio default Powershell terminal. Also if you want to use AWS Cli in ubuntu, it has to be separately installed there (apart from Windows installation) Moreover Docker doesn't work in WSL v1 so you either have to containerize application in Windows Environment or move to WSL v2 (make sure check your default WSL version to be 2 before starting ubuntu). Fortunately most of the issues is quickly answered. I think it would be also beneficial to give 1minute description on relationships between virtual env / WSL / VSC. Also with any linux command used for the first time, it would be good to have explanation of options used (e.g why 'ls -ltr'?) This course could have been shortened by 20-30% if: Sometimes instructor is troubleshooting his own issues which takes time (e.g. some commands not running etc) Or repeating steps from previous lessons - e.g. Checking whether configuration is ok, which as a learning exercise may be ok, but takes time. Or repeating for 1 minute what we did in an exercise which took 3 minutes Exercices/lessons shouldn't be taking 3 minutes, rather ~10min. Or repeating topics already covered -e.g. sections on S3: lessons 36-38 vs 94-95. Or giving trivial statements - e.g. if you won't tick acknowledge you want I think the author favorite statement is 'this is nothing but…' which is frustrating to hear for 100th time. On some topics lecturer just repeats himself without any further meaningful insight. On the other topics an overview would be helpful -e.g. lecture 77 would greatly benefit from just providing a schema on how deployment process would be arranged (a picture is worth 1000 words). Accent is not that strong and well understandable but I think the instructor could be 10-15% slower - it would help to understand instructor better.

By Smita on 11/22/2022

The instructor is doing the setup without giving details about 1. what is ubuntu 2. why we need virtual env for this project.

By Jose Caldas on 9/22/2022

The content is really good, I highly recommend it. There are some concepts that are repeated several times (unnecessary). Some good additions could be versions/aliases, enabling lambda in VPC.

By Deepak Garg on 9/15/2022

I was looking for this course from long time. Thanks for creating this wonderful course. Covered each topic in detail and lots of learning is there.

By Elmer Olivera on 8/25/2022

I some chapters your explanation goes too fast and in other chapters you have an entire module only to explain how to configure your environment in windows.

Showing all 10 reviews on CourseDuck

Read all 268 reviews on Udemy

icon
Quality Score

No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.

Content Quality
/
Video Quality
/
Qualified Instructor
/
Course Pace
/
Course Depth & Coverage
/

Overall Score : 86 / 100