Master Data Engineering using GCP Data Analytics (Udemy.com)
Learn GCS for Data Lake, BigQuery for Data Warehouse, GCP Dataproc and Databricks for Big Data Pipelines
Created by: Durga Viswanatha Raju Gadiraju
Last updated November 2023
What you will learn
- Data Engineering leveraging Services under GCP Data Analytics
- Setup Development Environment using Visual Studio Code on Windows
- Building Data Lake using GCS
- Process Data in the Data Lake using Python and Pandas
- Build Data Warehouse using Google BigQuery
- Loading Data into Google BigQuery tables using Python and Pandas
- Setup Development Environment using Visual Studio Code on Google Dataproc with Remote Connection
- Big Data Processing or Data Engineering using Google Dataproc
- Run Spark SQL based applications as Dataproc Jobs using Commands
- Build Spark SQL based ELT Data Pipelines using Google Dataproc Workflow Templates
- Run or Instantiate ELT Data Pipelines or Dataproc Workflow Template using gcloud dataproc commands
- Big Data Processing or Data Engineering using Databricks on GCP
- Integration of GCS and Databricks on GCP
How it compares
Against the other 461 rated Udemy courses in Data Science on CourseDuck:
- Rating4.7 out of 5higher than 85% of them
- Reviews1,130#141 of 462 by number of reviews
- Length19.6 hourslonger than 79% of them
- Last updatedNovember 2023earlier than 78% of them
Course Description
Data Engineering is all about building Data Pipelines to get data from multiple sources into Data Lakes or Data Warehouses and then from Data Lakes or Data Warehouses to downstream systems. As part of this course, I will walk you through how to build Data Engineering Pipelines using GCP Data Analytics Stack. It includes services such as Google Cloud Storage, Google BigQuery, GCP Dataproc, Databricks on GCP, and many more.
As part of this course, first you will go ahead and setup environment to learn using VS Code on Windows and Mac.
Once the environment is ready, you need to sign up for Google Cloud Account. We will provide all the instructions to sign up for Google Cloud Account including reviewing billing as well as getting USD 300 Credit.
We typically use Cloud Object Storage as Data Lake. As part of this course, you will learn how to use Google Cloud Storage as Data Lake along with how to manage the files in Google Cloud Storage both by using commands as well as Python. It also covers, integration of Pandas with files in Google Cloud Storage.
GCP provides RDBMS as service via Cloud SQL. You will learn how to setup Postgresql Database Server using Cloud SQL. Once the Database Server is setup, you will also take care of setting up required application database and user. You will also understand how to develop Python based applications by integrating with GCP Secretmanager to retrieve the credentials.
One of the key usage of Data is nothing but building reports and dashboards. Typically reports and dashboards are built using reporting tools pointing to Data Warehouse. As part of Google Data Analytics Services, BigQuery can be used as Data Warehouse. You will learn the features of BigQuery as a Data Warehouse along with key integrations using Python and Pandas.
At times, we need to process heavy volumes of data which also known as Big Data Processing. GCP Dataproc is a fully manage Big Data Service with Hadoop, Spark, Kafka, etc. You will not only learn how to setup the GCP Dataproc cluster, but also you will learn how to use single node Dataproc cluster for the development. You will setup development environment using VS Code with remote connection to the Dataproc Cluster.
Once you understand how to get started with Big Data Processing using Dataproc, you will take care of building end to end ELT Data Pipelines using Dataproc Workflow Templates. You will learn all key commands to submit Dataproc Jobs as well as Workflows. You will end up building ELT Pipelines using Spark SQL.
While Dataproc is GCP Native Big Data Service, Databricks is another prominent Big Data Service available in GCP. You will also understand how to get started with Databricks on GCP.
Once you go through the details about how to get started with Databricks on GCP, you will take care of building end to end ELT Datapipelins using Databricks Jobs and Workflows.
Towards the end of the course you should be fairly comfortable with BigQuery for Data Warehouse and GCP Dataproc for Data Processing, you will learn how to integrate these two key services by building end to end ELT Data Pipeline using Dataproc Workflow. You will also understand how to include Pyspark based application with Spark BigQuery connector as part of the Pipeline.
In the process of building Data Pipelines, you will also revise application development life cycle of Spark, troubleshooting issues related to the spark using relevant web interfaces such as YARN Timeline Server, Spark UI, etc.
Instructor Details
- 4.7 Rating
1,130 Reviews
Durga Viswanatha Raju Gadiraju
20+ years of experience in executing complex projects using a vast array of technologies including Big Data and the Cloud.
ITVersity, Inc. - is a US-based organization that provides quality training for IT professionals and we have a track record of training hundreds of thousands of professionals globally.
Building an IT career for people with required tools such as high-quality material, labs, live support, etc to upskill and cross-skill is paramount for our organization.
At this time our training offerings are focused on the following areas:
* Application Development using Python and SQL
* Big Data and Business Intelligence
* Cloud
* Datawarehousing, Databases
More courses by Durga Viswanatha Raju Gadiraju
Data Engineering for Beginners: Learn SQL, Python & Spark (2026)
4.3 (8,867 Reviews)
Provider: Udemy
Time: 56h
$119.99
Data Engineering using AWS Data Analytics (2024)
4.5 (3,535 Reviews)
Provider: Udemy
Time: 25.4h
$119.99
Spark SQL and Spark 3 using Scala Hands-On with Labs (2023)
4.4 (3,127 Reviews)
Provider: Udemy
Time: 24.2h
$74.99
Master Apache Spark using Spark SQL and PySpark 3 (2024)
4.3 (2,655 Reviews)
Provider: Udemy
Time: 32.2h
$10.99
Data Engineering using Databricks on AWS and Azure (2024)
4.6 (2,138 Reviews)
Provider: Udemy
Time: 19h
$14.99
Databricks Certified Associate Developer - Apache Spark (2025)
4.0 (2,015 Reviews)
Provider: Udemy
Time: 14.4h
$10.99
More Data Science courses
Master statistics & machine learning: intuition, math, code (2026)
4.9 (3,077 Reviews)
Provider: Udemy
Time: 38.3h
$124.99
Reviews
By Htcnxtcourse on 10/4/2024
The course was very informative and well-structured. It provided a solid understanding of the concepts, and the examples were practical. I especially appreciated the real-world scenarios discussed throughout the lessons. However, I think the course could benefit from more interactive exercises or hands-on projects to reinforce learning. Overall, it's a great course for anyone looking to get a good foundation on this topic.
By Juan Antonio reyes nava on 5/6/2024
Acabo de terminar el curso de Data Analytics en GCP de Udemy y lo recomiendo a nuevos managers de ingeniería de datos en la plataforma de Google Cloud. El curso me brindó la base para comprender conceptos clave como arquitecturas de datos en la nube, herramientas de análisis de datos de GCP y casos de uso. Fue un curso bien organizado, fácil de seguir y con una mezcla perfecta de teoría y práctica. El instructor fue muy conocedor y apasionado. Si buscas una introducción práctica a la analítica de datos en GCP, este curso es una excelente opción.
By Shashwat Daga on 2/21/2024
This course is worth spending money, time and effort for. It has covered nearly all the scenarios associated with the Data Engineering. Obviously, I do not expect to learn all the tools and applications in a course but the tools that are used in this course are in high demand in the industry and thoroughly explained as well. I have gained a good amount of knowledge while learning. Thanks Udemy.
By Martin Levy Daniel on 8/21/2023
This course is a great intro GCP, giving a good overview of the essential tools for data engineering. As a data scientist working with GCP, it has helped me understand the ropes and feel more at ease in my daily tasks. The only little downside is that it leans towards a practical approach. It would've been nice to dive a bit deeper into theory and get some more detailed insights. But I get time is limited, and the main focus here is giving an introduction to those tools. In a nutshell, I'd totally recommend this course to anyone working in data with GCP who needs to overview the platform. Despite the practical approach, I would sign up again.
By Mikhail Askerka on 8/3/2023
I would like to be able to see the alignment between small tasks with the larger goals of the project. It's sometime unclear to me why a certain direction is taken. I do appreciate the rigour with regards to testing/checking
By Anonymized User on 7/12/2023
Excellent Delivery of concepts and great explanation. At first, I gave this feedback. But Later I thought that this wasn't enough. I have enrolled in many courses related to data engineering which requires setting up the environment and I was so confused following the instructions. But frankly speaking, I haven't come across anything like the way Mr. Durga explained the process of setting the environment for practicing in this course. It was error-free and easy to set up and the Q/A helps easily if we come across anything. Lucky to learn things from you, sir. Thanks a tonne!!!!!
By Niraj Yadav on 6/19/2023
this course is just awsome and made with utmost care and hard work, if you want to learn GCP, then i would highly recommend this course. thank you so much for sharing your in-depth knowledge with us. the way you teach is like an actual working environment and that gives a great confidence. very smartly touched the other popular technologies like airflow, spark, and dbt. go ahead and buy this course, you will surely not regret.
By Umesh Potdar on 3/19/2023
I wanted to take a moment to express my gratitude for the excellent training that you provided. Your knowledge, expertise, and ability to communicate complex concepts in a clear and concise manner were truly impressive. Overall, I feel that I have gained a comprehensive understanding of the topics covered, thanks to your excellent coverage of the material and I am excited to continue to develop my skills in this area. Thank You ! Umesh
By Karthik Sriram on 2/25/2023
1000 stars I will put if given a chance. This is the best course I have come across and in fact I subscribed for most of Durga sir courses. I spent lot of hours wasting time in other courses in coursers, you tube but I always feel incomplete either in terms of practical knowledge or in terms of applying skills in real time work. But this course would surely prepare for the work. I request sir to keep updating the existing courses you put on udemy in time to time basis whenever there is a new release or changes happening with GCP, AZURE or AWS so that we need not search for other courses. One more thing your support team is really helpful in clarifying doubts which is very important for the people who use online courses. Thanks to the support team.
By Shree Siva on 11/2/2022
This is one of the best ever course for GCP data engineering. This course helps to understand the concepts in a very easy way. The main difference which i see in this course is how to make production deployment which can be used in our real time office working projects. I have not seen any other course for GCP data engineering. Also if we want to learn and do real time production deployment in data engineering, all his courses(AWS,Azure,Spark,Python,Scala) are awesome .I bought all the courses. Really worth for the money you pay. Also I request author to create new data engineering course fully dedicated to real time streaming using Google Cloud Pub-Sub,kafka,cloud function,dataflow integration with cloud composer. Also expecting more advance courses using google cloud workflows(similar to microservices),cloud fusion with CI/CD pipeline
Quality Score
No CourseDuck member has rated this course yet. Taken it? Give each part a thumbs up or down.
Overall Score : 94 / 100












