返回到 ETL and Data Pipelines with Shell, Airflow and Kafka
IBM

ETL and Data Pipelines with Shell, Airflow and Kafka

Delve into the two different approaches to converting raw data into analytics-ready data. One approach is the Extract, Transform, Load (ETL) process. The other contrasting approach is the Extract, Load, and Transform (ELT) process. ETL processes apply to data warehouses and data marts. ELT processes apply to data lakes, where the data is transformed on demand by the requesting/calling application. In this course, you will learn about the different tools and techniques that are used with ETL and Data pipelines. Both ETL and ELT extract data from source systems, move the data through the data pipeline, and store the data in destination systems. During this course, you will experience how ELT and ETL processing differ and identify use cases for both. You will identify methods and tools used for extracting the data, merging extracted data either logically or physically, and for loading data into data repositories. You will also define transformations to apply to source data to make the data credible, contextual, and accessible to data users. You will be able to outline some of the multiple methods for loading data into the destination system, verifying data quality, monitoring load failures, and the use of recovery mechanisms in case of failure. By the end of this course, you will also know how to use Apache Airflow to build data pipelines as well be knowledgeable about the advantages of using this approach. You will also learn how to use Apache Kafka to build streaming pipelines as well as the core components of Kafka which include: brokers, topics, partitions, replications, producers, and consumers. Finally, you will complete a shareable final project that enables you to demonstrate the skills you acquired in each module.

中级课程小时

精选评论

JJ

5.0评论日期:Jul 22, 2023

Labs in this course are very helpful and to the point. It took me a while to complete this course but i learned a lot.

SK

5.0评论日期:Jan 20, 2025

Relevant information in recordings, good recap of every video and hand-on lesson in the end to concrete the knowledge.

HT

4.0评论日期:Mar 31, 2023

Course offers valuable conceptual content but labs could be improved. Coursera assessment system is really poor.

PM

4.0评论日期:Mar 23, 2023

it was good course should have also given an information on industry related solution and they can implement the same.

OG

4.0评论日期:Jun 21, 2022

It takes 1 hour to connect the lab and start the service.

TK

5.0评论日期:Jul 22, 2023

Great hands-on (alike the others in the pack)! Practical and interactive.

DS

5.0评论日期:Jun 13, 2022

Excellent introduction to this topics. Labs contain all you need to know how to start using this type of technologies. Highly recommended.

MD

5.0评论日期:Mar 11, 2023

Great learning course for Kafka/ Airflow, well presented

DR

4.0评论日期:Jun 3, 2022

Good introduction to Airflow and Kafka however only one airflow operator is explored

MT

5.0评论日期:Jun 22, 2024

Great content but the labs can be challenging to work with.

RR

5.0评论日期:Aug 27, 2024

Muy satisfecho con el contenido del curso, y los laboratorios. Thank you very much!

SG

5.0评论日期:Jul 12, 2023

Learn a lot about Apache Airflow, Kafka from sketch.

所有审阅

显示:20/98

Chris Barton
2.0
评论日期:Apr 20, 2022
Dmitry Kisselev
2.0
评论日期:Sep 17, 2021
Benjamin Anton Andrew
1.0
评论日期:Aug 20, 2022
Tal Melamed
2.0
评论日期:Jul 17, 2022
Nataliya Sashnikova
5.0
评论日期:Oct 12, 2021
RLee
5.0
评论日期:Jan 13, 2022
Евгений Дериглазов
5.0
评论日期:Sep 29, 2021
Santiago Zuluaga Ayala
3.0
评论日期:Sep 15, 2022
bengisu pınar
3.0
评论日期:Aug 17, 2023
Ilya Kolobov
5.0
评论日期:Jan 13, 2022
Gorana Bosic
4.0
评论日期:Oct 6, 2024
Omar Hegazy
4.0
评论日期:Jan 26, 2022
YANGYANG CAI
4.0
评论日期:Jan 17, 2022
BO WANG
1.0
评论日期:Jul 8, 2022
Harald Männle
5.0
评论日期:Sep 29, 2024
David Arango Sampayo
5.0
评论日期:Jun 14, 2022
Natale Foata
5.0
评论日期:Dec 15, 2021
Hugo Adrian Osorio Ortiz
5.0
评论日期:Dec 6, 2021
Chris Weaver
4.0
评论日期:Apr 3, 2022
Sina Salemi Sarmast
4.0
评论日期:May 7, 2022