Imagine deploying schema changes with confidence—knowing your pipeline will handle them gracefully, consumers will stay healthy, and your data will stay consistent. That's the difference between hoping your CDC pipeline works and knowing it will. In this course you will learn how to build a working, vendor‑neutral CDC pipeline and a single, unified table from evolving source schemas. Starting with Debezium streaming changes from Postgres/MySQL into Kafka, you will use Schema Registry to enforce compatibility, then apply streaming SQL in Flink (or ksqlDB) to map, cast, and merge divergent fields into a canonical model. Finally, you will persist results to an Apache Iceberg table and query it instantly with Trino. Along the way, you’ll learn practical strategies to manage schema drift, choose compatibility modes (backward/full), and avoid breaking downstream consumers. Everything runs locally with Docker so you can reproduce it anywhere and take the same patterns to your cloud stack later.
This course is designed for engineers working with Kafka, Debezium, and streaming SQL who need reliable schema evolution and canonical modeling skills.
Learners should be familiar with Basic SQL, Docker, and familiarity with Kafka or streaming concepts.
By the end of the course,you will be able to implement a small end‑to‑end CDC pipeline that streams from a source DB and unifies evolving schemas into a single queryable table.
Deploy a local Debezium, Kafka, Schema Registry, and Flink/ksqlDB stack to observe row-level changes in real-time. Intentionally modify the source schema, then employ streaming SQL to map, cast, and coalesce fields into a canonical table. Perform upserts using stable keys and verify the data is correctly stored in Iceberg. By the conclusion, you will have established an operational CDC loop and a unified, queryable dataset.
涵盖的内容
4个视频2篇阅读材料1个作业
显示有关单元内容的信息
4个视频•总计37分钟
Introduction and Welcome•4分钟
CDC to Analytics: Complete Architecture Overview•11分钟
Data Flow Deep Dive: Source to Lakehouse•12分钟
Live Build: Unify Schemas with Streaming SQL•10分钟
2篇阅读材料•总计10分钟
Welcome to the Course: Course Overview•5分钟
Schema Evolution Additional Resources•5分钟
1个作业•总计30分钟
Hands On Learning (HOL): CDC Basics & Safe Schema Evolution•30分钟
Operate the Pipeline: Registry Rules & Recovery
第 2 单元•小时 后完成
单元详情
Learn to prevent consumer disruptions by enforcing compatibility at both the subject and global levels. We will deliberately deploy an incompatible schema, observe the failure, and proceed safely using defaults and transitive modes. Implement practical safeguards such as CI schema checks, DLQs, alerts, and lag probes to ensure issues are promptly identified and contained. The emphasis is on repeatable recovery, not heroics.
涵盖的内容
3个视频1篇阅读材料1个作业
显示有关单元内容的信息
3个视频•总计30分钟
From Debezium to Kafka: Wiring CDC with Schema Registry•11分钟
Break a Schema on Purpose: And Fix It•9分钟
Observability & Guardrails•10分钟
1篇阅读材料•总计5分钟
Compatibility Modes in Practice•5分钟
1个作业•总计30分钟
Hands On Learning (HOL): Fix a Breaking Change•30分钟
Canonical Models, Iceberg Sinks & Fast Queries
第 3 单元•小时 后完成
单元详情
Develop a robust canonical model encompassing naming conventions, data types and units, nullability, and soft delete mechanisms, and store it in Iceberg on MinIO utilizing streaming upserts. Perform immediate queries with Trino and employ time-travel features for validation or debugging regressions. The project involves constructing a denormalized “latest per customer” view for analytical purposes, as well as discussing partitioning strategies, equality deletes, and data compaction. Participants will acquire scalable patterns suitable for deployment from laptops to cloud environments.
Coursera brings together a diverse network of subject matter experts who have demonstrated their expertise through professional industry experience or strong academic backgrounds. These instructors design and teach courses that make practical, career-relevant skills accessible to learners worldwide.
When will I have access to the lectures and assignments?
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.