This is a technical, hands-on course that teaches you how to implement DevOps best practices to build data pipelines, and how to implement observability to maintain and monitor data pipeline health. The course focuses on the most practical Snowflake concepts, features, and tools to get you up and running quickly with these concepts.
You'll start by learning about DevOps, DevOps practices, and how DevOps fits into the context of data engineering. You'll incorporate source control, declarative management of database objects, continuous delivery, and use a command-line interface to implement DevOps best practices into a data pipeline. You'll specifically learn how to:
- Use Snowflake's git integration to add source control to your data pipeline
- Use GitHub for team-wide collaboration on your data pipeline
- Use CREATE OR ALTER to declaratively manage database objects
- Use GitHub Actions to implement continuous delivery for your pipeline
- Use Snowflake CLI to deploy changes into dedicated data environments
You'll also learn about observability, and how to implement it to maintain and monitor the health and performance of your data pipeline. You'll specifically learn how to:
- Use logs to keep a record of events that occur within your pipeline
- Use traces to maintain a detailed journey of events for operations in your pipeline
- Use alerts to monitor for specific conditions in your pipeline, and combine them with notifications to encourage action among team members if critical errors occur in the pipeline
Throughout the course, you'll follow along with the instructor using a combination of Snowflake, Visual Studio Code, GitHub, and the command line. The course is supplemented with readings containing resources to level up your understanding of specific concepts.
You'll come away understanding how to incorporate DevOps best practices into data pipelines, and how to use observability to monitor the health and performance of pipelines.
In this module, you'll understand how DevOps helps software development teams iterate safely and efficiently, and understand how those practices can be applied in the field of data engineering. You'll learn how to implement a few key DevOps best practices for data pipelines. Namely, you'll learn how to implement source control for pipeline objects, how to declaratively manage database objects, and how to introduce changes to dedicated data development environments using continuous integration. By the end of the module, you'll understand how data pipelines can be built collaboratively by large teams, and how they can be evolved efficiently and reliably.
涵盖的内容
12个视频5篇阅读材料1个作业
显示有关单元内容的信息
12个视频•总计72分钟
Scaling data pipelines to meet modern demands•4分钟
What this course will cover•3分钟
DevOps in the world of data engineering•4分钟
DevOps with Snowflake•3分钟
What we'll build•1分钟
Source control in Snowflake with git•8分钟
Set up the data pipeline using Snowflake CLI•11分钟
Database Change Management (DCM)•6分钟
Declarative approach with CREATE OR ALTER•14分钟
Continuous integration and continuous delivery (CI/CD) for data pipelines•4分钟
Implementing continuous delivery for our data pipeline•12分钟
Recap and best practices for DevOps with Snowflake•2分钟
5篇阅读材料•总计50分钟
How to successfully complete the course•10分钟
[IMPORTANT] Have Questions? Join the Q+A Forum for this course•10分钟
Clone your fork, install Snowflake CLI, and configure config.toml•10分钟
Weather data from Snowflake Marketplace•10分钟
Additional resources on DevOps with Snowflake•10分钟
1个作业•总计20分钟
Module 1 Assessment (Knowledge Check)•20分钟
Observability with Snowflake
第 2 单元•小时 后完成
单元详情
In this module, you'll learn about observability, and how it can be implemented to monitor the health and performance of data pipelines. You'll specifically learn about Snowflake's observability framework, Snowflake Trail, and how to implement its core components. You'll use event tables, logs, and traces to implement detailed records of events occurring within your data pipeline. You'll also learn how to generate alerts to detect specific conditions in your data environment, and how to combine them with notifications to communicate information to key stakeholders, like a broader data engineering team.
涵盖的内容
11个视频3篇阅读材料2个作业
显示有关单元内容的信息
11个视频•总计51分钟
Observability for data engineering•4分钟
Foundational concepts of observability•3分钟
Observability with Snowflake Trail•2分钟
Event Tables in Snowflake•4分钟
Logging in Snowflake•9分钟
Traces in Snowflake•8分钟
Alerts in Snowflake•8分钟
Notifications in Snowflake•8分钟
Observability with third-party tools•1分钟
Recap and best practices for observability with Snowflake•2分钟
Conclusion•1分钟
3篇阅读材料•总计30分钟
Clean up•10分钟
Additional resources on observability with Snowflake•10分钟
A single, global platform that powers the Data Cloud. Snowflake is uniquely designed to connect businesses globally, across any type or scale of data and many different workloads, and unlock seamless data collaboration.
When will I have access to the lectures and assignments?
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Certificate?
When you enroll in the course, you get access to all of the courses in the Certificate, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.