Snowflake - Build and Architect Data Pipelines Using AWS - Section Overview - Snowflake with Python, Spark, and Airflow
Interactive Video
•
Information Technology (IT), Architecture
•
University
•
Practice Problem
•
Hard
Wayground Content
FREE Resource
This video tutorial covers writing Python scripts to interact with Snowflake tables and deploying them in AWS Glue. It introduces the concept of Pushdown in Spark 3.1 and demonstrates deploying a Spark job to transform data in Snowflake. The tutorial also explores setting up a managed Airflow cluster on AWS, connecting it with Snowflake, and creating a DAG with two tasks: one for direct communication with Snowflake and another for triggering a PySpark job. The video concludes with deploying the DAG in the Airflow cluster.
Read more
2 questions
Show all answers
1.
OPEN ENDED QUESTION
3 mins • 1 pt
Describe the two tasks involved in the Airflow DAG mentioned in the text.
Evaluate responses using AI:
OFF
2.
OPEN ENDED QUESTION
3 mins • 1 pt
What is the significance of deploying the DAG in the Airflow cluster?
Evaluate responses using AI:
OFF
Access all questions and much more by creating a free account
Create resources
Host any resource
Get auto-graded reports
Continue with Google
Continue with Email
Continue with Classlink
Continue with Clever
or continue with
Microsoft
%20(1).png)
Apple
Others
Already have an account?
