BigQueryHook

Google

Interact with BigQuery. This hook uses the Google Cloud connection.

View on GitHub

Last Updated: Apr. 8, 2023

Access Instructions

Install the Google provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

gcp_conn_idThe Airflow connection used for GCP credentials.
delegate_toThis performs a task on one host with reference to other hosts.
use_legacy_sqlThis specifies whether to use legacy SQL dialect.
locationThe location of the BigQuery resource.
api_resource_configsThis contains params configuration applied for Google BigQuery jobs.
impersonation_chainThis is the optional service account to impersonate using short term credentials.
labelsThe BigQuery resource label.

Documentation

Interact with BigQuery. This hook uses the Google Cloud connection.

Example DAGs

Data Science Modeling Using Google Cloud Storage XCom Backend

A sample data science pipeline demonstrating extraction from BigQuery to modeling that uses an XCom backend in Google Cloud Storage to pass intermediary data between tasks.

Google
Data ScienceETL/ELT
MLFlow Example

Airflow can integrate with tools like MLFlow to streamline the model experimentation process. By using the automation and orchestration of Airflow together with MLflow's core concepts Data Scientists …

Google
AI + Machine LearningData Science
MLFlow Multi-Model Example

Airflow can integrate with tools like MLFlow to streamline the model experimentation process. By using the automation and orchestration of Airflow together with MLflow's core concepts Data Scientists …

Google
AI + Machine LearningData Science
MLFlow Multi-Model Config Example

Airflow can integrate with tools like MLFlow to streamline the model experimentation process. By using the automation and orchestration of Airflow together with MLflow's core concepts Data Scientists …

Google
AI + Machine LearningData Science
MLFlow Multi-Model Register Example

Airflow can integrate with tools like MLFlow to streamline the model experimentation process. By using the automation and orchestration of Airflow together with MLflow's core concepts Data Scientists …

Google
AI + Machine LearningData Science
MLFlow Multi-Model Register With Great Expectations Example

Airflow can integrate with tools like MLFlow to streamline the model experimentation process. By using the automation and orchestration of Airflow together with MLflow's core concepts Data Scientists …

Google
AI + Machine LearningData ScienceData Quality

Was this page helpful?