![](/astronomer-providers.png)
BigQueryGetDataOperatorAsync
Astronomer ProvidersFetches the data from a BigQuery table (alternatively fetch data for selected columns) and returns data in a python list. The number of elements in the returned list will be equal to the number of rows fetched. Each element in the list will again be a list where element would represent the columns values for that row.
Access Instructions
Install the Astronomer Providers provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
Documentation
Fetches the data from a BigQuery table (alternatively fetch data for selected columns) and returns data in a python list. The number of elements in the returned list will be equal to the number of rows fetched. Each element in the list will again be a list where element would represent the columns values for that row.
Example Result: [['Tony', '10'], ['Mike', '20'], ['Steve', '15']]
Note
If you pass fields to selected_fields
which are in different order than the order of columns already in BQ table, the data will still be in the order of BQ table. For example if the BQ table has 3 columns as [A,B,C]
and you pass ‘B,A’ in the selected_fields
the data would still be of the form 'A,B'
.
Example:
get_data = BigQueryGetDataOperator(task_id='get_data_from_bq',dataset_id='test_dataset',table_id='Transaction_partitions',max_results=100,selected_fields='DATE',gcp_conn_id='airflow-conn-id')