SnowflakeOperatorAsync
Astronomer ProvidersSnowflakeOperatorAsync uses the snowflake python connector execute_async method to submit a database command for asynchronous execution.
Access Instructions
Install the Astronomer Providers provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
Documentation
SnowflakeOperatorAsync uses the snowflake python connector
execute_asyncmethod to submit a database command for asynchronous execution.Submit multiple queries in parallel without waiting for each query to complete.
Accepts list of queries or multiple queries with ‘;’ semicolon separated string and params. It loops through the queries and execute them in sequence. Uses execute_async method to run the query
Once a query is submitted, it executes the query from one connection and gets the query IDs from the response and passes it to the Triggerer and closes the connection (so that the worker slots can be freed up).
The trigger gets the list of query IDs as input and polls every few seconds to snowflake and checks for the query status based on the query ID from different connection.
- param snowflake_conn_id
Reference to Snowflake connection id
- param sql
the sql code to be executed. (templated)
- param autocommit
if True, each command is automatically committed. (default value: True)
- param parameters
(optional) the parameters to render the SQL query with.
- param warehouse
name of warehouse (will overwrite any warehouse defined in the connection’s extra JSON)
- param database
name of database (will overwrite database defined in connection)
- param schema
name of schema (will overwrite schema defined in connection)
- param role
name of role (will overwrite any role defined in connection’s extra JSON)
- param authenticator
authenticator for Snowflake. ‘snowflake’ (default) to use the internal Snowflake authenticator ‘externalbrowser’ to authenticate using your web browser and Okta, ADFS or any other SAML 2.0-compliant identify provider (IdP) that has been defined for your account ‘https://<your_okta_account_name>.okta.com’ to authenticate through native Okta.
- param session_parameters
You can set session-level parameters at the time you connect to Snowflake
- param handler
(optional) the function that will be applied to the cursor (default: fetch_all_handler).
- param return_last
(optional) if return the result of only last statement (default: True).
- param poll_interval
the interval in seconds to poll the query