KylinCubeOperator
Apache KylinThis operator is used to submit request about kylin build/refresh/merge, and can track job status . so users can easier to build kylin job
Access Instructions
Install the Apache Kylin provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
kylin_conn_idThe connection id as configured in Airflow administration.
projectkylin project name, this param will overwrite the project in kylin_conn_id:
cubekylin cube name
dsn(dsn , dsn url of kylin connection ,which will overwrite kylin_conn_id. for example: kylin://ADMIN:KYLIN@sandbox/learn_kylin?timeout=60&is_debug=1)
command(kylin command include ‘build’, ‘merge’, ‘refresh’, ‘delete’, ‘build_streaming’, ‘merge_streaming’, ‘refresh_streaming’, ‘disable’, ‘enable’, ‘purge’, ‘clone’, ‘drop’. build - use /kylin/api/cubes/{cubeName}/build rest api,and buildType is ‘BUILD’, and you should give start_time and end_time refresh - use build rest api,and buildType is ‘REFRESH’ merge - use build rest api,and buildType is ‘MERGE’ build_streaming - use /kylin/api/cubes/{cubeName}/build2 rest api,and buildType is ‘BUILD’ and you should give offset_start and offset_end refresh_streaming - use build2 rest api,and buildType is ‘REFRESH’ merge_streaming - use build2 rest api,and buildType is ‘MERGE’ delete - delete segment, and you should give segment_name value disable - disable cube enable - enable cube purge - purge cube clone - clone cube,new cube name is {cube_name}_clone drop - drop cube)
start_timebuild segment start time
end_timebuild segment end time
offset_startstreaming build segment start time
offset_endstreaming build segment end time
segment_namesegment name
is_track_job(whether to track job status. if value is True,will track job until job status is in(“FINISHED”, “ERROR”, “DISCARDED”, “KILLED”, “SUICIDAL”, “STOPPED”) or timeout)
intervaltrack job status,default value is 60s
timeouttimeout value,default value is 1 day,60 * 60 * 24 s
eager_error_status(jobs error status,if job status in this list ,this task will be error. default value is tuple([“ERROR”, “DISCARDED”, “KILLED”, “SUICIDAL”, “STOPPED”]))
Documentation
This operator is used to submit request about kylin build/refresh/merge, and can track job status . so users can easier to build kylin job
For more detail information in Apache Kylin