In Airflow, a DAG
–Directed Acyclic Graph – is a collection of all the tasks you want to run, organized in a way that reflects their relationships and dependencies.
A DAG is defined in a Python script, which represents the DAGs structure (tasks and their dependencies) as code.
How to run DAG in Airflow?
1.Log onto the Punjab Prod server using the credentials:
username : admin
password : admin
2.Trigger the DAG by clicking on the “Trigger DAG with Config” option.
3.Enter date and click on Trigger button
4.The Logs can be viewed by expanding on the DAG and choosing a stage for any module and
Clicking on the Log option.
Configure the Airflow variables
Key | Value |
---|---|
password | eGov@123 |
username | SYSTEMSU3 |
token | ZWdvdi11c2VyLWNsaWVudDo= |
tenantid | pg |
usertype | SYSYTEM |
totalulb_url | https://raw.githubusercontent.com/egovernments/punjab-mdms-data/master/data/pb/tenant/tenants.json |
Configure the connections
ConnectionId | Connection Type | Host | Port | Schema | Remark |
es_conn | ElasticSearch | elasticsearch-data-v1.es-cluster | 9200 | For the ES server | |
digit-auth | HTTP | https | For the auth api conenction |
Add Comment