Implementing Cross Task Communication (XCom) in STACKIT Workflows

Airflow’s TaskFlow API integrates seamlessly with XCom. You can pass data automatically between tasks or explicitly pull it when needed. In this tutorial you’ll learn how to use XCom to share values between tasks in two ways and insert a transformation step in between.

1. Create a DAG in your project

   dags/my_xcom_dag.py

   from datetime import datetime
   from airflow.decorators import dag, task
   
   @dag(
       dag_id="02_xcom",
       start_date=datetime(2024, 1, 1),
       schedule=None,
       catchup=False,
       tags=["demo", "airflow", "xcom"],
   )
   def xcom_taskflow_demo():
   
       @task(task_id="produce_value")
       def produce() -> str:
           value = "data-from-upstream"
           print(f"[produce] Producing value: {value}")
           return value
   
       @task(task_id="consume_via_param")
       def consume_via_param(transformed_value: str):
           print(f"[consume_via_param] Received via param: {transformed_value}")
   
       @task(task_id="consume_via_pull")
       def consume_via_pull():
           from airflow.operators.python import get_current_context
           ti = get_current_context()["ti"]
           pulled = ti.xcom_pull(task_ids="produce_value", key="return_value")
           print(f"[consume_via_pull] Pulled from XCom: {pulled}")
   
       # Automatic pass chain
       produced = produce()
       consume_via_param(produced)
   
       # Explicit pull chain
       pull_task = consume_via_pull()
       produced >> pull_task
   
   dag = xcom_taskflow_demo()

   How it works

   • produce_value returns a string; TaskFlow stores it in XCom under key return_value.
   • consume_via_param gets the produced value automatically as a function argument.
   • consume_via_pull reads the original value explicitly via ti.xcom_pull(...).

2. Push the DAG to your environment and trigger it in Airflow.

   1. Open the task logs:

      • consume_via_param prints the uppercased value.
      • consume_via_pull prints the original value.

   2. In the Airflow UI, inspect the XCom entry for produce_value.

   After execution you’ll have seen both ways of working with XCom:

   • Automatic pass (via function arguments)
   • Explicit pull (via xcom_pull)