Create and Manage Intakes

Diese Seite ist noch nicht in deiner Sprache verfügbar. Englische Seite aufrufen

Prerequisites

To begin creating and managing your Intakes via the STACKIT CLI, ensure you have the following in place:

You have a STACKIT customer account: Create a customer Account
You have a STACKIT user account: Create a user account
You have a STACKIT project: Create a project

You have installed the STACKIT CLI on your system: User Guide
You have configured Dremio and possess the necessary Dremio Personal Access Token (PAT), Catalog URI, Warehouse name, and Token Endpoint URL: Configure Dremio for STACKIT Intake integration
You have created an Intake Runner: Create and Manage and Intake Runners

Create an Intake

STACKIT CLI

An Intake defines the data pipeline that connects your data stream to the target Dremio Iceberg table.

Decide on Dremio Connection and Partitioning

The Intake must be configured with details about the Dremio Iceberg REST catalog into which to write the data stream. Also, one needs to decide about the partitioning scheme:

none: Data is not partitioned. This is the default. For production workloads, this may not be advisable since table optimizations such as compactions would conflict with the arrival of new data.
intake-time: Data is partitioned by day based on the time Intake received the data. This time is reflected in the field __intake_ts which Intake automatically adds to each received message. This allows you to perform optimizations on older partitions while data is still arriving in the current day’s partition.
manual: You can specify an arbitrary field of your data stream to use for partitioning.

Flags

Flag	Description	Options	Default	Optional
display-name	a human-readable name for the Intake	String, not longer than 32 characters
labels	labels in key=value format. Allows one to classify the purpose of the Intake	key=value format, separated by commas. Example: “key1=value1,key2=value2”	[]	✓
runner-id	the ID of the Intake Runner into which the Intake will be deployed	UUID of the runner
catalog-table-name	the name of the Iceberg table to receive the data stream	String, not longer than 1024 characters	”intake-<INTAKE_ID>“	✓
catalog-warehouse	the name of the warehouse in the catalog	Usually `catalog-s3`
catalog-namespace	the namespace in which the Iceberg table gets created in the catalog	String, not longer than 1024 characters	”intake”	✓
catalog-partition-by	the partition expression / field to use for partitioning, if catalog-partitioning is not none	Name of a field or Iceberg partitioning expression	If partitioning is intake-time, this defaults to `day(__intake_ts)`	✓
catalog-partitioning	the partition scheme to apply	One of:<br>• none<br>• intake-time<br>• manual	none	✓
dremio-pat	the PAT of the user to access Dremio	String, not longer than 1024 characters
catalog-uri	the URL of the Dremio Iceberg REST catalog endpoint	Usually `https://dremio-<your-dremio-instance-name>-catalog.data-platform.stackit.run/iceberg/`
dremio-token-endpoint	the URL of the Dremio OAuth 2 token endpoint	Usually `https://dremio-<your-dremio-instance-name>.data-platform.stackit.run/oauth/token`
catalog-auth-type	which authentication scheme to follow for the catalog	One of:<br>• none<br>• dremio<br>Set to dremio if you want to connect to Dremio.

Execute the creation command

To create your Intake, execute the following command, providing the relevant flags:

stackit beta intake create --runner-id <RUNNER_ID> --display-name <NAME> --catalog-uri <URI> --catalog-warehouse <WAREHOUSE> --catalog-auth-type --dremio-token-endpoint <TOKEN_ENDPOINT> --dremio-pat <PAT> --catalog-partitioning <PARTITION_SCHEME> --catalog-partition-by <PARTITION_FIELD>

After executing the command you need to confirm with y. When the instance is created successfully, the CLI will return the instance ID:

Created Intake for Intake Runner "3fa81897-0cfe-445e-9133-9bba744ff2c6". Intake ID: 17802315-32c2-48ce-a33d-3043d8aec89b

Note that the CLI will validate the Dremio endpoint URLs and the PAT before creating an Intake. In case there is a problem with these values, you will receive an error message describing the problem.

Example

This example creates an Intake on the runner f1e2d3c4-a5b6-7890-1234-567890abcdef, partitions data by ingestion time, and targets a specific table:

stackit beta intake create --runner-id 3fa81897-0cfe-445e-9133-9bba744ff2c6 \
--display-name iot-telemetry-daily \
--catalog-uri "https://dremio-iot-catalog.data-platform.stackit.run/iceberg/" \
--catalog-warehouse "catalog-s3" \
--catalog-auth-type dremio \
--dremio-token-endpoint "https://dremio-iot.data-platform.stackit.run/oauth/token" \
--dremio-pat "MY_SECURE_DREMIO_PAT_12345" \
--catalog-table-name "telemetry_logs" \
--catalog-partitioning intake-time

View an Intake

You can examine the details of an Intake with the describe command.

Argument	Description
INTAKE_ID	The ID of the Intake you would like to update

stackit beta intake describe <INTAKE_ID>

You will see a response like the following:

ATTRIBUTE            │ VALUE
──────────────────────┼───────────────────────────────────────────────────────────────────────
ID                   │ 17802315-32c2-48ce-a33d-3043d8aec89b
Name                 │ PurchaseOrderService
State                │ active
Runner ID            │ 3fa81897-0cfe-445e-9133-9bba744ff2c6
Created              │ 2025-10-16 12:26:21 +0000 UTC
Labels               │ <nil>
──────────────────────┼───────────────────────────────────────────────────────────────────────
Ingestion URI        │ 3fa81897-0cfe-445e-9133-9bba744ff2c6.intake.eu01.onstackit.cloud:9094
Topic                │ intake-17802315-32c2-48ce-a33d-3043d8aec89b
Dead Letter Topic    │ deadletter-intake-17802315-32c2-48ce-a33d-3043d8aec89b
Undelivered Messages │ 0
──────────────────────┼───────────────────────────────────────────────────────────────────────
Catalog URI          │ https://dremio-intake-catalog.data-platform.stackit.run/iceberg/
Catalog Warehouse    │ catalog-s3
Catalog Namespace    │ intake-demo
Catalog Table Name   │ purchase_order_events
Catalog Partitioning │ intake-time

List all Intakes

STACKIT CLI

To list all Intakes provisioned in your cloud project, call the list command:

Flag	Description
limit	Maximum number of entries to list.

stackit beta intake list

This will result in output like the following:

ID                                   │ NAME                 │ STATE  │ RUNNER ID
──────────────────────────────────────┼──────────────────────┼────────┼──────────────────────────────────────
17802315-32c2-48ce-a33d-3043d8aec89b │ PurchaseOrderService │ active │ 3fa81897-0cfe-445e-9133-9bba744ff2c6

Update an Intake

STACKIT CLI

You can update the properties of an existing intake with the update command.

Argument	Description
INTAKE_ID	The ID of the Intake you would like to update

Flag	Description	Options
display-name	a human-readable name for the Intake	String, not longer than 32 characters
catalog-uri	the URL of the Dremio Iceberg REST catalog endpoint	Usually `https://dremio-<your-dremio-instance-name>-catalog.data-platform.stackit.run/iceberg/`
catalog-warehouse	the name of the warehouse in the catalog	Usually `catalog-s3`
dremio-token-endpoint	the URL of the Dremio OAuth 2 token endpoint	Usually `https://dremio-<your-dremio-instance-name>.data-platform.stackit.run/oauth/token`
dremio-pat	the PAT of the user to access Dremio	String, not longer than 1024 characters
catalog-auth-type	which authentication scheme to follow for the catalog	One of:<br>• none<br>• dremio<br><br>Set to dremio if you want to connect to Dremio.
catalog-namespace	the namespace in which the Iceberg table gets created in the catalog	String, not longer than 1024 characters
catalog-table-name	the name of the Iceberg table to receive the data stream	String, not longer than 1024 characters
catalog-partitioning	the partition scheme to apply	One of:<br>• none<br>• intake-time<br>• manual
catalog-partition-by	the partition expression / field to use for partitioning, if catalog-partitioning is not none	Name of a field or Iceberg partitioning expression
labels	labels in key=value format. Allows one to classify the purpose of the Intake	key=value format, separated by commas. Example: “key1=value1,key2=value2”

Execute the update command

To update your Intake, execute the command providing the ID of the Intake Runner and the relevant flags:

stackit beta intake update <INTAKE_ID> --display-name <NAME> --catalog-uri <URI> --catalog-warehouse <WAREHOUSE> --catalog-auth-type --dremio-token-endpoint <TOKEN_ENDPOINT> --dremio-pat <PAT> --catalog-partitioning <PARTITION_SCHEME> --catalog-partition-by <PARTITION_FIELD>

When the update command has been sent, the CLI will return respond with a text like this:

Update Request for Intake "17802315-32c2-48ce-a33d-3043d8aec89b" sent successfully.

Examples

Change the labels of an Intake:

stackit beta intake update 17802315-32c2-48ce-a33d-3043d8aec89b --labels "dept=billing,stage=test"

Refresh the PAT for the catalog connection of an Intake:

stackit beta intake update 17802315-32c2-48ce-a33d-3043d8aec89b --dremio-pat "NEW_SUPER_SECRET_DREMIO_PAT"

Delete an Intake

STACKIT CLI

Deleting an Intake is an irreversible operation. It will also delete all associated Intake Users if you pass in the —force flag. If you do not pass force, the state of the runner will change to deleting. It will stay in that state and continue to function until all associated Intake Users have been deleted by you manually.

Argument	Description
INTAKE_ID	The ID of the Intake you would like to delete

Flag	Description
force	Delete Intake Users as well

stackit beta intake  delete <INTAKE_ID> --force

You have to confirm the deletion of the Intake Runner by pressing y. The delete command will result in output like this:

Deletion request for Intake "ef907939-6fcd-42b0-a635-1b33d3ba7cf3" sent successfully.