Configure Dremio for STACKIT Intake integration

Overview

This guide explains how to configure Dremio to work with STACKIT Intake. You will create a dedicated technical user and grant it the necessary permissions. This process provides the credentials required for Intake to write data to your Dremio catalog.

Prerequisites

Before you begin, please ensure you have the following:

• You have a user with administrative privileges in your STACKIT Dremio environment
• You have access to the Dremio user interface

Step-by-Step configuration

1. Create the technical Dremio user

   A technical user is a dedicated, non-human Dremio account created for programmatic access by an application or service. It is a security best practice that ensures services connect to Dremio with their own isolated identity and limited permissions.

   In this case, the technical user will be used exclusively by the STACKIT Intake service. When an Intake receives data from a source stream, it will use this user’s identity and credentials to authenticate with Dremio and execute its primary function: writing the data into the target Iceberg table within the specified namespace.

   1. Navigate to the Users section in the Dremio administrative settings.
   2. Click on Add User (top right).
   3. Choose between a local or external user and fill in the account details. We recommend using a descriptive name, such as intake_write_user.
   4. Assign a password and save the user. Notice that roles will be granted at the catalog-level.
   If you can’t add a user, check that your account has sufficient permissions by checking Settings > Users.

2. Grant catalog permissions

   Next, give the new user the permission to create folders and write data to tables within your desired catalog.

   1. From the Dremio UI, select the catalog where Intake will write data.

   2. Click the Settings icon in the top-right corner.

   3. Open the Privileges tab.

   4. Find the user you just created (e.g., intake_write_user).

   5. Grant the following privileges: USAGE, CREATE FOLDER, COMMIT, and CREATE TABLE.

      

   6. Click Save.

   The CREATE FOLDER permission can be omitted if you prefer to create the target folders manually.

3. Generate a personal access token (PAT)

   Once you have created your user and you provided the required configurations for the folder that you will work with. You will generate a secure token for the new user. This token will be used for authentication instead of a password.

   1. Log out of from admin account and log in to Dremio as the new technical user (e.g., intake_write_user).

   2. Go to Account Settings in the bottom-left corner.

   3. Select the Personal Access Tokens tab.

   4. Click Generate Token.

   5. Give the token a descriptive name (e.g STACKIT Intake Service Access) and set an expiration time.

      

      Copy the generated token immediately and store it in a secure location. You will not be able to see it again after you close the window.

   6. Now that you have a technical user for Intake and the PAT token, use Setting Up Intakes Via The API to create Intakes.

4. Connection parameter for Intake