API Loki Alerts Observability

Prerequisites

1. You have created an Observability instance with Loki
2. You’re familiar with creating Alerts via API
3. Your Alertmanager is exposed
4. You have API credentials
5. Observability-API access

How to create Loki alerts with a new Alert Group

To create a new Loki Alert Group you can send a HTTP POST request to the following URL: https://argus.api.eu01.stackit.cloud/v1/projects/[projectId]/instances/[instanceId]/logs-alertgroups.

You have to provide your project ID and observability instance ID.

An example request body for creating a new Loki Alert Group follows:

Request Body example

{
 "name": "YourAlertGroupName",
 "interval": "30m",
 "rules": [
 {
 "alert": "error",
 "expr": "count_over_time({app=\"yourAppName\", severity=\"critical\"} |~ \".*error*\"[5m]) > 25",
 "for": "0m",
 "labels": {
 "app": "yourAppName",
 "project_name": "yourProjectName",
 "id": "yourAppId",
 "severity": "critical"
 },
 "annotations": {
 "summary": "To many errors are coming frequently!"
 }
 }
 ]
}

To create the Loki Alert Group from the example above you would issue the following HTTP request:

curl -X POST "https://argus.api.eu01.stackit.cloud/v1/projects/$PROJECT_ID/instances/$INSTANCE_ID/logs-alertgroups"
-H "accept: application/json" \
-H "Authorization: Bearer $API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"YourAlertGroupName","interval":"30m","rules":[{"alert":"error","expr":"count_over_time({app=\"yourAppName\", severity=\"critical\"} |~ \".*error*\"[5m]) > 25","for":"0m","labels":{"app":"yourAppName","project_name":"yourProjectName","id":"yourAppId","severity":"critical"},"annotations":{"summary":"To many errors are coming frequently!"}}]}'

Verify the creation of Loki Alert Groups and Loki Alert-Rules

You can verify the creation of your Loki Alert Group and Alert-Rules by issuing a HTTP GET request to the following API https://argus.api.eu01.stackit.cloud/v1/projects/[projectId]/instances/[instanceId]/logs-alertgroups.

You have to provide your project ID and observability instance ID.

How to create Loki alerts with existing Alert Group

1. You have to get the information that is already existing by calling https://docs.api.eu01.stackit.cloud/documentation/argus#tag/logs/operation/v1%5Fprojects%5Finstances%5Flogs-alertgroups%5Fread
   1. Copy that information you’ve got from the request (if it was successful)
2. To add a new alert just call https://docs.api.eu01.stackit.cloud/documentation/argus#tag/logs/operation/v1%5Fprojects%5Finstances%5Flogs-alertgroups%5Fupdate
   1. Paste the body which you have copied from step one
   2. Add the new rule to this group
   3. Send the request