API Alertmanager Receiver Observability

Possible receivers

You can configure the following possible receivers:

• email
• webhook (Google Chat, Microsoft Teams or other)
• Opsgenie

How to add an email receiver

To create an email receiver use the following URL with your project ID and Observability instance ID:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/receivers

With the following HTTP message body you have the minimum configuration to add a new email receiver:

{
 "name": "MyEmailReceiver",
 "emailConfigs": [
 {
 "to": "emailTo@your-company.com",
 }
 ],
}

Per default Observability will send all messages from a STACKIT Observability email address. Optionally, you can configure your own email address as sender address. In this case you need to provide SMTP credentials:

{
 "name": "MyEmailReceiver",
 "emailConfigs": [
 {
 "to": "emailTo@your-company.com",
 "from": "observability-alerting@your-company.com",
 "smarthost": "your-company.com",
 "authUsername": "username",
 "authPassword": "password",
 "authIdentity": "yourIdentity"
 }
 ],
}

More information about the API and the request body can be found in the API documentation.

Create a route using the receiver

You will also need to create an alert config route for the receiver you just created using the URL:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/routes

{
 "receiver": "MyEmailReceiver",
 "groupBy": [
 "LatencyHigh"
 ],
 "groupWait": "30s",
 "groupInterval": "5m",
 "repeatInterval": "4h"
}

More information on the expected request body can be found in the API documentation.

How to add a Webhook receiver

You can use any compatible webhook. In case you are using a Google Chat or Microsoft Teams webhook you have to add a specific parameter as you can see in the example below.
To create a webhook receiver use the following URL with your project ID and Observability instance ID:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/receivers

Webhook examples

With the following HTTP message body examples you have the minimum configuration to add a new webhook receiver.
More information about the API and the request body can be found in the API documentation.

Google Chat Webhook

{
 "name": "MyGoogleChatWebhookReceiver",
 "webHookConfigs": [
 {
 "googleChat": true,
 "url": "https://chat.googleapis.com/v1/spaces/xxx/messages?key=xxx&token=xxx"
 }
 ]
}

MS Teams Webhook

{
 "name": "MyMsTeamsWebhookReceiver",
 "webHookConfigs": [
 {
 "msTeams": true,
 "url": "string"
 }
 ]
}

Custom Webhook

{
 "name": "MyWebhookReceiver",
 "webHookConfigs": [
 {
 "url": "string"
 }
 ]
}

Create a route using the receiver

You will also need to create an alert config route for the receiver you just created using the following URL and body. Please make sure to use the same name of the receiver you created.

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/routes

{
 "receiver": "MyReceiverName",
 "groupBy": [
 "LatencyHigh"
 ],
 "groupWait": "30s",
 "groupInterval": "5m",
 "repeatInterval": "4h"
}

More information on the expected request body can be found in the API documentation.

How to add an Opsgenie receiver

You can also configure your own Opsgenie. You will need your Observability instance ID to interact with the Observability API.
To create a webhook receiver use the following URL with your project ID and Observability instance ID:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/receivers

With the following HTTP message body, you have the minimum configuration to add a new Opsgenie receiver:

{
 "name": "MyOpsGenieReceiver",
 "opsgenieConfigs": [
 {
 "apiKey": "yourApiKey",
 "apiUrl": "yourApiURL",
 "tags": "TagsYouWantToHave",
 "httpConfig": {
 "proxyUrl": "yourProxyServerURL"
 }
 }
 ],
}

More information about the API and the request body can be found in the API documentation.

Create a route using the receiver

You will also need to create an alert config route for the receiver you just created using the URL:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/routes

{
 "receiver": "MyOpsGenieReceiver",
 "groupBy": [
 "LatencyHigh"
 ],
 "groupWait": "30s",
 "groupInterval": "5m",
 "repeatInterval": "4h"
}

More information on the expected request body can be found in the API documentation.

How to check the configuration

Basic syntax checking is provided by the Observability API.

Grafana visualizes triggered Alerts:

• Log in to Grafana
• Select the function Explore
• Select data source Thanos
• Choose metric ALERTS
• And visualize metric with Run query

If you don’t get any mail, please check your SPAM folder. Sometimes mail agents filter alert emails, because they contain hyperlinks. Additionally, you should check if you’re using a global configuration for all email receivers. If yes, verify the global section of the response from the GET alertconfigs endpoint. It should look like the response example of the next chapter “How to reset the global configuration”.

How to reset the global configuration

By default, Observability sends all emails from a STACKIT Observability address. You can optionally configure your own email as sender address. In this case, you need to provide SMTP credentials. You either can configure your individual email sender address globally or on a per email receiver basis. To set it globally, use the global section of the payload sent to the PUT alertconfigs endpoint.

If you want to switch back to using our default email address for sending alerts, you can use the following JSON body with a minimal configuration for the PUT alertconfigs endpoint.

Be careful: If you leave the global section empty as it is shown in this example, everything inside will be overwritten with our default values.

{
 "route": {
 "receiver": "email-example-receiver",
 "groupWait": "30s",
 "groupInterval": "5m",
 "repeatInterval": "4h"
 },
 "receivers": [
 {
 "name": "email-example-receiver",
 "emailConfigs": [
 {
 "to": "your-email@example.com",
 "sendResolved": true
 }
 ]
 }
 ]
}

Afterwards, you can check the configuration using the GET alertconfigs endpoint. You should see a global section like shown below. Attributes masked by ’*****’ indicate that our credentials will be used to send emails.

"global": {
 "resolveTimeout": "5m",
 "smtpFrom": "our-email@example.com",
 "smtpSmarthost": "*****",
 "smtpAuthUsername": "*****",
 "smtpAuthPassword": "*****",
 "smtpAuthIdentity": "*****"
}

How to change your receiver configuration

You can consult the API documentation to learn how to configure a recipient.

How to read your receiver configuration

To read all your receiver configurations send a HTTP GET request to the following URL:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/receivers

You will need to insert your own project ID and Observability instance ID. More information on this can be found in the API documentation.

How to delete your receiver configuration

To delete a receiver configuration send a HTTP PUT request to the following URL:

https://argus.api.eu01.stackit.cloud/v1/projects/{projectId}/instances/{instanceId}/alertconfigs/receivers/{receiver}

To remove a receiver do not include it in the HTTP PUT request body you send to the URL. Keep in mind that you always need a valid configuration, therefore you can make use of template values. More information on this can be found in the API documentation.