Grafana/Signoz Webhook Error Code 400

[roomit-xyz]

Dear Dev,

I have deployed matrix-webhook

If I hit manual using curl, All is running well

curl -d '{"body":"Test WEBHOOK MATRIX", "key": "xxxxxxxxxxxxx"}'   'http://matrix-webhook.xxxxxx.xxx:8443/!lcGnHYvcRftvBdvRgr:matrix.xxxxxx.xxx'

{"status": 200, "ret": "OK"}

And when I added alert I webhook in signoz or grafana I got error 400 code.

level=error ts=2024-03-26T07:42:18.583Z caller=api.go:808 component=api version=v1 msg="API error" err="server_error: unexpected status code 400:

I have added in last URL for grafana format

?formatter=grafana&key=MYTOKEN

like :

http://webhook.xxxxxx.xxx:8443/!lcGnHYvcRsftvBdvxRgr:webhook.xxxxxx.xxx?formatter=grafana&key=MYTOKEN

[nim65s]

Hi @roomit-xyz,
Can you provide the logs of the matrix-webhook app in this case ? Maybe incrementing the log level to the max first ?

[roomit-xyz]

I have increased VERBOSE variable in .env from 1 - 4, but log still not show. Any configuration for show log?

[nim65s]

This is not a feature, yet, I'll fix this, thanks.
For now, to increase the verbosity to the maximum, we need to pass -vvvv

[roomit-xyz]

Hallo @nim65s

This following log

2024-04-02 04:41:27,729 - matrix_webhook.handler - 21 - DEBUG - Handling request=<BaseRequest POST /!yyyyyyy:xxx.xxx>
2024-04-02 04:41:27,729 - matrix_webhook.utils - 37 - DEBUG - Creating json response: status=<HTTPStatus.BAD_REQUEST: 400>, ret='Missing body'
2024-04-02 04:41:27,729 - aiohttp.access - 211 - INFO - 172.22.0.6 [02/Apr/2024:04:41:27 +0000] "POST /!yyyyyyy:xxxx.xxx?formatter=grafana&key=TOKEN HTTP/1.1" 400 205 "-" "Alertmanager/0.23.0"

Best Regards
Dwi

[roomit-xyz]

[roomit-xyz]

Any update?

[nim65s]

Thanks for the heads up, I had missed your previous comments.

So yes, the error tells you that the received payload is lacking a body.
I looked at https://signoz.io/docs/alerts-management/notification-channel/webhook/#sample-webhook-message and see that we need to format these data to display them.

You can spend some time writing a python function that take this dict as entry and output a nicely formated markdown string, with all the data in a pretty way, or we could throw this at pprint, and call it a day, eg.:

>>> from pprint import pprint
>>> pprint(data["alerts"])
[{'annotations': {'info': 'The disk sda3 is running full',
                  'summary': 'please check the instance example1'},
  'endsAt': '0001-01-01T00:00:00Z',
  'fingerprint': 'ad592b0afcbe2e79',
  'generatorURL': '',
  'labels': {'alertname': 'DiskRunningFull',
             'dev': 'sda3',
             'instance': 'example3',
             'severity': 'critical'},
  'startsAt': '2022-04-25T14:35:19.490146+05:30',
  'status': 'firing'}]

or yaml:

>>> print(yaml.dump(data["alerts"]))
- annotations:
    info: The disk sda3 is running full
    summary: please check the instance example1
  endsAt: '0001-01-01T00:00:00Z'
  fingerprint: ad592b0afcbe2e79
  generatorURL: ''
  labels:
    alertname: DiskRunningFull
    dev: sda3
    instance: example3
    severity: critical
  startsAt: '2022-04-25T14:35:19.490146+05:30'
  status: firing