You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Logs and metrics correlation is very powerful concept that easy to achieve in Grafana with alloy, if logs/metrics are collected with the same agent.
However, it is not always the case:
For network devices logs can be received via syslog protocol or other push protocol, while metrics could be collected with snmp_exporter.
Proposal
To make sure that logs/metrics correlation is possible even with logs pushed without any additional metadata/labels, I would like to propose a component similar to https://www.rsyslog.com/doc/configuration/lookup_tables.html but instead of files, powerful prometheus discovery components can be used, since good probability that those could also be used for snmp metrics collection as well.
Examples:
1)Enrich Network devices syslog messages with device metadata(as labels) from IP address management (IPAM)/Data center infrastructure management(DCIM) like Netbox (https://github.com/FlxPeters/netbox-plugin-prometheus-sd). It could be same labels as used by prometheus.exporter.snmp or other metric scrapers.
2) Ensure metrics and logs use same labelset (gathered from same discovery.* component) even if metrics are scraped from prometheus endpoints and logs are received via loki.source.api.
Going forward, i created a component loki.enrich that could just do exactly that: PR,
However there is also function array.combine_maps that does something similar, but, unfotrunately not working for logs scenarios describred right now.
The text was updated successfully, but these errors were encountered:
Background
Logs and metrics correlation is very powerful concept that easy to achieve in Grafana with alloy, if logs/metrics are collected with the same agent.
However, it is not always the case:
Proposal
To make sure that logs/metrics correlation is possible even with logs pushed without any additional metadata/labels, I would like to propose a component similar to https://www.rsyslog.com/doc/configuration/lookup_tables.html but instead of files, powerful prometheus discovery components can be used, since good probability that those could also be used for snmp metrics collection as well.
Examples:
1)Enrich Network devices syslog messages with device metadata(as labels) from IP address management (IPAM)/Data center infrastructure management(DCIM) like Netbox (https://github.com/FlxPeters/netbox-plugin-prometheus-sd). It could be same labels as used by prometheus.exporter.snmp or other metric scrapers.
2) Ensure metrics and logs use same labelset (gathered from same discovery.* component) even if metrics are scraped from prometheus endpoints and logs are received via loki.source.api.
Going forward, i created a component loki.enrich that could just do exactly that:
PR,
However there is also function array.combine_maps that does something similar, but, unfotrunately not working for logs scenarios describred right now.
The text was updated successfully, but these errors were encountered: