logo
Published on

Crazy good Observability using Grafana Alloy

Authors
  • avatar
    Name
    Athos Georgiou
    Twitter
alt text

🚀 Hey there!

Super excited to share this example, as I've had a ton of fun putting it together. It sets up a complete Grafana Alloy observability stack with just a few commands and Docker compose. The environment is pre-configured with datasources and dashboards for monitoring Grafana Alloy.

If you want to skip this guide and jump straight to the code, here you go: https://github.com/athrael-soju/grafana-alloy-example.

My inspiration comes from the official Grafana Alloy project example, which unfortunately isn't working right now. So, I decided to fix and improve it. You can find the original project here.

Prerequisites

Ensure that Docker and Docker Compose are installed and running on your system.

NOTE: This environment is not intended for production use, and is maintained on a best-effort basis.

By default, only Grafana and databases are exposed:

  • Prometheus, for collecting metrics (localhost:9090)
  • Node Exporter, for collecting host metrics (localhost:9100)
  • Grafana, for visualizing telemetry (localhost:3000)
  • Grafana Mimir, for storing metrics (localhost:9009)
  • Grafana Loki, for storing logs (localhost:3100)
  • Grafana Tempo, for storing traces (localhost:3200)
  • Grafana Pyroscope, for storing profiles (localhost:4040)

Grafana is automatically provisioned with the appropriate datasources and dashboards for monitoring Grafana Alloy.

Running Alloy

Alloy can either be run locally or within Docker Compose. The example configuration can be used to send self-monitoring data from a local Alloy to the various databases running in Docker Compose.

To run Alloy within Docker Compose, pass --profile=alloy to docker compose when starting and stopping the environment:

docker compose --profile=alloy up -d

docker compose --profile=alloy down

Usage

  • Proceed to localhost:12345 to view the dashboard:

    alt text

  • Click the graph button to see all services and how they interconnect:

    alt text

  • To view grafana dashboards, proceed to localhost:4000/dashboards. There you will see a set of Alloy dashboards that have been provisioned, as well as a Node Exporter dashboard.

    alt text

  • To view logs, proceed to localhost:4000/explore. You can:

    • Select the Loki datasource and query logs (You'd have to select an instance).
    • Select the Tempo datasource and query traces.
    • Select the Pyroscope datasource and query profiles.
    • Select the Mimir (Or Prometheus) datasource and query metrics.

Just a note

  • When Alloy starts, you'll notice that install-dashboard-dependencies and provision-dashboards are stopped. This is expected.
  • Also, If you don't have an application exporting prometheus, metrics you won't see any data in the relevant dashboards. alt text

That's it!

You now have a fully functional Grafana Alloy Observability Stack running on your local machine. Feel free to explore the various services and dashboards to monitor your applications and infrastructure.

If you have any questions or feedback, please let me know!