Monitoring API Endpoints Speed on Laravel/Lumen using Loki and Grafana [2/3]

Now that our API is all set up, recording the activities and time executions in the logs, we need to scrap these data so it can be queried by Grafana. To do this, we are going to use Grafana Loki and Promtail. Grafana Loki is a log aggregation platform based on Prometheus. Its implementation with Grafana is seamless.

#1 – Promtail Configuration

Before installation, you will need to prepare the configuration.

You can download Promtail configuration file first:

CLI

wget https://raw.githubusercontent.com/grafana/loki/v2.9.1/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml

URL

https://raw.githubusercontent.com/grafana/loki/v2.9.1/clients/cmd/promtail/promtail-docker-config.yaml

And now you can customize your Promtail configuration promtail-config.yaml:

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log

- job_name: my-api-project
  static_configs:
  - targets:
      - localhost
    labels:
      job: registration-api-id
      __path__: /var/www/my-api-project/storage/logs/*
  pipeline_stages:
    - json:
        expressions:
          metrics: message
          status: level
          channel: channel
          timestamp: datetime
          path: context.path
          value: context.value
          category: context.category
          start_at: context.start_at
          end_at: context.end_at
    - metrics:
        execution_time:
          type: Histogram
          description: "Execution time duration"
          source: value
          config:
            buckets: [0.001,0.0025,0.005,0.010,0.025,0.050]

#2 – Loki Configuration

You can download Promtail configuration file first:

CLI

wget https://raw.githubusercontent.com/grafana/loki/v2.9.1/cmd/loki/loki-local-config.yaml -O loki-config.yaml

URL

https://raw.githubusercontent.com/grafana/loki/v2.9.1/cmd/loki/loki-local-config.yaml

And now you can keep the original loki configuration file the way it is.

#3 – Install and run Grafana Loki and Promtail

There are multiple ways to install Grafana Loki, chose the one you prefer:

• Install using Docker
• Install locally
• Install using Helm
• Install using Tanka
• Install on Istio
• Install from source

When installing Promtail, make sure to target your logs directory.

For example if you install using docker compose:

...
  promtail:
    ...
    volumes:
      - /var/www/my-api-project/storage/logs:/var/log
...

Continue to next part

Now the logs are being scrapped, let’s prepare our dashboard in Grafana and connect it to Loki.