Observe the agent

The formae agent exposes its own telemetry through OpenTelemetry: metrics, logs, and traces pushed over OTLP, plus a Prometheus-compatible /metrics endpoint for pull-based collection. This guide turns it on and shows what you get. The full list of configuration knobs lives in the configuration reference.

What gets exported

With OpenTelemetry enabled, the agent exports three signals over OTLP:

Signal	Transport	What it covers
Metrics	OTLP push	Agent stats, Go runtime, host metrics, database performance
Logs	OTLP push	Structured application logs, correlated with traces
Traces	OTLP push	API requests, resource operations, and database queries

A Prometheus-compatible endpoint is also available at http://localhost:49684/api/v1/metrics for pull-based scraping.

Enable OpenTelemetry

Add an oTel block to your configuration file:

agent {
  oTel {
    enabled = true
    serviceName = "formae-agent"
    otlp {
      enabled = true
      endpoint = "localhost:4317"
      protocol = "grpc"
      insecure = true
    }
  }
}

Restart the agent and it begins pushing telemetry to the OTLP endpoint. For the complete property table (including otlp.temporality and the prometheus settings), see the configuration reference.

Metric temporality. Keep the default delta for OpenTelemetry-native backends such as Grafana Cloud, or a collector running the deltatocumulative processor. Switch to cumulative for Prometheus or Mimir backends that do not support delta temporality.

Scrape with Prometheus instead

To scrape metrics rather than push them, keep the Prometheus endpoint on and turn OTLP push off:

agent {
  oTel {
    enabled = true
    serviceName = "formae-agent"
    otlp {
      enabled = false
    }
    prometheus {
      enabled = true
    }
  }
}

Key metrics

Alongside Go runtime, host, Ergo actor-system, and database metrics, the agent exports formae-specific stats:

Metric	Labels	Description
`formae_stacks_total`		Number of stacks
`formae_targets_total`	`plugin`	Targets by plugin
`formae_resources_managed`	`plugin`	Managed resources by plugin
`formae_resources_unmanaged`	`plugin`	Unmanaged resources by plugin
`formae_commands_total`	`command_type`	Commands by type (apply, destroy, sync, eval)
`formae_commands_by_state`	`state`	Commands by state (Pending, InProgress, Success, Failed)
`formae_resources_by_type`	`plugin`, `resource_type`	Resource count by type
`formae_resource_errors`	`plugin`, `resource_type`	Resources with errors by type

Logs and traces

Structured logs are pushed to the OTLP endpoint with trace and span IDs attached, so you can jump from a log line to the operation that produced it. Local file logging stays available whatever the OpenTelemetry setting:

agent {
  logging {
    filePath = "~/.pel/formae/log/formae.log"
    fileLogLevel = "debug"
    consoleLogLevel = "info"
  }
}

Traces cover API request handling, resource create/read/update/delete operations, database queries (with query text and latency), and plugin interactions, all correlated with the matching logs.

Grafana dashboards

Pre-built dashboards live in the formae-grafana-dashboards repository. To import one by hand, open Dashboards > Import in Grafana and upload the dashboard JSON, then select your Prometheus and Loki datasources. To provision them automatically, point a Grafana dashboard provider at the cloned repository:

apiVersion: 1
providers:
  - name: 'formae'
    orgId: 1
    folder: 'Formae'
    type: file
    options:
      path: /path/to/formae-grafana-dashboards/dashboards

Get started

Guides

Concepts

Reference

Observe the agent

What gets exported

Enable OpenTelemetry

Scrape with Prometheus instead

Key metrics

Logs and traces

Grafana dashboards

See also

​What gets exported

​Enable OpenTelemetry

​Scrape with Prometheus instead

​Key metrics

​Logs and traces

​Grafana dashboards

​See also

What gets exported

Enable OpenTelemetry

Scrape with Prometheus instead

Key metrics

Logs and traces

Grafana dashboards

See also