Skip to content

rubberduck203/ex_prometheus_logger

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ExPrometheusLogger

ExPrometheusLogger is a custom Elixir logger backend. It creates counters and increments them on logging events, providing easy insight into the number of warning & errors occuring in your applications.

ex_logger: Rising counts of errors and warnings

rate(ex_logger[30s]): Rate of increase per second of warnings and errors

Installation

mix.exs

def deps do
  [
    {:prometheus_logger, git: "https://github.com/rubberduck203/ex_prometheus_logger.git", tag: "0.1.1"},
  ]

Configuration

Just add the logger backend to your config

config.exs

config :logger,
  backends: [:console, Logger.Backends.Prometheus]

and ensure that :prometheus is started before :logger.

mix.exs

def application do
  [
    # :prometheus *must* be started before :logger
    extra_applications: [:prometheus, :logger]
  ]
end

Currently, the only supported configuration is the log level. It defaults to :warn. Setting or changing the base logger level has no effect on ExPrometheusLogger's level.

config :logger, Logger.Backends.Prometheus,
  level: :warn

Exposing Metrics

ExPrometheusLogger doesn't actually publish the metrics. It just creates and increments prometheus_ex counters.

It's the user's responsibility to expose those metrics, either by exposing a /metrics endpoint or by using the pushgateway.

See the example directory for examples of how to use pushgateway and Cowboy & Plug to publish the metrics.

Alerting

The logger backend uses a counter to collect the number of times different level logging events have occured. Because this is a counter, to set up an alert, it us recommended to use the rate() function to calculate the rate at which errors/warnings are increasing over a period of time.

rate(
  ex_logger{level="error"}[1m]
)

See example/rules.yml for sample alerting rules.

There are times you may want to alert on a hard number of log messages. That should be a relatively rare, but straight forward, use case.

Roadmap

  • Provide examples for using
    • Pushgateway
    • Exposing a /metrics endpoint
  • Use pushgateway automatically if :prometheus, :pushgateway config is present in app env.
  • Publish on Hex
  • Publish docs
  • Travis CI build/release
  • Custom labels?
  • Take :logger, :level into consideration
  • Leverage logger meta-data?