This is my mental model of current state of the loggers:

1. Logger handlers is the main interface for logging
2. Logger handlers are recommended to support something called “overload protection”
3. The default :logger_std_h handler does have an overload protection
4. If you want to write a custom handler, you’ll have to implement overload protection, like for example, Sentry handler does.
5. LoggerBackends is a logger handler which allows you to write logger backends which is a concept unique to this package. Custom backends won’t need overload protection as they are not logger handlers.

So my rule of thumb is as follow: logger handlers that implement overload protection can be all used as handlers alongside each other:

config :my_app, :logger, [
  {:handler, :file_log, :logger_std_h, ...},
  {:handler, :sentry, Sentry.LoggerHandler, ...}
]

Whenever you want a custom logger but don’t want to implement overload protection, you can take advantage of LoggerBackends and write a custom logger backend instead.

In reality, I’ve yet to find myself writing a custom handler as the built-in :logger_std_h in combination with filters and formatters is very very flexible. So flexible, in fact, that LoggerJSON project was completely rewritten from being a logger backend to being a formatter.

Aaaaah, that makes sense! I could have implemented my example (file logger) using the logger_std_h handler indeed. So, the idea here is to rather configure the existing logger_std_h or logger_disk_log_h implementations unless you need something special, but which doesn’t need overload protection. In that case, you can use the LoggerBackends library. Thanks for the clarification!

Just for future reference, I managed to implement a logger_handler myself, but it’s terrible and shouldn’t be used except for inspiration for how to use e.g. :logger_formatter, how to convert the old format-option to the new template-option, or how to convert the gregorian microsecond timestamp of the new log_event back to date and time.

It’s on GitHub here: run-elixir/assets/archived_code/file_logger.ex at main · PJUllrich/run-elixir · GitHub

And I also implemented an “old” handler, which you should now add using the LoggerBackends.add/2 function instead of the deprecated Logger.add_backend/2 function. It’s on GitHub here: run-elixir/assets/archived_code/old_file_logger.ex at main · PJUllrich/run-elixir · GitHub

Interested in what you put together if you can share? I’m currently working on Logflare and we actually need beta testers for our new logger backend.

I really like being able to keep the raw event data as you can drill into it vs the time series approach. As long as your db is fast enough to query, you’re able to insert in real-time and you’re happy with the visualization / BI tools available … I think the time series approach is less favorable.

With our Logger backend you can structure logs and add any data you want. Logflare uses BigQuery as the db and manages the BigQuery schema so you can easily query each field specifically. And BigQuery plus Data Studio is very nice although I wish it would give me minutely graphs out of the box.

I do not quite get what is the problem here. Erlang’s logger allows to use structured logging, so instead of (in Elixir syntax) doing Logger.info("Responded HTTP 200 in #{time}") you can do: Logger.info(%{http_response: 200, http_response_time: time}) and at the same time provide more information in metadata. This allows to eat cookie and have cookie because:

• you can programmatically get meaningful data without processing text
• you still can insert metadata (that aren’t part of the log message) into metadata

Grafana describes difference quite nicely, so you can see that in most (all?) cases metrics can be extracted from logs. Of course some of these logs will need to be discarded to not needlessly increase logs volume, but that can be done quite easily later.

I agree. I am part of the EEF Observability WG and my question isn’t whether we should rely only on the logger but whether we should include metrics in logs and then provide an logger handler that will extract these from logs and aggregate them. So everything would happen without sending high volume messages to external log handler.

So in the end there still would be some kind of aggregate that would reduce volume, so even when there happen error that is gigabyte long it shouldn’t be a problem, as most of the data would be discarded by handler anyway.

So my question is mostly about libraries like Ecto which use telemetry right now. Would it be better to keep telemetry or maybe use structured logs that could provide the same informations.