Fl4m3Ph03n1x

Fl4m3Ph03n1x

Does Logger have a hidden cap for message size?

Background

In one of our projects a client of ours complained that the logs he is getting are being capped at 2000 characters.

This client is gettign his logs via a tool called Splunk and perhaps some other systems I am not aware of.

Instead of capping the messages at 2K characters, I need to cap them at 8K.

Config

To me this is strange, because we specifically truncate the log to :infinity, as our config shows:

use Mix.Config

config :logger,
  level: :info,
  backends: [:console],
  utc_log: true,
  sync_threshold: 100,
  truncate: :infinity

if Mix.env() != :prod do
  config :logger,
    level: :debug
end

config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [
    :module,
    :line,
    :function,
    :trace,
    :perf,
    :duration,
    :namespace
  ]

Furthermore, I didn’t find any specific Logger limits documented:

https://hexdocs.pm/logger/Logger.html

Question

  • Does the Logger have some internal limit that cuts messages down to 2000 characters? If so, how can I change it?

Marked As Solved

kip

kip

ex_cldr Core Team

At least on the console backend I’m not seeing such a limit. You can test it easily with:

iex> require Logger
iex> Logger.debug String.duplicate("A", 8000)
:ok

23:04:58.307 [debug] AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.......

Also Liked

KristerV

KristerV

just so my headache is logged somewhere.

there’s two different layers you can config in the logger. one for backends, one for logger itself. so in my case this helper a lot:

config :logger, truncate: :infinity
config :logger, :console, truncate: :infinity

thanks goes to @JonRowe and @LostKobrakai in Slack.

c4710n

c4710n

Updates that fit the current situation:

disable log truncation

As of Elixir 1.14:

config :logger, truncate: :infinity is enough for disabling log truncation.

config :logger, :console, truncate: doesn’t exist anymore.

log data

Another point to note if you are planing to print data with inspect/2:

Logger.info( inspect(data) )

Make sure that inspect will print all the information you want. The following line is a good start:

Logger.info( inspect(data, structs: false, limit: :infinity, printable_limit: :infinity) )

performance consideration

If the data is large, carefully consider before using the above two steps.

kip

kip

ex_cldr Core Team

Just for giggles I put it in a test case:

defmodule ThingTest do
  use ExUnit.Case
  import ExUnit.CaptureLog
  require Logger

  @message_size 8_000

  test "Logger backend end" do
    assert capture_log(fn -> Logger.error(String.duplicate("A", @message_size)) end) >= @message_size
  end
end

And ran the test:

kip@Kips-iMac-Pro thing % mix test
.

Finished in 0.04 seconds
1 test, 0 failures
rlopzc

rlopzc

To enable infinite truncation in the new Logger (since Elixir 1.15) do:

config :logger, :default_formatter,
  ...,
  truncate: :infinity
NobbZ

NobbZ

Is it perhaps the :truncate option? By default it’s 8kiB.

:truncate - the maximum message size to be logged (in bytes). Defaults to 8192 bytes. Note this configuration is approximate. Truncated messages will have " (truncated)" at the end. The atom :infinity can be passed to disable this behavior.

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
wernerlaude
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
New

Other popular topics Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement