josevalim

josevalim

Creator of Elixir

Proposal: Introduce help catalogs

Note: this is a language proposal so please keep the discussion on topic. If you want to talk about related behaviour but not strictly part of the proposal, please start a new conversation.

Elixir focuses on good warning and error messages whenever possible. After all, an unclear warning/error should be a bug.

In some cases, however, to keep those messages as clear as possible, they end-up spanning multiple lines:

iex(1)> defmodule Foo do
...(1)> def bar(baz) do
...(1)> if true do
...(1)> baz = :other
...(1)> end
...(1)> end
...(1)> end
warning: variable "baz" is unused

Note variables defined inside case, cond, fn, if and similar do not leak. If you want to conditionally override an existing variable "baz", you will have to explicitly return the variable. For example:

    if some_condition? do
      atom = :one
    else
      atom = :two
    end

should be written as

    atom =
      if some_condition? do
        :one
      else
        :two
      end

Unused variable "baz" found at:
  iex:4

here is another example:

iex(2)> defmodule Bar do
...(2)> def foo(:a, b \\ :omg), do: :a
...(2)> def foo(:b, b), do: b
...(2)> end
warning: def foo/2 has multiple clauses and also declares default values. In such cases, the default values should be defined in a header. Instead of:

    def foo(:first_clause, b \\ :default) do ... end
    def foo(:second_clause, b) do ... end

one should write:

    def foo(a, b \\ :default)
    def foo(:first_clause, b) do ... end
    def foo(:second_clause, b) do ... end

  iex:4

The downside of those messages are that, for experienced developers, they end-up being too much noise. There is also a “scare” factor when you download a dependency and it ends-up printing long multiple lines of warnings.

This is aggravated by the fact that Elixir does not allow warnings to be disabled. We promise to keep all warnings relevant and worth of your time, but, as a trade-off, we don’t allow you to disable them.

Note: this is NOT a discussion about disabling or removing some warnings. If there are warnings you don’t agree with, please open up a separate discussion.

In order to keep our promise of relevant warnings for new and experienced developers alike, we would like to introduce help catalogs. This proposal is broken in 3 parts. First we introduce the idea of a help catalog. Then we propose a particular implementation. Finally we discuss improvements in related areas.

Help catalogs

The idea behind help catalogs is very simple. Instead of long warnings, we will provide users with a mechanism to get more information about that warning. For example, the unused variable warning above could be written as:

warning: nested variable "baz" is unused (elixir --explain nested_var)

Once the user invokes the proposed command, they will get a detailed explanation about the warning and how to fix it:

$ elixir --explain nested_var

This warning yada yada yada yada yada yada
yada yada yada.

For the clauses one, we could say:

warning: def foo/2 has multiple clauses and also declares default values. Please define default values in a header (elixir --explain defaults_and_clauses)

Do not worry about the styling of the warnings and of the command for now. We will address them later.

I believe this will be an improvement for long warnings but there is another reason why I believe this feature can be extremely useful. Let’s get a very simple warning, the unused variable warning, as seen below:

iex(3)> defmodule Baz do
...(3)> def bar(used, unused), do: used
...(3)> end
warning: variable "unused" is unused
  iex:5

How to address this warning? We change unused to _unused. But how would somebody in their first week with Elixir know this is the case? We could make the warning longer but everyone would agree it is counter-productive. By having a catalog, we can include detailed information even on simple warnings like above. Similar feature exists in languages like Rust and PureScript.

Question 1: what do think about the idea of supporting help catalog in general? (regardless of the command structure, syntax, etc)

Implementing catalogs

If agreed that help catalogs will be a good addition to the language, then it is time to talk about its implementation.

In the examples above, we have used the following syntax to invoke them: elixir --explain nested_var. Such syntax has two issues:

  1. It seems specific to Elixir. However, as an extensible language, it would be great if help catalogs were available to all libraries

  2. If we also want to introduce the explain functionality to IEx, a natural confusion between explain and help would arise. After all, what is the difference between help and explain? When to use one or the other?

Therefore, I propose the following syntax for help catalogs:

$ elixir -h elixir:nested_var
$ elixir --help elixir:nested_var

Then in IEx, you can read a catalog as:

iex> h "elixir:nested_var"

In other words, we are simply extending the help mechanism to support catalogs. The catalog is given in two parts, the application name (elixir) and the entry name (nested_var), separated by :.

Implementation wise, it will work like this:

  • We will receive the entry name as “app:entry” and split it by “:”
  • We get the application name and look for its .app file
  • Inside the app file, we will look for an entry named help_catalog that points to a module
  • The help_catalog will then be loaded and it must export a function of zero arity with the same name as the entry. The function should return a string in markdown that will then be formatted and printed.

Question 2: what do you think about the suggested syntax for catalogs and its implementation?

Closing the gap

Now that we have introduced elixir -h "app:entry", does it make sense to close the gap between the command line and IEx? In other words, should we be allowed to run elixir -h String and show the documentation for the String module?

One reason to say yes is completeness. However, I personally open IEx multiple times only to retrieve the documentation or to open a module, so I would definitely use this feature too.

In particular, I propose to support all of --help, --type-help, --behaviour-help and --open in the elixir command line. I think being able to do elixir --open String and have the module open in my editor would be fantastic.

Implementation-wise, this is very straight-forward, as all of those features already exist in IEx, and we are simply discussing an option to make it available in more places.

Note that we will also have to support those entries in mix run, as we also need them available in the context of a project.

Question 3: should we close the gap and allow help and open generally available in the elixir and mix run commands?

Feedback

Feedback on the proposal is welcome. If you agree or disagree with the proposal, make sure to detail why, and remember to provide insight on the questions above. Thanks!

Most Liked

josevalim

josevalim

Creator of Elixir

Thanks everyone for the feedback.

I will archive this proposal for now. It seems that the majority of developers prefer the long warnings and, if short warnings will be opt-in, it will likely be used only by a few and therefore I do not see the benefits of adding this complexity to the tool chain.

grych

grych

Creator of Drab

Ad. 1. I like the idea in general, but I also liked the long messages in the beginning with Elixir. It is extremely useful for beginners, and this (and great docs) is why Elixir is easier to learn than any other language.

On an very early stage of learning Elixir, if I try to reassign the value of variable, I will got the warning that the variable is unused:

iex(1)> a = 1
1
iex(2)> if true, do: a = 2
warning: variable "a" is unused (elixir --explain nested_var)

Imagine you’ve never seen Elixir or Erlang before. Isn’t it confusing? And it is perfectly explained with the long help. Yes, you can alway have a long explanation, but it is not available at the first sight. And not everyone would understand that it is possible to get the longer help message.

What if we do short messages optional? Like, by default, showing long messages (warning + help from the catalogue) and allow to turn on short messages by a switch --short-messages or an entry in .elixir?

Ad. 3. YES!

josevalim

josevalim

Creator of Elixir

There have been some concerns regarding the loss of precision in warnings, which is an excellent concern. I will address them altogether instead of individually.

We are not proposing to remove all long warnings and all long errors. Rather, this is a mechanism that would allow them to do so, if desired and if relevant.

@Eiji mentioned an excellent example: function clause errors. Those errors are long, and they will continue to be long, because everything they show is contextual information. @blatyo brought another good example. Both of those cases should continue as is. The help catalog is useful to store complementary and non-contextual information about warnings and errors. All contextual information should remain as part of the warning/error.

===

The other concern that seems to be common is about emitting detailed (long) warnings by default. This seems like a good idea based on the warnings we have today but I would like to remind everyone that, once the catalog exists, it is very likely that many warnings will provide a detailed variant.

For example, “unused variable x” may now have one or two paragraphs about prepending an underscore to the variable name. Similarly, when you use a non-guard function in a guard, I would like the detailed information to show all available guards. Therefore, if we are detailed by default, we may end-up showing a lot more information than we do today. For those reasons, I think detailed by default is practical today, but not when the catalog will be place.

I would like to flip the question. Instead of discussing if we should have “shorter vs long” by default, what can we do to make sure that a newcomer will understand what elixir -h elixir:foobar in a warning/error means and make sure that they will be able to access the detailed information?

josevalim

josevalim

Creator of Elixir

I think we should be able to integrate the catalog with ExDoc, which means both Elixir and libraries should have online versions of those. But I’d personally prefer a flow that works on the command line (or from my editor) and does not require a browser. Thoughts?

voughtdq

voughtdq

I’d like a help catalog, but I do not think it would be helpful to new developers. I gained an understanding of Elixir partly because the warnings were clear and obvious and sometimes verbose. It can be a frustrating experience to get a bunch of brief errors and then have to refer to another source of information. It can also be exhausting and make a developer want to give up. This is part of the friction I got when starting with Erlang – the errors were often too brief and hard to interpret.

Instead, I think

Long warnings should be opt out, not opt elsewhere

The long warning should be opt out. Seasoned developers can put the relevant config options in mix.exs. Or something like elixirc --brief-warnings. I do like the idea of this proposal, but I also know that there is a scare factor for new developers seeing a warning like:

warning: def foo/2 has multiple clauses and also declares default values (elixir --explain default_values)

Then running that command and seeing a whole page of details about function heads, but not seeing it in context. This can be really exhausting to have to switch between the compiler output and then referring to another source of information.

In general, yes to help catalogs

Yes to help catalogs, but with the caveat that the long warnings are there by default and can be shortened with an option like --brief-warnings.

The syntax and formats proposed are reasonable to me.

Another problem is having to write two different versions of the warning, but that would happen with my suggestion and with the catalog.

Where Next?

Popular in News Top

josevalim
Announcement: https://elixir-lang.org/blog/2019/01/14/elixir-v1-8-0-released/ Release notes: https://github.com/elixir-lang/elixir/relea...
New
josevalim
Hello everyone, Two vulnerabilities have been disclosed to Plug. Applications that provide file uploading functionality to a local file...
New
Elixir
Official announcement: Elixir v1.15 released - The Elixir programming language This release requires Erlang/OTP 24 and later. Elixir v1...
New
josevalim
Hi everyone, The next (and hopefully last) release candidate for Elixir v1.14 is out: Release v1.14.0-rc.1 · elixir-lang/elixir · GitHub...
New
Elixir
1. Enhancements Elixir [JSON] Encode any JSON key to string [Kernel] Allow <<_::3*8>> in typespecs Mix [mix loadpaths] Sup...
New
Elixir
1. Enhancements Elixir [Code] Emit :defmodule tracing event on module definition Mix [Mix] Add Mix.install_project_dir/0 [Mix] Add env...
New
Elixir
Release: Release v1.12.3 · elixir-lang/elixir · GitHub 1. Bug fixes Elixir [Code] Make sure that bindings in the default context return...
New
Elixir
1. Enhancements IEx [IEx.Autocomplete] Speed up loading of struct suggestions 2. Bug fixes Elixir [Code.Fragment] Fix Code.Fragment.su...
New
josevalim
Elixir v1.5.0-rc.1 has been released. This is the second release candidate for the upcoming Elixir v1.5. It includes bug fixes, enhance...
New
josevalim
moderators note: A conclusion by @josevalim has been drawn in Proposal: Private modules (general discussion) - #143 by josevalim While...
320 12877 159
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

We're in Beta

About us Mission Statement