MDEx v0.9 is now available with some new cool features!

Delta format

Thanks to @Jskalc we can now convert Markdown to Delta:

mdex_delta.exs1200×445 27.9 KB

Streaming

Initial (experimental) support to put incomplete fragments of Markdown into the document. Useful for AI/LLM tools:

mdex_streaming.exs3024×1112 137 KB

You can see a demo (video) at https://x.com/leandrocesquini/status/1974460068054843575

Inspect Document as a tree

To help visualize the nodes in a document:

mdex_display.exs2140×1204 93.3 KB

It can be disabled if you prefer the regular struct inspect.

Unified Document API

Removed the MDEx.Pipe module in favor of unifying all operations in the MDEx.Document. Note this is a breaking change:

mdex_pipeline.exs2696×664 67.2 KB

I’ll be working on Support Phoenix tags? · Issue #571 · kivikakk/comrak · GitHub so we can (hopefully) get Elixir and Phoenix components rendering inside Markdown. Enjoy!

MDEx v0.7.0 is out with a new ~MD sigils supporting assigns and Elixir expressions:

import MDEx.Sigil

assigns = %{lang: "elixir", sample: "spawn(fn -> send(current, {self(), 1 + 2}) end)"}

~MD"""
## Lang: <%= String.capitalize(@lang) %>

```elixir
<%= @sample %>
```
"""

Outputs:

%MDEx.Document{
  nodes: [
    %MDEx.Heading{nodes: [%MDEx.Text{literal: "Lang: Elixir"}], level: 2, setext: false},
    %MDEx.CodeBlock{
      nodes: [],
      fenced: true,
      fence_char: "`",
      fence_length: 3,
      fence_offset: 0,
      info: "elixir",
      literal: "spawn(fn -> send(current, {self(), 1 + 2}) end)\n"
    }
  ]
}

Or using the HTML modifier:

"<h2>Lang: Elixir</h2>\n<pre class=\"athl\" style=\"color: #abb2bf; background-color: #282c34;\"><code class=\"language-elixir\" translate=\"no\" tabindex=\"0\"><span class=\"line\" data-line=\"1\"><span style=\"color: #61afef;\">spawn</span><span style=\"color: #848b98;\">(</span><span style=\"color: #c678dd;\">fn</span> <span style=\"color: #abb2bf;\">-&gt;</span> <span style=\"color: #61afef;\">send</span><span style=\"color: #848b98;\">(</span><span style=\"color: #abb2bf;\">current</span><span style=\"color: #848b98;\">,</span> <span style=\"color: #848b98;\">&lbrace;</span><span style=\"color: #61afef;\">self</span><span style=\"color: #848b98;\">(</span><span style=\"color: #848b98;\">)</span><span style=\"color: #848b98;\">,</span> <span style=\"color: #d19a66;\">1</span> <span style=\"color: #abb2bf;\">+</span> <span style=\"color: #d19a66;\">2</span><span style=\"color: #848b98;\">&rbrace;</span><span style=\"color: #848b98;\">)</span> <span style=\"color: #c678dd;\">end</span><span style=\"color: #848b98;\">)</span>\n</span></code></pre>"

That’s one step into the direction of supporting Markdown in LiveViews with components.

Hello! One more update. Now mdex supports code block decorators, ie: you can add metadata in the code fence info string to overwrite theme, highlight colors, etc. See examples below and docs at MDEx — MDEx v0.10.0

theme1200×1371 38 KB

lines1200×1371 41.6 KB

Very interesting package with lots of features that took my attention!

Thanks!

1. It would be nice to see a comparison table.

Done. Here’s a comparison table and a livebook to compare the output of some markdown libraries.

2. Would it be hard to add support for a custom markdown rules or even custom sets of rules

Hard to tell if it’s easier or harder, it depends on your needs, but I’d argue it tends to be easier. For instance here’s the code to render Mermaid graphs and here are some examples - you’ll notice it’s all about transforming a tree of nodes. MDEx use structs for a couple of reasons but it’s not much different than earmark_parser ast or even floki ast.

For example see how many interesting features adds ex_doc to the markdown.

Many features on ex_doc are actually implemented on the HTML data structure/AST either using external libraries to perform operations like syntax highlight or emoji (both native to MDEx) or to autolink module/function. So technically any other Markdown library could implement ExDoc.Markdown and all the rest would just work, although I’ve never tried it

3. Would it be hard to add support for Earmark (and other packages)? Imagine a case where there is a critical bug in the Rust parser - for Elixir app it means stopping entire app.

I’m not sure I follow this question. Do you mean as a backend for MDEx? If so, then no because Earmark doesn’t fully support CommonMark which was a requirement for MDEx in the first place. And comrak is pretty stable, the author is very responsive (even contributed to MDEx already) and it’s used by many libraries in the Rust ecosystem including for example to build the Deno documentation.

writing parser is not trivial

Nope it’s not. Markdown is way more complex than it looks but all the credit on the parser goes to comrak and cmark-gfm.

Maybe this is not the same kind of feedback you are interested

That was super useful, really good topics. Let me know if something is not clear.

Thanks! Yep you can create either provide your own %Autumn.Theme{} or modify existing ones, for example:

%{highlights: highlights} = theme = Autumn.Theme.get("github_dark")
red_text_style = %Autumn.Theme.Style{fg: "red"}
highlights = Map.put(highlights, "string", red_text_style)
theme = Map.put(theme, :highlights, highlights)

Then format using that theme:

MDEx.to_html!("""
```elixir
String.upcase("elixir")
```
""",
syntax_highlight: [formatter: {:html_inline, theme: theme}])

You’ll see <span style="color: red;">&quot;elixir&quot;</span> in the output.

I should probably implement the Access behaviour to let you just call put_in(theme, [...], value)

EDIT: Autumn is another library, you can find docs at Autumn — Autumn v0.3.3