@RobertDober thanks for all your work on Earmark, its a critical part of the ecosystem and you’ve made it better through your efforts.

I started experimenting with the AST. Pretty straight forward for what I want to do:

581568×1442 98.5 KB

defmodule GitGud.Web.Markdown do
  @moduledoc """
  Conveniences for rendering Markdown.
  """

  @doc """
  Renders a Markdown formatted `content` to HTML.
  """
  @spec markdown(binary | nil) :: binary | nil
  def markdown(nil), do: nil
  def markdown(content) do
    case Earmark.as_ast(content) do
      {:ok, ast, _warnings} ->
        ast
        |> transform_ast()
        |> Floki.raw_html()
    end
  end

  #
  # Helpers
  #

  defp transform_ast(ast) do
    ast
    |> Enum.map(&transform_ast_node/1)
    |> List.flatten()
  end

  defp transform_ast_node({tag, _attrs, _ast} = node) when tag in ["code"], do: node
  defp transform_ast_node({tag, attrs, ast}) do
    {tag, attrs, transform_ast(ast)}
  end

  defp transform_ast_node(content) when is_binary(content) do
    content = Regex.replace(~r/:([a-z0-1\+]+):/, content, &emojify_short_name/2)
    auto_link(content, Regex.scan(~r/#[0-9]+|@[a-zA-Z0-9_-]+|[a-f0-9]{7}/, content, return: :index))
  end

  defp emojify_short_name(match, short_name) do
    if emoji = Exmoji.from_short_name(short_name),
     do: Exmoji.EmojiChar.render(emoji),
   else: match
  end

  defp auto_link(content, []), do: content
  defp auto_link(content, indexes) do
    {content, rest, _offset} =
      Enum.reduce(List.flatten(indexes), {[], content, 0}, fn {idx, len}, {acc, rest, offset} ->
        {head, rest} = String.split_at(rest, idx - offset)
        {link, rest} =
          case String.split_at(rest, len) do
            {"#" <> number, rest} ->
              {{"a", [], ["##{number}"]}, rest} # TODO
            {"@" <> login, rest} ->
              {{"a", [{"class", "has-text-black"}], ["@#{login}"]}, rest} # TODO
            {hash, rest} ->
              {{"a", [], [{"code", [{"class", "has-text-link"}], [hash]}]}, rest} # TODO
          end
        {acc ++ [head, link], rest, idx+len}
      end)
    List.flatten(content, [rest])
  end
end

I just released Earmark 1.4.6

• Exposes the now stable AST in the quadruple format.

• There are many important bug fixes, all known crashes have been fixed, some issues like code blocks in lists and better HTML support did not make it.

Thanks to all of you for your great bug reporting after 1.4.5 which was a rough one.

Here go the Release Notes:

• 350-some-complicated-autolinks-cut

• 359-unexpected-ws-in-html

• 337-quadruple-ast-format

• 366-simplify-transform
  Kudos to Eksperimental

• 353-oneline-html-tags

• 351-html-tags-without-newlines

• 335-content-inside-table-cells-reversed

• 348-no-crashes-for-invalid-URIs
  Kudos to José Valim

• 347-dialyxir-errors
  Fixed some of them, alas not all

I have just released, what I consider my last release of

Earmark 1.4.41 2023-09-19 Palindrome Edition

Isolate from EarmarkParser in order to prevent conflicts with indirect dependencies
on EarmarkParser via ExDoc.
This shall mark the end of my role as maintainer of Earmark.

Rationale

Removing the dependency on EarmarkParser allows to

• let the maintainer of Earmark decide how to parse markdown
• avoid indirect version conflicts or problems for libs using Earmark → EarmarkParser and ex_doc → EarmarkParser as has occurred here

I have just released EarmarkParser v1.4.8

I always wanted to release a new library starting with that high a version

actually I will very soon use EarmarkParser in Earmark

and EarmarkParser is just Earmark without HTML Transfomation and CLI, thusly implementing only what is needed for ex_doc.

I had the following motivations for this step

• Why installing unused code in all libraries that use ex_doc?

• Allow Earmark to work on configurable transformers (maybe also other formats)

• Allow Earmark to use libraries which use ex_doc as soon as ex_doc agrees to use EarmarkParser

Here is the Release Note

EarmarkParser

1.4.8 2020/06/29

This marks the first release of the parser isolated from the rest of Earmark.

It is feature identical to the 1.4.7 release of Earmark.

All releases below were Earmark, all releases above are only EarmarkParser.

Earmark

1.4.7 2020/06/29

• 371-still-spurious-ws-after-inline-tags