For anyone following this, it turns out I’m a total idiot… Elixir is handling tens of thousands of elements just fine, as I have just discovered with better benchmarking. Even if each element contains a number of totally not optimised polynomials implemented as structs.

It turns out that unlike what I previously reported, the problem is with typst. Problems start to happen when I try to compile typst files of > 20.000 lines with a large number of graphical objects. To be fair to typst, these files are probably much larger than anything anyone has tried to use typst on (I don’t know whether the critical part is the patching or the rendering, but I’m not that interested in finding out).

Because of this, I have decided to keep the core constraint-based architecture the same and drop typst. I’ll be try to use resvg (the rust svg rendering library, which also has Elixir bindings) for text rendering and measuring and to render SVGs into PNGs. I assume that resvg is probably better optimised than typst for rendering larger numbers of shapes.

The only thing I lose is math typesetting, but it turns out plots don’t use that much math typesetting. One can get by with unicode symbols, superscripts, subscripts and not much else.

Quartz (this library is no longer called playfair) can now draw line plots (by very inefficiently drawing line segments individually instead of using a normal path because I haven’t iomplemented paths yet).

Example here:

image635×497 22.7 KB

Code:

defmodule Quartz.Benchmarks.LinePlot do
  use Dantzig.Polynomial.Operators
  require Quartz.Figure, as: Figure
  alias Quartz.Plot2D
  alias Quartz.Length

  def build_plot() do
    figure =
      Figure.new([width: Length.cm(8), height: Length.cm(6), debug: false], fn _fig ->
        [[bounds]] =
          Figure.bounds_for_plots_in_grid(
            nr_of_rows: 1,
            nr_of_columns: 1,
            padding: Length.pt(16)
          )

      x = for i <- 1..100, do: 0.01 * i
      y = for x_i <- x, do: x_i * 0.3 + (0.05 * :rand.uniform())

      data = %{x: x, y: y}

      _plot =
        Plot2D.new(id: "plot_A")
        |> Plot2D.set_bounds(bounds)
        |> Plot2D.line_plot("x", "y", data)
        # Use typst to explicitly style the title and labels ――――――――――――――――――――――――――――――――
        |> Plot2D.put_title("A. Line plot")
        |> Plot2D.put_axis_label("y", "Prediction: $f(x)$", text: [escape: false])
        |> Plot2D.put_axis_label("x", "Predictor: $x$", text: [escape: false])
        |> Plot2D.finalize()
      end)

    path = Path.join([__DIR__, "line_plot", "example.pdf"])
    Figure.render_to_pdf_file!(figure, path)
  end
end

Some more examples, this time for distribution plots.

KDE smoothing of the posterior distributions for a NUTS bayesian simulation, showing all 4 chains in the same plot:

image945×472 21.9 KB

The same data plotted using boxplots (boxplots are not very configurable yet, but that is just a question of adding more options):

image945×591 9.73 KB

Finally, the contour plot I’ve shown before, but with a much higher resolution now that we don’t depend on typst and can use the more performant (for this use case) resvg renderer (note that we can fake powers of integers with superscript unicode characters, although for more advanced things one really needs proper math typesetting):

image945×709 58.6 KB

And no, Quartz still can’t rotate text, which messes up the case where the labels of the y-axis are longer, but it works fine for these short labels.

Big landmark! Quartz can now accurately rotate text, which means it can render actual real-world labels for the Y-axis of plots:

Some questions for whoever may be following this:

1. What do you think of the choice of using the Sans-Serif Ubuntu font as a default? The license is quite permissive, and it’s certainly prettier than other options such as DejaVu, used by Matplotlib. However, I’ve been thinking of making the default a Serif font, such as Linux Libertine (which I actually use to typeset math characters)

2. Now that I can support rotation of text and legends, I think that it might be the right time to think about releasing a 0.1 version. What do you think would be the “essential” plot types for an actual release on Hex? I already support the histogram, the KDE for continuous distributions, line plots, scatterplots and “fill-between” plots. For things like survival analysis, plotting a step function is useful (survival functions are usually represented as Kaplan-Meier plots, which are pretty much always displayed as step functions)

3. Anyone here is interested in diving into the constraint-solving part of things? Currently it works very well, and allow for very useful dynamic layouts in which things “fit into place” magically. The problem is that failures aren’t really debuggable. Is anyone here experienced with Linear programming solvers or other kinds of constraint solvers? When the linear solver fails, the mains problem is that I just raise an error and can’t give the user any feedback on which constraints are causing problems or why. Is there any literature on this? Should I just randomly remove constraints from the problem (with something like a binary search) and try to detect which of them cause problems? The way things are implemented now, there won’t be any constraint-solving problems unless the user creates “raw” constraints using the Figure.assert/1 macro, but I wonder if one can enhance debuggability

4. For actual real-life plots, how important would it be to support proper mathematical formulas beyond math symbols, subscripts and superscripts? I figure I can probably integrate with Typst to get proper math typesetting, but that would split text handling into two. Or maybe I could integrate with Typst for all text and leave the shape renfering to Quartz

Version 0.8.1

With this new version, I was finally able to update the docs to hexdocs (API Reference — Quartz v0.8.1) by moving all images to a new website: https://tmbb.github.io/quartz/. This is currently hosted on GitHub pages, but when I have the time, I’ll host it in its own domain. These changes allow me to remain within the size limits of hexdocs.

Demo for some of the supported plots here: https://tmbb.github.io/quartz/plot_types.html

Plotting API

The plotting API is inspired by Matplotlib. Despite being quite old, Matplotlib is very functional for my needs and makes it very easy to customize the plots, which is important for publication-quality figures.

I really don’t like frameworks which encourage any kind of “grammar of graphics”, which always seem very artificial to me and obscure what would usually be simple plotting commands.

Contributing

I will have a bit more free time in April, and I would like to refactor the code to stabilize the interface and make it easier for external contributors. I think this package could be a good alternative to generate plots for scientific paper or other static media (like Matplotlib). Together with my Ulam package, one can already create, fit and plot Bayesian models, and honestly I prefer to manage Elixir projects as opposed to Python projects (packaging is python projects is rather problematic).

I don’t know if people are still trying to develop an ecosystem for scientific programming in Elixir, but if so, Quartz would be an interesting part of such ecosystem.