mathieuprog

mathieuprog

QueryBuilder - Compose Ecto queries without effort

I’ll start right with an example :point_down:

User
|> QueryBuilder.where(firstname: "John", city: "Anytown")
|> QueryBuilder.where({:age, :gt, 30})
|> QueryBuilder.order_by(lastname: :asc)
|> QueryBuilder.preload([:role, authored_articles: :comments])
|> Repo.all()

With associations:

User
|> QueryBuilder.where([role: :permissions], name@permissions: "delete")
|> Repo.all()

Query Builder allows to build and compose Ecto queries based on data.
Concise, no need to deal with bindings and macros.

Its primary goal is to allow Context functions to receive a set of filters and options:

# in a Controller
Blog.list_articles(preload: [:comments], order_by: [title: :asc])
Blog.list_articles(preload: [:category, comments: :user])

This avoids having to create many different functions in the Context for every combination of filters and options, or to create one general function that does too much to satisfy all the consumers.

The calling code (e.g. the Controllers), can now retrieve the list of articles with different options. In some part of the application, the category is needed; in other parts it is not; sometimes the articles must be sorted based on their title; other times it doesn’t matter, etc.

The options may be added to the query as shown below:

# in the Blog context
def get_article_by_id(id, opts \\ []) do
  QueryBuilder.where(Article, id: id)
  |> QueryBuilder.from_list(opts)
  |> Repo.one!()
end

Inspired by the libraries token_operator and ecto_filter.

More examples are available in the doc:

Most Liked

mathieuprog

mathieuprog

It only does a small optimization for you when you want to preload data.

Imagine a User has one Role and many Articles.

If you want to preload the user with its associated role and articles, it’s better to join user and role table (as it is a one-to-one association); but it is better to execute a separate query for loading the articles (the cost of executing a separate SQL query to the DB for loading a one-to-many association is lower than Ecto’s processing if all the rows are in the result of one single query).

If you execute the following:

QueryBuilder.preload(User, [:role, :articles])
|> Repo.all()

It will generate something like:

from u0 in User,
 join: r1 in assoc(u0, :role),
 preload: [:articles],
 preload: [role: r1]

As you can see above, the library has joined user and role as it is a one-to-one association, but didn’t join articles with user (as long as there are no where clauses on articles, in which case there will be of course a join).

That’s the only optimization regarding queries it does for you. But you could write those queries directly with Ecto.Query’s API of course. The purpose of the library is really to work with data instead of macros, which allows you to

  • have a more flexible API for your Context; Context functions can receive data (querying options such as what you want to preload, if you want to sort, pagination params, etc.) which can be passed to QueryBuilder;
  • compose queries more easily without having to take into account binding positions or arbitrary named bindings.

There’s still a lot of work to be done, and the library will be updated progressively according to my needs or other library users’ need. Currently supported are basic where clauses:
QueryBuilder.where(User, age: 30) ( == )
same as: QueryBuilder.where(User, {:age , :eq, 30})
QueryBuilder.where(User, {:age, :ne, 30}) ( != )
QueryBuilder.where(User, {:age, :gt, 30}) ( > )
QueryBuilder.where(User, {:age, :ge, 30}) ( >= )
QueryBuilder.where(User, {:age, :lt, 30}) ( < )
QueryBuilder.where(User, {:age, :le, 30}) ( <= )

You can pass a list with multiple filters:
QueryBuilder.where(User, name: "Bob", age: 30)

With associations:
QueryBuilder.where(User, :role, name@role: "author")
QueryBuilder.where(User, [role: :permissions], name@permissions: "write")

Order by:
QueryBuilder.order_by(User, age: :desc)
QueryBuilder.order_by(User, :articles, title@articles: :asc)

The functions above will make the necessary joins automatically, but sometimes you need to left join:
QueryBuilder.join(User, :articles, :left)

And of course, preload:
QueryBuilder.preload(User, :articles, role: :permissions)

mathieuprog

mathieuprog

Here are some new convenient features in QueryBuilder:

Grouped OR expressions:

query
|> QueryBuilder.where([], [name: "John"], or: [name: "Alice", age: 42], or: [name: null])
|> QueryBuilder.where(:address, [address@city: "Venice"], or: [address@city: null])

maybe_where/3 for easier piping:

query
|> QueryBuilder.where(name: "Alice")
|> QueryBuilder.maybe_where(some_condition, age: 42, active: true)
wolfiton

wolfiton

Hi,

Did you checked your library with a security package for elixir for sql injections or other problems?

Also is it compatible with absinthe and absinthe_ecto?

Thanks in advance

mathieuprog

mathieuprog

The library makes use of Ecto.Query to build queries, so it comes with the same security features as Ecto. SQL injection is impossible; Ecto always uses parameterized queries which prevent SQL injection attacks.

I still have to gain more knowledge about Absinthe, which I plan to delve into in the coming months. So I can’t comment about that now sorry:) But I will look into that, thank you for the idea.

wolfiton

wolfiton

Thank you for providing examples and explanations of your libraries features.
The way you design it it looks very refreshing form the traditional Ecto queries and it looks a lot more human friendly and short.

Also i think will help other team members to understand easily the code base.

I will give it a try and come back with the results in a couple of days.

Thanks for sharing it.

Where Next?

Popular in Libraries Top

pkrawat1
Hey guyz We at @aviabird are working on a payment library in elixir/phoenix. We are targeting March 2018 to add 56 Gateways to it. Have...
New
tompave
Hello there, I would like to share a feature toggles library (AKA feature flags) I’ve been working on. The main package is FunWithFlags...
New
nikokozak
Hello all, I’ve been working on Svonix - a library for quickly integrating Svelte components into Phoenix views. It’s a much-needed succ...
New
kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
woutdp
Hi! I wanted to introduce my latest project LiveSvelte. It allows you to render Svelte inside LiveView with end-to-end reactivity. It’s ...
New
achempion
Hi, I would like to tell about my initiative to further maintain and develop Waffle project which is the fork of Arc library. The progre...
New
Qqwy
While not as prevalent as in imperative languages, arrays (collections with efficient random element access) are still very useful in Eli...
New
gjaldon
As the title states, EctoEnum has just been updated after some time of hardly any activity in the repo. Here’s the latest release: https:...
New
Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Th...
New
versilov
Could not wait for the missing Elixir ML libraries to appear, so, I wrote one myself, taking https://github.com/sdwolfz/exlearn as a foun...
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

Sub Categories:

We're in Beta

About us Mission Statement