Frogglet

Frogglet

Ecto not allowing string interpolation in fragments?

I am trying to use where exists (), which requires a fragment in ecto. I need to use a different table name for the subquery in different situations, so I thought a variable that gets interpolated would be ideal. No, the value does not come from an unsafe source. I am aware of the dangers of SQL injection attacks and how to avoid them. I am selecting the value of the variable from another ecto schema inside a case expression. Surely there must be some way of doing this, without having to entirely disconnect from the Ecto abstractions we already depend on?

Here is an example:

  def thing(schema) do
    other_table = Other.Schema.__schema__(:source)
    from(a in My.Schema,
      where: fragment("exists (
        SELECT 1
        FROM #{other_table} o
        WHERE o.column_name = ?)", ^a.my_field)
    )
    |> Repo.all()
  end

And here is the error:

(Ecto.Query.CompileError) to prevent SQL injection attacks, fragment(...) does not allow strings to be interpolated as the first argument via the `^` operator, got: `"exists (\n        SELECT 1\n        FROM #{other_table} o\n        WHERE o.column_name = ?)"

We are trying to use exists because it is significantly more performant than the alternatives that ecto provides for our use-case. We are reaching the scale where inefficient queries that can’t use indexes properly are becoming critical problems.

Marked As Solved

NobbZ

NobbZ

Which can luckily be macro’ed away… Roughly:

[
  {"omg", "o"}
]
|> Enum.each(fn {tbl, t} ->
  defp add_exists(qry, unquote(tbl)) do
    from q in qry,
      where: fragment(unquote("exists (
        SELECT 1
        FROM #{tlb} #{t}
        WHERE #{t}.column_name = ?)"), ^q.my_field)
      )
  end
end

This is a very rough draft though… And I’m not exactly sure about unquoteing the string passed to fragment. But this way it should be evaluated at compiletime and “look like” a static string to fragment.

Also Liked

josevalim

josevalim

Creator of Elixir

Yeah, unfortunately then I think you will have to hard code the table name. If you have multiple table names, you will need a case and a clause for each:

  def thing(schema) do
    other_table = Other.Schema.__schema__(:source)
    from(a in My.Schema) |> add_exists(other_table) |> Repo.all()
  end

  defp add_exists(query, "omg")
    from q in query,
      where: fragment("exists (
        SELECT 1
        FROM omg o
        WHERE o.column_name = ?)", ^q.my_field)
      )
  end

Sorry. :confused:

josevalim

josevalim

Creator of Elixir

Hi @Frogglet! We don’t allow interpolation because it can be unsafe and lead to SQL injection attacks.

Have you tried passing the table name as fragment interpolation: fragment(“... FROM ? ....”, ^table_name)?

tfwright

tfwright

I had a similar issue trying to dynamically build fragment strings (from a trusted source) and it seems like a general escape hatch for these kind of issues, when trying to use third party macros with dynamic inputs where not explicitly supported is to use Code.eval_quoted. So, for example, you can wrap an entire select statement and fool Elixir/Ecto into thinking the variables are in fact literals:

Code.eval_quoted(
      quote do
        select(unquote(escaped_q), [table], %{
          dynamic_select: fragment(unquote(some_var), table.field))
        })
      end
    )

I am pretty positive this is the last tool you should reach for and maybe it would be better to use postgrex directly for these cases. I haven’t tried that so I can’t speak to the pros/cons but I’d love to hear people’s thoughts about the practice, and maybe whether Ecto could adopt some “blessed” way of circumventing protections like this one because I certainly feel dirty doing this.

hauleth

hauleth

Are you by chance using PostgreSQL? If so then SELECT 1 FROM table WHERE pattern LIMIT 1 will be equally performant. Unfortunately there is no way to do string interpolation in fragment as IIRC it is compile-time checked for correctness. There was fragment_unsafe for brief moment but IIRC it was removed.

max-konin

max-konin

@taylordowns2000
working example:

def search_text(query, %{"search_string" => search_string}) do
  from(r in query,
    where:
      fragment(
        "jsonb_path_exists(?, '$.** \\? (@ == $search_string)', jsonb_build_object('search_string', ?::text))",
        r.body,
        ^search_string
      )
  )
end

Where Next?

Popular in Questions Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

We're in Beta

About us Mission Statement