Ankhers

Ankhers

Ecto and CockroachDB issue: (CaseClauseError) no case clause matching

I was wondering if anyone has been able to get ecto working properly with CockroachDB? I am in the process of setting up a project to evaluate cockroach but am having some issues. I set up a new phoenix project and ran mix phx.gen.auth Accounts User users and tried running the tests. A couple tests failed but I was able to figure out what the reason for it was.

Now that I have the tests “fixed”, I have 1-2 tests failing each time I run them. Unfortunately the error I am getting (see below) does not seem to be helpful and seems to be inside the ecto_sql code. So I’m assuming this is a difference between postgres and cockroach. But I think it would also be strange if this would be unable to work in some way.

If anyone has any ideas, I would appreciate it.

Error:

$ mix test
................................................................................13:04:25.165 [error] GenServer #PID<0.954.0> terminating
** (CaseClauseError) no case clause matching: {:transaction, %Postgrex.Protocol{buffer: "", connection_id: 0, connection_key: 0, disconnect_on_error_codes: [], null: nil, parameters: #Reference<0.4070665280.3737387013.242708>, peer: {{127, 0, 0, 1}, 26257}, postgres: :transaction, queries: #Reference<0.4070665280.3737518081.247362>, sock: {:gen_tcp, #Port<0.10>}, timeout: 15000, transactions: :naive, types: {Postgrex.DefaultTypes, #Reference<0.4070665280.3737518081.246564>}}}
    (ecto_sql 3.7.2) lib/ecto/adapters/sql/sandbox.ex:589: Ecto.Adapters.SQL.Sandbox.post_checkout/3
    (db_connection 2.4.1) lib/db_connection/ownership/proxy.ex:101: DBConnection.Ownership.Proxy.handle_info/2
    (stdlib 3.17) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:db_connection, {#PID<0.953.0>, #Reference<0.4070665280.3737387009.249387>}, {:checkout, [#PID<0.953.0>], -576460749748, true}}


  1) test apply_user_email/3 validates current password (MyApp.AccountsTest)
     test/my_app/accounts_test.exs:164
     ** (MatchError) no match of right hand side value: {:error, {{{:case_clause, {:transaction, %Postgrex.Protocol{buffer: "", connection_id: 0, connection_key: 0, disconnect_on_error_codes: [], null: nil, parameters: #Reference<0.4070665280.3737387013.242708>, peer: {{127, 0, 0, 1}, 26257}, postgres: :transaction, queries: #Reference<0.4070665280.3737518081.247362>, sock: {:gen_tcp, #Port<0.10>}, timeout: 15000, transactions: :naive, types: {Postgrex.DefaultTypes, #Reference<0.4070665280.3737518081.246564>}}}}, [{Ecto.Adapters.SQL.Sandbox, :post_checkout, 3, [file: 'lib/ecto/adapters/sql/sandbox.ex', line: 589]}, {DBConnection.Ownership.Proxy, :handle_info, 2, [file: 'lib/db_connection/ownership/proxy.ex', line: 101]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 695]}, {:gen_server, :handle_msg, 6, [file: 'gen_server.erl', line: 771]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}, {DBConnection.Holder, :checkout, [#PID<0.954.0>, [post_checkout: #Function<0.60388935/2 in Ecto.Adapters.SQL.Sandbox.checkout/2>, pre_checkin: #Function<1.60388935/3 in Ecto.Adapters.SQL.Sandbox.checkout/2>, timeout: 15000, pool: DBConnection.Ownership, pool_size: 10]]}}}
     stacktrace:
       (ecto_sql 3.7.2) lib/ecto/adapters/sql/sandbox.ex:403: Ecto.Adapters.SQL.Sandbox.start_owner!/2
       (my_app 0.1.0) test/support/data_case.ex:31: MyApp.DataCase.__ex_unit_setup_0/1
       (my_app 0.1.0) test/support/data_case.ex:1: MyApp.DataCase.__ex_unit__/2
       test/my_app/accounts_test.exs:1: MyApp.AccountsTest.__ex_unit__/2

........................

Finished in 1.8 seconds (1.1s async, 0.6s sync)
105 tests, 1 failure

I created a similar post on the cockroachdb forms here in case they are able to answer as well.

Most Liked

fireproofsocks

fireproofsocks

Ding! I got this to work by leveraging the :hints option as you suggested. I ended up defining a custom Ecto adapter which was a complete copy of the Ecto.Adapters.Postgres.Connection module except for the from/3 function. I modified from/3 to this:

    defp from(%{from: %{hints: hints, source: source}} = query, sources) when hints != [] do
      {from, name} = get_source(query, sources, 0, source)
      from = [" FROM ", from, " AS " | name]
      [from | hints]
    end

    defp from(%{from: %{source: source}} = query, sources) do
      {from, name} = get_source(query, sources, 0, source)
      [" FROM ", from, " AS " | name]
    end

I also made a full copy of Ecto.Adapters.Postgres (no changes).

In my Repo (i.e. the module that use’s Ecto.Repo), I referenced the new adapter, e.g.

  use Ecto.Repo,
    otp_app: :foo,
    adapter: Ecto.Adapters.CockroachDB

This made things work if I was importing Ecto.Query and supplied a :hints option. I took things a bit further and I overrode the prepare_query/3 function in my Ecto Repo:

  def prepare_query(_operation, %{from: %{hints: []}} = query, opts) do
    case Keyword.get(opts, :hints, []) do
      [] -> {query, opts}
      hints -> {put_in(query.from.hints, hints), opts}
    end
  end

  def prepare_query(_operation, query, opts), do: {query, opts}

This let me do more “simple” lookups such as:

Repo.all(MySchema, hints: " AS OF SYSTEM TIME '-5m'")

Note that you have to leave a space before AS.

This all seems to work and it generates queries in the proper syntax, e.g.

[debug] QUERY OK source="my_table" db=6.5ms queue=6.9ms idle=1334.0ms
SELECT c0."id", c0."foo", c0."bar" FROM "foobar" AS c0 AS OF SYSTEM TIME '-5m' []

Probably this could be cleaned up, but would this be worthwhile as a standalone package/adapter?

fceruti

fceruti

I’ve been trying to port from Postgres to Cockroach, and while the process has been mostly uneventfull, that last 10% is giving me headaches.

It’s been a long day, and I may formulate better questions in the future, but for now, have any of you guys been able to perform follower reads, within the confines of a regular ecto query building? By regular confine I mean, without Repo.execute ( << raw sql>> ), or something to that nature.

I’ve tried to use many adapter out there in the hexphere, but all seem to be out of sync. Are there any community members looking at this problem? I’m sold on cockroach and can afford to look deeper into this rabbithole, but I would like to first check if there’s some active development somewhere.

Ankhers

Ankhers

I haven’t tried it, but you might be able to do something like

from(u in fragment("? AS OF SYSTEM TIME with_max_staleness('10s')", ^User.__schema__(:source)), 
select: u.email)

Unfortunately you will need to define all of the fields that you want to return from the table in the :select portion of the query.

Either that or you will need to build an ecto_cockroach library that can integrate with the ecto_sql library. This would allow the ecto_sql library do that majority of the work and the ecto_cockroach library would only deal with the differences, such as this. This is already how the ecto_sql library works for the various official implementations just as postgres and mysql. I will admit though, I don’t know if the ecto_sql library is setup for this type of integration.

fireproofsocks

fireproofsocks

I’m interested in this as well… unfortunately, it looks to be a bit trickier than using a simple fragment.

For example, here’s the above query for users:

iex(26)> from(u in fragment("? AS OF SYSTEM TIME '-5m'", ^User.__schema__(:source)), select: [:name, :email])
#Ecto.Query<from f0 in fragment("? AS OF SYSTEM TIME '-5m'", ^"users"),
 select: [:name, :email]>
iex(27)> |> Repo.all()
[debug] QUERY ERROR db=0.0ms queue=5.4ms idle=1805.7ms
SELECT f0."name", f0."email" FROM $1 AS OF SYSTEM TIME '-5m' AS f0 ["users"]
** (Postgrex.Error) ERROR 42601 (syntax_error) at or near "1": syntax error

    query: SELECT f0."name", f0."email" FROM $1 AS OF SYSTEM TIME '-5m' AS f0

    hint: try \h <SOURCE>

source SQL:
SELECT f0."handle", f0."page_title" FROM $1 AS OF SYSTEM TIME '-5m' AS f0
                                         ^
    (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:1047: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:945: Ecto.Adapters.SQL.execute/6
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
    (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
    iex:27: (file)

`
Note that the generated query is not correct:

-- This is invalid
SELECT f0."name", f0."email" FROM $1 AS OF SYSTEM TIME '-5m' AS f0

The SYSTEM TIME needs to place the f0 immediately after the FROM. The correct variant would be:

SELECT f0."name", f0."email" FROM $1 f0 AS OF SYSTEM TIME '-5m'

I’m not sure how to go about modifying this or even if it’s possible.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement