D4no0

D4no0

Sandbox test setup for redis

I have currently some functionality that fetches/stores some data in redis using redix. From what I see when it comes to testing, there is nothing specified in the documentation, so I guess the most straightforward way would be to mock it.

Instead of mocking, I was wondering if it would be possible to take the approach the Ecto took, using the real database but being able to test in isolation. It does seem that redis supports concept of transaction, even though not sure how much of it overlaps with how it works in DB engines like Postgres.

What are your thoughts on this?

Most Liked

sorentwo

sorentwo

Oban Core Team

You don’t necessarily need multiple Redis instances. Every instance has 16 “databases” by default, addressable from 0 through 15. The data in each of those databases is completely isolated, and you’re actually able to selectively flush a single db.

You can use separate database instances for each test case (anything that is testing for the presence of specific values) and then call flushdb afterwards to clear only that database.

whatyouhide

whatyouhide

Elixir Core Team

I don’t think Redix itself should change to accommodate testing here. Redix is concerned with talking to the Redis database, and it gives you enough tools that you can build testing utilities on top of it. The databases examples that @sorentwo mentioned is already a really good way to go with at least 16 parallel tests using Redis. Multiple instances also work—Redis is really cheap to run.

Redis itself doesn’t support complex transactions the way relational DBs (Postgres and co) that Ecto supports do, so implementing a sandbox here would not really be possible.

In case mocks are your choice, I don’t think Redix should declare and/or implement behaviours for that either. Redix’s API is pretty slim and chances are you’re not using all of it, so declaring the interface you need to talk to Redis for your use case and then declaring mocks for that seems like a good approach to me.

mudasobwa

mudasobwa

Creator of Cure

If I had a logic that requires a real redis instance to test (e. g. pubsub subscription, polling from the other place, or like,) I’d definitely supply a PR to redix implementing the testing framework.

I never had such a condition (e. g. I used redis as a pure kv-store,) and since I trust in Andrea’s coding skills, I mock it completely.

mudasobwa

mudasobwa

Creator of Cure

What an original approach! :slight_smile:

Well, then you might want to open the issue to:

  • declare Redix.Behaviour
  • make Redix implement it
  • create Redix.Test implementing this same behaviour and mimicking Redix by optionally wrapping each and every call to transaction_pipeline/3 and telemetrying it

Since redis doesn’t have a rollback mechanism at all, you’ll need to WATCH the data and/or implement the rollback manually. Here is a good blog about why You Don’t Need Transaction Rollbacks in Redis.

You anyway would need to implement all the above, hence my proposal to do a PR.

D4no0

D4no0

I won’t have time to do this now, as I will need some time to understand how test setups work and how to do the rollbacks correctly (so it would work with async tests too), but I would be most certainly interested in adding the feature as soon as I get a little bit more time as there is a lot of redis usage at the current projects I work on.

I wonder if it makes sense to make the implementation more abstract, as I will be most probably be interested in doing this with MongoDB tests too and I bet there is no way to do that ATM.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
wernerlaude
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement