mekusigjinn

mekusigjinn

Anybody actually use Litestream (the SQLite backup/syncing layer) with Elixir?

The creator is being supported by Fly.io (hired) to work full time on Litestream.
I have a new project I’m ramping up and it’s going to be an e-commerce project.

I’m wondering how Ecto Sqlite3 layer can work with Litestream, in an e-commerce platform.
Because if that works, then I can:

  • use S3 as a very cheap backup solution
  • auditing
  • provide fast user experience because no need for network hop (like when using Postgres)
  • it seems like this is somewhat of a next generation data store architecture

Thanks in advance!

Most Liked

derpycoder

derpycoder

Litestream is a backup solution, you have to deploy S3 separately or pay for it.

“Litestream has limitations. I built it for single-node applications, so it won’t work well on ephemeral, serverless platforms or when using rolling deployments. It needs to restore all changes sequentially which can make database restores take minutes to complete.” - Ben Johnson


Why not use a distributed Rqlite?!?

When you absolutely must not lose any of your data.

rqlite is an easy-to-use, lightweight, distributed relational database, which uses SQLite as its storage engine. rqlite is simple to deploy, operating it is very straightforward, and its clustering capabilities provide you with fault-tolerance and high-availability.

It also has an in memory database mode with max 2 gb per node capacity! Which won’t loose data on restarts, since the Raft log is the authoritative store for all data, and it is stored on disk by each node, an in-memory database can be fully recreated on start-up from the information stored in the Raft log.


Also don’t be afraid to use the right tool for the right job.

Like Meilisearch for faster text search.
Or running MinIO if you want object storage.
Or CockroachDB if you want distributed Postgres like DB.

I’m using all 3 and still going to add Rqlite for storing data close to application, in memory. For config management that doesn’t belong on main db.


By no means thats a complete list of databases you will need.

You will end up needing ClickHouse for OLAP or Time series data storage, prometheus for monitoring, loki for logging, a key value store for something & who knows what, eventually.


P.S. Please be wary of the GitHub issues section.

I have been burned by jumping the gun and boarding the hype train, only to reach the end of the road.
Big teams can afford to migrate away from a tech debt, but you will be left hanging.

There’s a reason people prefer Postgres, and now I know why!!

engineeringdept

engineeringdept

I use it on a personal project that needs a sizeable database that’s predominantly read only, for which I want to minimise the number of moving (and costly) parts. It works well! But I’d stick with hosted Postgres for most things.

daveboo

daveboo

… I like that!

Where Next?

Popular in Questions Top

bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
wernerlaude
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement