shahryarjb

shahryarjb

Mnesia load records one by one or lazy like Stream

Hi, I have a Mnesia table in my project, and there is a node on my OTP which sends a request every 24 hours to clean expired data. but my table size on my disc is more than 1.8g and I Don’t want to use select function, because this table is big and I need a way that is lazy like Stream.map.

1 - how can delete expired data one by on after a timeout in my GenServer handel_info ? is this way right ?
or what is your suggestion to delete records lazy like Stream

2 - how can count all records on disc mnesia, is there a lazy way to count ?

Thanks

Code which I am using and I think it is not lazy:

Mnesia.transaction(fn ->
     Mnesia.select(Token, [{{Token, :"$1", :"$2", :"$3", :"$4", :"$5", :"$6"}, [], [:"$$"]}])
end)
|> case do
  {:atomic, []} -> "no token"
  {:atomic, data} ->

    Enum.map(data, fn [id, _user_id, _token, access_expires_in, _create_time, _os] ->
      if access_expires_in <= System.system_time(:second) do
        Mnesia.dirty_delete(Token, id)
      end
    end)

    
  _ -> "no token"
end

Marked As Solved

tangui

tangui

Regarding your first question, you might want to use the Mnesia functions first/1 and next/2 (or dirty_first/1 and dirty_next/2 if you don’t want to use a transaction).

Alternatively, foldr/ and foldl/3 can also be used to browse a whole table.

Also Liked

dimitarvp

dimitarvp

Have you checked this out? GitHub - Nebo15/ecto_mnesia: Ecto adapter for Mnesia Erlang term database.

But I am not seeing a support for Repo.stream, not on the first glance anyway, so it likely won’t help you. :frowning:

I don’t know anything about Mnesia. Does it support any streaming?

Blindly shooting in the dark, you can also check the Mnesia GitHub topic?

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement