Fl4m3Ph03n1x

Fl4m3Ph03n1x

Best way to get multiple keys from static ETS table

Background

I have an ets table that has, for each key, a list of items. ETS tables have O(1) complexity when using :ets.lookup, as we all know and they are super fast, but for my specific use case, I need to ask to the ets table for more than 1 key at a time.

This means I have 2 options:

  1. Perform multiple lookups
  2. Perform a match or select on the keys

Multiple lookups

I don’t really like this approach because it would mean my operation is not atomic. This is a big no no.

Match or Select on keys

This is my preferred approach since I want the operation to be as efficient as possible. I am also not going to look for patterns in keys, I am simply going to look directly for keys that match my spec, so it should be fast.

I do have several questions about match and select though (see Questions section)

Issue

The problem here is that I don’t know how to use :ets.match or it’s select variant correctly. This is an example:

iex(27)> table = :ets.new(:table, [:public])
#Reference<0.830523521.2311716867.20185>
iex(28)> :ets.insert(table, {:a, [{:banana, :orange, 10}, {:tomato, :potato, 6}]})
true
iex(29)> :ets.insert(table, {:b, [{:car, :moto, 1}, {:plane, :heli, 60}]})        
true
iex(30)> :ets.tab2list(table) 
[
  b: [{:car, :moto, 1}, {:plane, :heli, 60}],
  a: [{:banana, :orange, 10}, {:tomato, :potato, 6}]
]
ex(32)> :ets.match(table, [:a, :b]) # I want to get the values from both keys :a :b
[]

Questions

  1. What am I doing wrong?
  2. How do I optimize my match queries so they are atomic? From the docs:

Traversals using match and select functions may not need to scan the entire table depending on how the key is specified. A match pattern with a fully bound key (without any match variables) will optimize the operation to a single key lookup without any table traversal at all.

  1. The data on my table will never change. Does it matter if I make the table an ordered_set or are the gains for match and select not worth it?

Marked As Solved

sorentwo

sorentwo

Oban Core Team

This sounds like a good use case for :persistent_term. The data will never change so there isn’t a write penalty and you get extremely fast reads.

I’m curious though, if the data doesn’t change why are you concerned about atomicity?

Also Liked

jola

jola

iex(13)> :ets.select(table, [{{:"$1", :_}, [{:or, {:==, :"$1", :a}, {:==, :"$1", :b}}], [:"$_"]}]) 
[
  b: [{:car, :moto, 1}, {:plane, :heli, 60}],
  a: [{:banana, :orange, 10}, {:tomato, :potato, 6}]
]

Boom! You can adjust the match body to what you’re interested in. Also note that the match head has to match the table format.

I have a weird fascination with match specs.

jola

jola

The erlang code from that mailinglist could be translated into Elixir like this (using the sample code from the OP):

iex(5)> my_keys = [:b, :a]
[:b, :a]
iex(6)> :ets.select(table, (for key <- my_keys, do: {{key, :_}, [], [:"$_"]}))
[
  a: [{:banana, :orange, 10}, {:tomato, :potato, 6}],
  b: [{:car, :moto, 1}, {:plane, :heli, 60}]
]

Basically, it uses the comprehension to build a list of match specs, where each one has a fully bound key.

NobbZ

NobbZ

You can’t have a fully bound key in a match spec that checks for many keys. Either do one lookup per key and throw away atomicity (which might not be a great deal as @sorentwo already mentioned) or build a matchspec as shown by @jola (perhaps even use ex2ms and just write functions rather than weirdly nested tuples).

But you’ll need to give up either atomicity or O(1) reads…

dimitarvp

dimitarvp

The persistent_term module is a very recent addition. And a very useful one, too!

NobbZ

NobbZ

Well, ETS does not need to be 2 element tuples with the key beeing the first element… When you create the table with :ets.new/2, you can specify the position of the key using the :keypos option.

You can then use :ets.lookup/2 to lookup all entries that have match your key.

Where Next?

Popular in Questions Top

logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New

Other popular topics Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New

We're in Beta

About us Mission Statement