anurag.peshne

anurag.peshne

Arrays

what is the closest thing to Arrays in Elixir. By arrays I mean, a container for values which I can access in constant time.

I’ve looked at tuple, but according to the documentation:

Tuples are not meant to be used as a “collection” type (which is also suggested by the absence of an implementation of the Enumerable protocol for tuples): they’re mostly meant to be used as a fixed-size container for multiple elements.

What I actually want to do:
I want to store n processes in an array and periodically pick a random process and send it a message.
I’m open to other suggestions too.

Marked As Solved

rvirding

rvirding

Creator of Erlang

For what you describe a tuple actually seems like a good choice. Some questions though:

  1. How many elements in this “array”?
  2. How static is this “array” to be?
  3. Is it to be globally accessed or through one process?
  4. When randomly picking a process is it enough just to generate a random number and use that as index?
  5. Is there any other semantics to this “array” than just storing the pids and picking one randomly?

Having very large tuples is inefficient when you update them. You basically make a copy of the whole tuple every time, but not the elements themselves of course. If it is relatively static then a tuple will work fine, doing your occasional deletes would probably work fine. Otherwise the :array module mentioned by @NobbZ is a good alternative. Maybe just using a map would work as well.

Tuples :array's and maps are local to one process so if multiple processes need to access it then perhaps using an ETS table would be better. These can be globally accessed however they have no built in transaction mechanism so you might need a process to manage them.

Also Liked

NobbZ

NobbZ

As a direct answer to your question: there is the :array module from erlang.

That module uses a set of nested tuples and reading is O(1). I’m not sure about its writing runtime though, and you’d have to implement filling logic on delete by yourself (either take the last in the hole, or let every following item go one step forward, thats up to you, depending on if original order matters).

There is also the array package which is a wrapper around the erlang module, providing some protocols as well to make it easier to handle them from elixir site. But it hasn’t been updated in the last 3 years, therefore it might spit some warnings in a current elixir or not work at all.


To try to find another solution to your problem. It sounds a bit as if you want to create some kind of load balancer which is giving away its work randomly on the workers.

By picking randomly from a fixed set, you might get non-uniform distribution due to how PRNGs work.

Therefore I’d suggest one of these solutions:

  1. Only shuffle once, use that as a base for roundrobbin. You can use :queue module for a datastructure which can pull from the head and push to the tail in amortized o(1).
  2. Shuffle the original list and keep a copy of it around. Each time you need to send a job, use the head of the shuffled list until it is empty, then reshuffle the original list and keep going.
Qqwy

Qqwy

TypeCheck Core Team

I have two fine packages for you:

  • Arrays exposes a common array interface, with multiple different implementations that you can try if the default performance characteristics are not to your liking. (The two built-in implementations are a map-based implementation and an :arrays-based implementation.)

This is probably perfect for your needs, although the last time I had time to work on it I was unable to write proper documentation and tests, so these are still lacking.

If you need to do more mathematical stuff with them:

  • Tensor gives you one-dimensional vectors, two-dimensional matrices and n-dimensional tensors, and defines many common operations on them. Tensor stores these in a sparse format, which may or may not be suited well to your problem domain.

Tensor is very stable and greatly documented and tested.

kylethebaker

kylethebaker

Does order matter? Do you need to access items by key or just at random? Map has been mentioned, but there is also MapSet if you don’t need keys. Both implement the enumerable protocol, so to get a random value you can pass it to Enum.random. What I’ve done in the past when I want constant time lookups by key and also maintain order is to create two structures: a map for the constant lookups, and a list of keys for the order.

You may also consider looking into using a Registry if you want a container for processes, though for your use case it might be easier to just keep simple collection of pids.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

You may want to elaborate on your use case. Various datastructures (including arrays) have various performance characteristics for different kinds of operations and without knowing what operations you have in mind, it’s hard to recommend the best approach.

Tuples have constant lookup time, but have to be copied wholesale for every change, which is generally undesirable.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Ah yeah this changes things. You almost certainly want to use Registry, which will ensure that individual values are tied to the life cycle of the process.

Registry is backed by :ets, which is a very high performance key value store that also enables concurrent reads / writes.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement