OvermindDL1

OvermindDL1

Erlang 21.2 erts-10.2 released

http://www.erlang.org/news/125

A few good fixes and enhancements, SSL added more methods and is faster, socket polling is faster, however 2 big things that I like!

  • New counters and atomics modules supplies access to highly efficient operations on mutable fixed word sized variables.

The new :atomics module wraps the low level hardware atomic instructions without any locking, it’s use-cases are limited but EXTREMELY useful for where they are useful. I find it odd how the array part is 1-indexed though, erlang has a lot of mixed 0-index and 1-index code, and 1-indexed means the math is harder… blah…

The new :counters module uses the new :atomics module to do ETS-style counter handling but using hardware atomic instructions so there is no locking or cost. Same thing with the really weird non-mathematical 1-indexed arrays though.

  • New module persistent_term!. Lookups are in constant time! No copying the terms!

You know the global module that discord put out that recompiles a module to bake terms into it for super-fast referencing and no-copies and so forth? That’s basically the new erlang persistent_term library! Here’s a use session in Elixir!

iex(1)> :persistent_term.info()                        
%{count: 0, memory: 152}
iex(2)> :persistent_term.put(:blah, 42)
:ok
iex(3)> :persistent_term.info()        
%{count: 1, memory: 192}
iex(4)> :persistent_term.get(:blah)
42
iex(5)> :persistent_term.get()     
[blah: 42]
iex(6)> :persistent_term.put(:blorp, make_ref())
:ok
iex(7)> :persistent_term.get()                  
[blah: 42, blorp: #Reference<0.3525248772.1957167107.195526>]
iex(8)> :persistent_term.erase(:blah)
true
iex(9)> :persistent_term.get()                  
[blorp: #Reference<0.3525248772.1957167107.195526>]
iex(10)> :persistent_term.info()
%{count: 1, memory: 216}

So it’s slow to put but crazy-fast to get, reference, and pass around the values stored within! In general you shouldn’t use this unless you know what you are doing as there is no garbage collection, should generally be used for global settings and data, etc…

Most Liked

michalmuskala

michalmuskala

Changing anything in persistent_term means running a GC over all the processes in the system and copying all of the data in persistent_term. It can get very expensive on bigger systems.

michalmuskala

michalmuskala

One thing to ask is, if it would be actually beneficial. Binaries (and I assume your entries are just plain binaries) are stored off-heap, so they are not copied when retrieved from ETS anyway. This removes the biggest advantage of persistent_term over plain old ETS.

OvermindDL1

OvermindDL1

Think of it this way: If it’s something you would normally compile ‘into’ a models code for efficiency then use it, else you probably want ETS (or mnesia to keep it serialized to disk and hold it in memory on load).

So for a blog I’d personally use either flat-files on disk or store it in mnesia with disk and memory copies, I’m not sure I’d use this for that, though your data is static enough that it ‘should’ be fine.

Just remember, :persistent_term has a max storage size of 1 gig unless you set the _MIscs 2048 or so option (that one would raise it to 2 gigs, the value is in megabytes).

When a new value is put then it is visible everywhere immediately on the node because when an old value is replaced or erased then a global garbage collection is run, which can pause things, that collection will replace any links to the global data in all processes to a local copy of the data instead.

Just remember that accessing data in :persistant_term is way fast, but changing data in any way is very slow, and in some cases it can be really really slow. But it’s still faster than recompiling a module to ‘intern’ the terms (which would destroy linked processes if it got updated, so :persistent_term is definitely better there). :slight_smile:

archan937

archan937

Hi there, I created a Hex package to experiment for distributed code compilation within an Elixir cluster which would take away the “pain” of writing: https://github.com/archan937/clustorage

I actually started building this with Discord’s FastGlobal package as inspiration.

I’m curious for your thoughts about it :slight_smile:

kip

kip

ex_cldr Core Team

Are their any anecdotes yet reflecting relative performance of :persistent_term versus functions that bake in data?

:persistent_term looks like a possible good approach for storing locale data in my ex_cldr package since its large and static. However it is a fairly complex Map.t() so their would be some penalty to pay in keeping the data as one large map as apposed to the current approach which decomposes the map into different functions.

Where Next?

Popular in Erlang News Top

Devtalk
A new Erlang news item has been posted! Link: Release OTP 23.3.2 · erlang/otp · GitHub Posted via Devtalk.
New
Devtalk
A new Erlang news item has been posted! Link: Release OTP 24.1.6 · erlang/otp · GitHub
New
Devtalk
A new Erlang news item has been posted! Link: Release OTP 21.3.8.23 · erlang/otp · GitHub Link: Release OTP 22.3.4.18 · erlang/otp · ...
New
Devtalk
A new Erlang news item has been posted! Link: Release OTP 24.0 Release Candidate 1 · erlang/otp · GitHub Posted via Devtalk.
New
Devtalk
A new Erlang news item has been posted! Link: Release OTP 22.3.4.17 · erlang/otp · GitHub Posted via Devtalk.
New
Nicd
The Erlang Ecosystem Foundation is a new non-profit organization dedicated to furthering the state of the art for Erlang, Elixir, LFE, an...
New
garazdawi
Until the 10th of May the Erlang/OTP 24.3.4.17 patch had an incorrect prebuilt archive on github (the otp_src_24.3.4.17.tar.gz artifact),...
New
kennethL
You don’t want to miss the third post in the series about BEAM the execution engine for Erlang. This time we are going to explore the jus...
New
kennethL
Here is a new interesting blog post from the OTP team, “The Road to the JIT”. This post gives a history lesson that outlines the major Er...
New
bjorng
The third release candidate for Erlang/OTP 22.0 has been released. Erlang/OTP 22 is a new major release with new features and improveme...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New

We're in Beta

About us Mission Statement