zacky1972
Pelemay Fast Parallel map has just been released!
This is a present of the end of year for Elixir programmers from me: The new package “Pelemay Fast Parallel map” has been released! It drives multi cores and is 3x faster than Enum! Check it out, soon!
Pelemay Fast Parallel map (or pelemay_fp) - Fast parallel map function for Elixir
- https://hex.pm/packages/pelemay_fp/0.1.0
- https://github.com/zeam-vm/pelemay_fp
- https://hexdocs.pm/pelemay_fp/0.1.0/PelemayFp.html
Pelemay Fast Parallel map: provides fast Parallel map function, similar to the Enum module, although computations will be executed in parallel using Process.spawn/4.
Here is a quick example on how to calculate the square of each element with PelemayFp:
list
|> PelemayFp.map(& &1 * &1)
We conducted performance evaluation of PelemayFp, Pelemay, Flow, Enum and Pmap on iMac Pro (2017):
## PelemayFpBench
benchmark name iterations average time
PelemayFp 200 10177.85 µs/op
Pelemay 100 16762.24 µs/op
PelemayFp and Pelemay 100 18532.70 µs/op
Flow (without sorting) 100 18731.43 µs/op
Enum 50 31283.36 µs/op
Flow (with sorting) 10 105091.00 µs/op
Pmap 1 1213749.00 µs/op
Thanks! A happy new year!
Most Liked
zacky1972
PelemayFp that fixed the issue that unreceived messages remained has just been released!
zacky1972
I added benchmark of Stream and Task.async_stream. Thanks
## PelemayFpBench
benchmark name iterations average time
PelemayFp 100 14143.22 µs/op
Pelemay 100 18627.19 µs/op
PelemayFp and Pelemay 100 20414.04 µs/op
Flow (without sorting) 100 20720.33 µs/op
Enum 50 31957.12 µs/op
Stream 50 38792.98 µs/op
Flow (with sorting) 10 116906.90 µs/op
Pmap 2 830539.50 µs/op
Task.async_stream 1 1099493.00 µs/op
al2o3cr
I tried out some changes to increase the per-element workload, because there’s a fair bit of overhead with starting communicating processes. See notes in https://github.com/zeam-vm/pelemay_fp_benchmark/pull/1
LostKobrakai
How does this compare to Task.async_stream, which would be the core way of separating computations into different processes?
al2o3cr
PelemayFp splits the incoming value into chunks of 12000, so I’m not sure what this benchmark demonstrates - it will only spawn one worker. FWIW, the code in pelemay_fp_benchmark/bench/pelemay_fp_bench.exs at main · zeam-vm/pelemay_fp_benchmark · GitHub use a limit of 100k.
Popular in Libraries
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
The deployment platform built for Elixir: PaaS ease, VPS control.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"