webuhu

webuhu

Count number of occurrences for each attribute

I have the honor to program a plant filter search.
Means you can search plants meeting certain conditions (attributes).
But I need some advice for counting number of occurring attributes.

Over simplified records:

- id: 1; attributes: [color: [red, green], size: 10];
- id: 2; attributes: [color: [red], light: [moderate];

The attributes are varying!

The filter search functionality should provide a counter for each attribute:

color: red (2), green (1)
size: 10 (1)
light: moderate (1)

Now if you’ll select moderate - green & size will be (0), red (1).

Sadly PostgreSQL is getting to its limit with this requirement.
Counting attributes is too slow.
So I’m in search of an alternative approach. → Advice needed!


More details:

  • It concerns about 2,5 k records
  • Attributes of all records will have a size of “just” about 10 - 15 MB
  • The first most common filters will reduce the record amount just by about 20%
  • Update of records / attributes won’t be very frequent

Ideas:

Should I just cache a list of all records + attributes in for example ETS?
And than do the counting / filtering with some Enum operations?

Or are there any other good approaches I’m currently blind for?

Marked As Solved

joey_the_snake

joey_the_snake

It sounds like you are trying to do faceted search. There are some tips in this article that might be helpful for you: Faceted navigation and search | Bun

It is focused on how to do it in Postgres. With 2.5K records I believe you should be able to make this fast just using Postgres.

Also Liked

webuhu

webuhu

So this works perfectly fine. Thanks a lot.

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement