brainlid

brainlid

Podcast: ThinkingElixir 129 - Time series data with Timescale DB

Episode 129 of Thinking Elixir. Tracking, analyzing and visualizing time series data can add a lot of business value to a project! We met up with Dave Lucia to learn more about Timescale DB, a PostgreSQL extension that adds time series tools to our regular database. Dave also created a timescale hex package to make it easier to work with hypertables and hyperfunctions. We learn why Timescale DB makes sense over other options, how to get started with it, example use cases, helpful resources and more!

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

We’ve been using TimescaleDB at CargoSense for a couple of years now, and it has worked really well for us. In particular, the compression feature gives us 96%+ compression rate which makes a huge difference. Concretely we have a ~terrabyte’s worth of data taking up 55 gigabytes and for our query patterns, we see negligible performance performance impact for queries on compressed chunks.

For us it’s a sweet spot on trade offs. We get to keep the simplicity of running “just” postgres, we get to keep all our transactional guarantees with other code, and we also get to store bunch of timeseries data.

The only “gotcha” is that, like all partitioned tables, you need to be storing “relevant time range” values around so that you can query the right range. Eg if you had a sensor and you wanted to query its temperature readings in a shipment, you’d want to do a query like

Repo.all(from r in Reading,
  where: r.sensor_id == ^sensor.id,
  where: r.recorded_at >= ^shipment.starts_at and r.recorded_at <= ^shipment.ends_at

The r.recorded_at >= ^shipment.starts_at and r.recorded_at <= ^shipment.ends_at bit is important, even if a sensor is never used in multiple shipments because you need to help the query planner know in which chunks to go looking for readings.

brainlid

brainlid

Hi @benwilson512!

I’m glad to hear you’ve had such a good experience with it as well. And thanks for the query tip. Very helpful!

Schultzer

Schultzer

You might not need timescale with Postgres today.

Designing high-performance time series data tables on Amazon RDS for PostgreSQL | AWS Database Blog

Schultzer

Schultzer

Yes, it’s Postgres in general!

Where Next?

Popular in Podcasts Top

brainlid
We talk with Marc-André Lafortune about reducing Elixir project compile times. On larger projects, when a single file like a view templat...
New
brainlid
Episode 160 of Thinking Elixir. After experience with a LiveView “anti-pattern”, Andy Glassman developed an “Async Pattern” for loading d...
New
brainlid
In episode 79 of Thinking Elixir, we talk with Nathan Willson about GEMS, his collaborative music generator written in LiveView. He expla...
New
brainlid
In episode 81 of Thinking Elixir, we talk with Digit and Quinn Wilton about the Burrito project. It wraps up Elixir to a single binary, e...
New
brainlid
In episode 76 of Thinking Elixir, we talk with host David Bernheisel about his recently released Safe Ecto Migrations guide. Intended as ...
New
brainlid
In episode 93 of Thinking Elixir, we talk with Michael Lubas about protecting our Phoenix applications from common automated bot attacks....
New
brainlid
Episode 111 of Thinking Elixir. Ever wanted to deploy a PR for show-and-tell, QA testing, or to get feedback on it? Jason Axelson joins u...
New
brainlid
In episode 105 of Thinking Elixir, we learn about Plausible Analytics, a privacy respecting alternative to Google Analytics that is cloud...
New
brainlid
In episode 83 of Thinking Elixir, We talk with Isaac Yonemoto about the Zig language and his Zigler Elixir library. We learn where Zig ca...
New
brainlid
In episode 75 of Thinking Elixir, we talk with Dave Lucia about Simplebet’s use of RabbitMQ and Commanded for solving unique real-time pr...
New

Other popular topics Top

yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New

We're in Beta

About us Mission Statement