brainlid

brainlid

Podcast: ThinkingElixir 129 - Time series data with Timescale DB

Episode 129 of Thinking Elixir. Tracking, analyzing and visualizing time series data can add a lot of business value to a project! We met up with Dave Lucia to learn more about Timescale DB, a PostgreSQL extension that adds time series tools to our regular database. Dave also created a timescale hex package to make it easier to work with hypertables and hyperfunctions. We learn why Timescale DB makes sense over other options, how to get started with it, example use cases, helpful resources and more!

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

We’ve been using TimescaleDB at CargoSense for a couple of years now, and it has worked really well for us. In particular, the compression feature gives us 96%+ compression rate which makes a huge difference. Concretely we have a ~terrabyte’s worth of data taking up 55 gigabytes and for our query patterns, we see negligible performance performance impact for queries on compressed chunks.

For us it’s a sweet spot on trade offs. We get to keep the simplicity of running “just” postgres, we get to keep all our transactional guarantees with other code, and we also get to store bunch of timeseries data.

The only “gotcha” is that, like all partitioned tables, you need to be storing “relevant time range” values around so that you can query the right range. Eg if you had a sensor and you wanted to query its temperature readings in a shipment, you’d want to do a query like

Repo.all(from r in Reading,
  where: r.sensor_id == ^sensor.id,
  where: r.recorded_at >= ^shipment.starts_at and r.recorded_at <= ^shipment.ends_at

The r.recorded_at >= ^shipment.starts_at and r.recorded_at <= ^shipment.ends_at bit is important, even if a sensor is never used in multiple shipments because you need to help the query planner know in which chunks to go looking for readings.

brainlid

brainlid

Hi @benwilson512!

I’m glad to hear you’ve had such a good experience with it as well. And thanks for the query tip. Very helpful!

Schultzer

Schultzer

You might not need timescale with Postgres today.

Designing high-performance time series data tables on Amazon RDS for PostgreSQL | AWS Database Blog

Schultzer

Schultzer

Yes, it’s Postgres in general!

Where Next?

Popular in Podcasts Top

brainlid
Episode 119 of Thinking Elixir. Kat Marchán was the maintainer of NPM CLI for 5 years but now works at Microsoft on the VisualStudio prod...
New
brainlid
It’s important to learn safe coding practices. As developers, we want people to love our products and happily pay to use them. We also wa...
New
brainlid
Episode 132 of Thinking Elixir. LiveView isn’t always the right answer. Chris Nelson created LiveState to enable embedding custom HTML co...
New
brainlid
Securing our apps is our responsibility as developers. We are the custodians and the guardians of our user’s data. We met up again with M...
New
brainlid
Episode 111 of Thinking Elixir. Ever wanted to deploy a PR for show-and-tell, QA testing, or to get feedback on it? Jason Axelson joins u...
New
brainlid
In episode 87 of Thinking Elixir, José Valim returns to continue with part 2 of our 5 part series as we count down to the 10 year anniver...
New
brainlid
In episode 75 of Thinking Elixir, we talk with Dave Lucia about Simplebet’s use of RabbitMQ and Commanded for solving unique real-time pr...
New
brainlid
Episode 115 of Thinking Elixir. ElixirConf US 2022 just finished! We cover the big announcements, talk highlights, and other relevant tec...
New
brainlid
AI topics are all the rage right now, so we team up with Sean Moriarity to unpack what’s going on and what it means for Elixir. Sean talk...
New
brainlid
News covers the new Phoenix.Template project, upcoming Nx work for Elixir using multiple GPUs, benefits of Elixir as seen by new people l...
New

Other popular topics Top

TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New

We're in Beta

About us Mission Statement