ananthakumaran

ananthakumaran

Distributed tracing in elixir

With the proliferation of microservice architecture, distributed tracing has become more and more important in recent days. Because the landscape is quite new, there are multiple open standards like opentracing, opencensus and multiple opensource implementations like zipkin, jaeger, commerical implementations like datadog, newrelic, elasticsearch etc. There are also multiple client implementations available for elixir language like spandex, ex_ray and opencensus

The problem is, as a user, I am stuck with analysis paralysis. I have been postponing the adoption, because I am afraid of making the wrong bet. With the current setup, switching from one implementation to another would be a time consuming process (depends on how big the codebase is).

beam telemetry project aims to solve the same issue in the metrics landscape. If it gets implemented by the majority of the community, then switching from let’s say graphite to prometheus would be quite straight forward.

Is there any similar effort for distributed tracing?

Most Liked

tristan

tristan

Rebar3 Core Team

Note that OpenCensus is the only actual standard for what has to be defined to really have distributed tracing. OpenTracing leaves almost everything up to the implementation, which does not produce compatible libraries. OpenCensus provides complete specs and implementations for distributed tracing across multiple systems. This also involves some W3C standards, https://www.w3.org/TR/trace-context/ and https://w3c.github.io/correlation-context/

OpenCensus and OpenTracing will ultimately be merged and the result live under CNCF. So choosing OpenCensus is a safe bet.

Also note that we who work on OpenCensus have been discussing with those who work on Spandex to combine efforts.

I hope the OpenCensus library can become the “telemetry library” of distributed tracing on beam :slight_smile: – we’ll also be integrating with telemetry for statistic reporting.

tristan

tristan

Rebar3 Core Team

The intent is to be mostly plug and play, but libraries are only instrumented when someone who needs it decides to do the work and release it.

So right now plug and tesla can have tracing enabled in a fairly simple way of adding middleware/plugs https://github.com/opencensus-beam/

Instrumenting hackney isn’t really possible without internal changes to hackney itself (it has no middleware support) and I’ve been talking to Benoit about it but we haven’t moved forward with any change yet.

GregMefford

GregMefford

I’ll echo what @tristan said, that we’re working on how the Spandex team can integrate and/or combine with the OpenCensus-BEAM project, because I think that’s the spec that makes the most long-term sense. I’d say that if you’re looking to add something to an app today, maybe try adding OpenCensus and file issues for any trouble you having (both documentation and technical issues), so we can figure out where the gaps are.

I still haven’t taken the time to figure out what that experience is like myself from an Elixir app, but if decide that you’d rather use Spandex, we have a fully-worked example that you can use to see what’s required to integrate Spandex with the commonly-used frameworks (Plug, Phoenix, Ecto, Datadog). Check out the links to the diffs in the README to see the exact changes you’d need to make in your app. Spandex only supports Datadog though, so that might be a deal-breaker for many environments. It wouldn’t be hard to add a new back-end, but no one has contributed one yet. We currently use Spandex at Bleacher Report, and it interoperates seamlessly with Datadog’s official clients for Ruby, NodeJS, .NET, etc.

tty

tty

This is really cool!

Originally I thought the questions was regarding Trace Tool Builder. :slight_smile:

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement