zachallaun

zachallaun

No longer able to connect local Livebook to remote Fly.io node

Hi friends,

This topic has come up a handful of times before on the forum, but I wanted to solicit some advice on how to debug an attached node connection issue I’ve started to experience with Livebook.

Context & setup

This setup was previously working. In fact, I posted on this forum to share how I got it set up and the script I was using to start the Livebook server with the correct flags. Sometime in the last couple of weeks, this setup has stopped working. I’ll describe exactly what I’m seeing a bit further down.

To get it out of the way, here are the versions of everything I’m using:

  • Latest Livebook - 0.8.1 (though also tried with 0.7.2, 0.7.1, and 0.7.1)
  • Latest Elixir/OTP - 1.14.3 / 25.2.1 (using same versions both locally and on server)

Other bits and bobs:

  • Connection is through a wireguard tunnel between me and the server, as described in this article on fly.io.
  • The node name and cookie is being set as expected.
  • Livebook is being started with -proto_dist inet6_tcp.
  • I am able to connect to the node through iex - more on that below.

Description of issue

After launching the Livebook server, opening a notebook, and attempting to connect to the attached node, Livebook hangs until the terminal eventually reports this error:

12:35:20.007 [error] GenServer #PID<0.592.0> terminating
** (CaseClauseError) no case clause matching: {:badrpc, :nodedown}
    (livebook 0.8.1) lib/livebook/runtime/erl_dist.ex:74: anonymous fn/3 in Livebook.Runtime.ErlDist.load_required_modules/1
    (elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (livebook 0.8.1) lib/livebook/runtime/erl_dist.ex:71: Livebook.Runtime.ErlDist.load_required_modules/1
    (livebook 0.8.1) lib/livebook/runtime/erl_dist.ex:60: Livebook.Runtime.ErlDist.initialize/2
    (livebook 0.8.1) lib/livebook/runtime/attached.ex:45: Livebook.Runtime.Attached.connect/1
    (livebook 0.8.1) lib/livebook/session.ex:1519: Livebook.Session.handle_action/2
    (elixir 1.14.3) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (livebook 0.8.1) lib/livebook/session.ex:878: Livebook.Session.handle_cast/2
Last message: {:"$gen_cast", {:queue_cell_evaluation, #PID<0.594.0>, "setup"}}

Critically, I am able to connect to the node manually through IEx. I can start a local iex using the following command, connect to the remote node, and run some code through Node.spawn(server, ...).

# ERL_AFLAGS="-proto_dist inet6_tcp" iex --name local@127.0.0.1 --cookie my-shared-cookie
iex> server = :"my-server-node-name@some:ipv6:address"

iex> Node.connect(server)
true

# Test module that prints "hello"
iex> Node.spawn(server, Hello, :world, [])
hello
#PID<...>

I’m not really sure where to go from here. Any suggestions would be greatly appreciated!

Marked As Solved

zachallaun

zachallaun

To close the loop on this:

It had nothing to do with Livebook! An upgrade in WSL2 caused the MTU for the eth0 network interface to be set to 1280, which was apparently too small with the added size of the Wireguard protocol. So any normal network requests would be fine, but requests “wrapped” by Wireguard would fail (unless they could fit in a single packet).

Fixed by the following:

$ sudo ip link set dev eth0 mtu 1500

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New

Other popular topics Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement