apr
Remote Observer Connection Issues
Hi,
I recently migrated from distillery to mix release. While on distillery, I had setup my release such that I was able to debug the remote production node with observer. After migrating to mix release, I find that whenever I try to connect to the remote node via observer (it shows up in the observer Nodes drop down menu), I get this error: Error occurred on remote node and the following error message in the console: [error] [node: :"production_node@127.0.0.1", call: {:observer_backend, :sys_info, []}, reason: {:badrpc, :nodedown}]
I have all the dependencies installed on the release (observer, wx and runtime_tools), so I am not sure why I am seeing this issue. Furthermore, it is weird that it worked on distillery and is giving me issues now. Anyone have any ideas?
Thanks!
Marked As Solved
vfsoraki
I had the exact same issue. I fixed it but then another problem occurred.
To fix, check
- You have the following in
env.sh[.eex]. Note that using short names won’t work, or I couldn’t get it to work. Use long names.
export RELEASE_DISTRIBUTION=name
export RELEASE_NODE=<%= @release.name %>@127.0.0.1
-
Restrict the port used by node to a specified one, e.g. adding
-kernel inet_dist_listen_min 60606 inet_dist_listen_max 60606toELIXIR_ERL_OPTIONSinenv.sh[.eex]. -
Forward ports like
ssh -L 4369:127.0.0.1:4369 -L 60606:127.0.0.1:60606 nodehost. Note you shouldn’t haveepmdrunning in your system, and you have to use the remote one. To make sure you are not runningepmdlocally, runepmd -killbefore opening ssh session. -
Start IEx shell like
iex --hidden --cookie 'yourcookie' --name local@127.0.0.1
Now when I run Node.connect :"distname@127.0.0.1" I get true and nodes connect. But when I do :observer.start locally and select the remote node in Nodes menu, I get the following error:
16:22:06.248 [error] [node: :"distname@127.0.0.1", call: {:observer_backend, :sys_info, []}, reason: {:badrpc, {:EXIT, {:undef, [{:observer_backend, :sys_info, [], []}, {:rpc, :"-handle_call_call/6-fun-0-", 5, [file: 'rpc.erl', line: 197]}]}}}]
Then I found about :runtime_tools which contains :observer_backend. See Troubleshooting in http://blog.plataformatec.com.br/2016/05/tracing-and-observing-your-remote-node.
After this, it works like a charm!
Also Liked
apr
Thanks for replying. I am tunneling via ssh and setting the same cookie in both places. I had the same setup previously when releasing vis Distillery as well when it did work, so it not working when releasing via Mix makes me suspect that maybe I am not packaging some dependency correctly (esp since the remote production node is discoverable locally)
steven7
I am having an exact same issue while migration from Distillery to Elixir 1.9 Mix Release, I would also appreciate if anyone can shed some light on this.







