Fl4m3Ph03n1x
EMPD ports prevent Obverver from running remotely
Background
I have invested the past 2 days trying to connect my localhost machine to a production machine so I can use :observer.start() and see why it is consuming so much CPU.
At this point I am so desperate I decided to scratch everything and try another approach. This time I read the entirety of this discussion:
But to no avail.
My approach
This new approach is based on this tutorial (which originated the above mentioned post)
On 1st terminal in local machine:
ssh user@remote-host "epmd -names"
epmd: up and running on port 4369 with data:
name my_super_duper_app at port 42267
ssh -L 4369:localhost:4369 -L 42267:localhost:42267 remote-host
On 2nd terminal in local machine:
erl -name debug@127.0.0.1 -setcookie super_duper_cookie -remsh my_super_duper_app@remote-host
Whereremote-host is the public IP of the machine I want to connect to. The rest should be self explanatory.
Problem
After executing the command:
ssh -L 4369:localhost:4369 -L 42267:localhost:42267 remote-host
I am logged into the remote machine. This is good. What is not good is the message I see at the top:
bind [127.0.0.1]:4369: Address already in use
channel_setup_fwd_listener_tcpip: cannot listen to port: 4369
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.15.0-1026-gcp x86_64)
…
Of course the port 4369 is being used. It is the port where empd is running. What does it mean it cannot bind and listen to it? which machine do I have to kill to fix this?
As a consequence (or so I believe), when I run the command of the 2nd terminal I see this:
*** ERROR: Shell process terminated! (^G to start new job) ***
I am so confused. What am I missing?
Marked As Solved
rodrigues
yes
This is what I run locally to connect to the remote node (the app in production, for example). USR is a local name, COOKIE is the cookie in the remote node, and APP the user in remote host.
Yes, you run it local just like the other iex command. The difference is that I use this one to run observer. The other to use the IEx.
You will see the applications in the observer menu, “Nodes”. There you should be able to switch to the remote application you wanna observe.
Does it work now? ![]()
Also Liked
Fl4m3Ph03n1x
What dark magic is this?
The past 2 days of my life have been consumed by this. And you, out of nowhere, somehow managed to bring me a solution. I never thought this would work. To be honest, I quite don’t understand why, but who cares, it works !
Summary
To summarize (since I am sure I will forget about this 4 steps of hell) I am putting it here everything I did.
All steps in local machine:
-
get the ports from the remote server:
ssh remote-user@remote-ip "epmd -names"epmd: up and running on port 4369 with data:
name super_duper_app at port 43175
2: create a ssh tunel with the ports:
ssh remote-user@remote-ip -L4369:localhost:4369 -L43175:localhost:43175
-
On another terminal in your local machine, run a
iexterminal with the cookie the app in your remote server is using. Then connect to it and start observer:iex --name observer@127.0.0.1 --cookie super_duper_cookieNode.connect :"super_duper_app@127.0.0.1"
true
:observer.start
With observer started, select the machine from the Nodes menu.
rodrigues
I have this module that generate the instructions to connect that I’ve posted here, so you don’t need to get the epmd port every time. You need to replace some module variables, and beware @app is the same as remote user for us, not sure if it’s just gonna work for you.
Cheers
axelson
Something like
sudo lsof -t -i:4369
Should show you the process that has bound port 4369 locally. So you can run sudo lsof -t -i:4369 | xargs kill
If lsof isn’t working for you you could also try something like netstat -anl |grep 4369
axelson
Aren’t step 1 and 2 the same as in your original post? It sounds like the major issue you fixed is whatever was originally causing you this error after step 2:
bind [127.0.0.1]:4369: Address already in use
channel_setup_fwd_listener_tcpip: cannot listen to port: 4369
Fl4m3Ph03n1x
Following your suggestions to kill processes using the port also help immensely, that is true.
Thanks for the script and for the help. We will change some parts of the script but the overall idea is clear!








