RooSoft

RooSoft

Async TCP subscription through a library, without GenServer

We have a lib that exports two functions:

  • Lib.connect(ip, port) simply calls :gen_tcp.connect(to_charlist(ip), port, [:binary])
  • Lib.subscribe(socket) uses :gen_tcp.send(socket, "init") and expects the server to start sending various messages over time. That data is rather cryptic and needs to be processed by the lib so that the lib’s client can process it easily.

Let’s say that the lib’s calls were encapsulated in a GenServer, we would have an easy solution: create a handle_info/2 function, process the data and send it to a pub/sub.

Now, we think a GenServer would cause a whole lot of unnecessary complexity, so we’re trying to avoid it.

What’s the best way to make sure the lib gets the messages and sends the processed version to the clients back without blocking any of the client’s process?

Most Liked

hst337

hst337

From what you wrote I can see that you want to different libraries:

  1. Lib just for Electrum RPC protocol. You want it process-less
  2. User-friendly Electrum interface with more high-level operations

So I can see 3 layers in this program

  1. TCP connection pool management. To connect to peers and communicate with them.
  2. Electrum JSONRPC protocol
  3. High-level API for the user (without scripthash, etc)

The connection management

Alright, I’ve read the python library and I can see that these RPC calls operate over some state (SessionManager class holds global state of all sessions, and connection to a PeerManager which maintains TCP connection to max 8 peers). And you’ll need to handle this state somehow. In erlang and elixir world we use processes for that.

And there are two options for the process. You can make it a global singleton, or a local process. If you want to maintain exactly the same API as in ElectrumX, you’ll need to create a named singleton process. But I’d suggest making library more flexible and isolating process names from it. So, I’d go with SessionManager as a Supervisor module which starts DynamicSupervisor, Registry and starts ElectrumConnection processes under the DynamicSupervisor. This SessionManager can take a name during the start. Each ElectrumConnection holds connection to one peer

And the module would provide these functions

  • SessionManager.add_peer(session_manager_pid, peer_opts) to add a peer. This will start the new connection process under the dynamicsup, name it in the registry.
  • SessionManager.connect_peer(), SessionManager.disconnect_peer, etc. for peer management
  • SessionManager.pick_connection() – randomly picks one connection (you can chose different strategy)
  • SessionManager.execute(connection, operation, reply_to) – this one just passes an Electrum operation to the connection process which should send the result to reply_to pid

This is basically how the process pool operates. I can suggest you to take a look at libraries like poolboy

Electrum JSONRPC

This part of the system is a thing that you wanted to be a process-less library for just parsing and handling responses. The problem with your approach was that you wanted it to be somehow coupled with tcp connection, which is a bad way to go.

I’d suggest spliting this into two things:

  1. Converting elixir structure for operation into the binary with jsonrpc request/response in it. This is a pretty straightforward thing to write. You can use any data validation library you prefer, but I don’t see any problem in hand-rolling this.

  2. Sending and receiving data over TCP as a JSONRPC spec defines. This was already done in libraries like this: GitHub - fanduel-oss/jsonrpc2-elixir: JSON-RPC 2.0 for Elixir

Highlevel user interface

Just create something like Electrum.subscribe which internally calls already started global SessionManager and awaits the response

RooSoft

RooSoft

The data comes from an Electrum RPC endpoint, which is what most Bitcoin wallets use to transact.

For instance, the blockchain.scripthash.subscribe is mostly what I am talking about. So, the data lib I was talking about would connect to this endpoint and make sure it subscribes to a specific scripthash.

But what’s a scripthash???

It’s jargon that Bitcoin wallet devs might not want to deal with. In other words, it’s another representation of an address. This call, in short, returns transactions relative to a specific Bitcoin address as they happen.

The goal here is to not only enable devs to connect to this thing, but make it easy and relatable. Ideally, they’d just initiate a connection, passing an IP and a port and then subscribe to a specific address. This is, of course, in the Bitcoin layer lib’s language. Under the hood, it would handle the electrum lib and isolate the dev from all the scripthash gibberish we initially talked about. Not to mention there’s some data conversion involved in the process.

Why not combine both in a single lib, you might ask… because the raw Electrum lib

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New

We're in Beta

About us Mission Statement