Maxximiliann

Maxximiliann

Passing JSON from Elixir to Python with Ports

JSON is serialized into Erlang unicode and then passed to a Python process:

defmodule Sandbox do
  def foo(json_data) do
    port =
      :erlang.open_port(
        {:spawn, 'python3 -u lib/Python/Sandbox.py'},
        [:binary, packet: 4]
      )

    execute = fn data ->
      send(port, {self(), {:command, :erlang.term_to_binary({data})}})

      receive do
        {_, {:data, response}} -> :erlang.binary_to_term(response)
      after
        60000 ->
          {:error, :timeout}
      end
    end

    {_, results_string} = execute.(json_data)

    Print.text("sending JSON data, python result: #{results_string}")
  end
end

Example of what the Python process receives:

json_data = "<<131, 109, 0, 0, 0, 212, 123, 34, 97, 109, 111, 117, 110, 116, 95, 97, 115, 115, 101, 116, 34, 58, 34, 87, 65, 86, 69, 83, 34, 44, 34, 110, 111, 114, 109, 97, 108, 105, 122, 101, 100, 95, 97, 109, 111, 117, 110, 116, 34, 58, 52, 54, 54, 57, 55, 49, 51, 55, 56, 44, 34, 111, 114, 100, 101, 114, 95, 108, 105, 102, 101, 116, 105, 109, 101, 34, 58, 54, 49, 44, 34, 112, 114, 105, 99, 101, 34, 58, 48, 46, 48, 56, 56, 54, 51, 48, 48, 55, 44, 34, 112, 114, 105, 99, 101, 95, 97, 115, 115, 101, 116, 34, 58, 34, 72, 90, 107, 49, 109, 98, 102, 117, 74, 112, 109, 120, 85, 49, 70, 115, 52, 65, 88, 53, 77, 87, 76, 86, 89, 116, 99, 116, 115, 78, 99, 103, 54, 101, 50, 67, 54, 86, 75, 113, 75, 56, 122, 107, 34, 44, 34, 117, 105, 100, 34, 58, 34, 50, 48, 50, 48, 45, 49, 49, 45, 50, 51, 32, 50, 48, 58, 48, 51, 58, 53, 49, 46, 48, 48, 53, 50, 49, 53, 90, 116, 55, 71, 79, 79, 54, 49, 71, 89, 117, 112, 119, 78, 117, 122, 115, 109, 102, 52, 119, 83, 119, 34, 125>>"

To deserialize this into a Python dictionary the “<<” and “>>” characters need to be stripped off, all of the numbers need to be converted from strs to ints, and then the new list of ints needs to be iterated over to decode it into UTF-8 characters.

Is there a more efficient way to pass JSON from Elixir to Python such that it’s readily parsed by a Python process?

Most Liked

al2o3cr

al2o3cr

As far as the BEAM’s concerned, a port is just a process where you can send bytes with {:command, io_list} and get bytes back by receiving {:data, io_list}. If it’s easier for your program, you can just send those JSON bytes instead of encoding term -> binary.

The term conversion process is handy if you’re sending complicated structures that aren’t already encoded. Take a look at erlport, which already handles decoding ETF and has a Protocol abstraction for dispatching to multiple commands through a single port.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement