CharlesO

CharlesO

Need help with basic crypto in Erlang/Elixir: Decryption Error: Failed to decrypt:

I have been battling with some code:

I have 2 variants I have been testing:

  def ds4 do
    try do
      private_key_pem = File.read!("priv/flows/private_unencrypted.pem")

      # Decode PEM key explicitly
      private_key_entry = :public_key.pem_decode(private_key_pem) |> hd()
      IO.inspect(private_key_entry, label: "Private Key Entry")

      # Correctly decode the private key
      case private_key_entry do
        {:RSAPrivateKey, rsa_key} ->
          IO.inspect(rsa_key, label: "RSA Private Key Structure")

          # Decrypt AES key
          encrypted_aes_key =
            Base.decode64!(
              "i/j13zUEy37M00eFJYchyIyk+HIlmLs8X6gmJLO9UfiljUmQCX1kYVPYfVRbS+5moWKtkIc0K/KG59CObmT8sMbhSXFEbnCKL7bbLka80NC5lqoill4LKrAtOaWJ/Zxbw7YjWS1+zinkIZZCbJ0OGZjH24XVgx3pVx5PYK53JDSYt6kUo1Kt15uc5M7zne1/T46c5YCw2BnSGDvXX+74W7T8xo+dTl6krOzFGuxNXFOYtG3coFK3Ad4eteMIhlsxpoRjFggfIWv9VuA7TT9AOQ9mdF2juN98Hu0hm3kSi5IHHGagTq+UwrspnMl76kw+GZXinjQNSI2ZV8yYYHxQ=="
            )

          decrypted_aes_key =
            :crypto.private_decrypt(:rsa, encrypted_aes_key, rsa_key,
              rsa_padding: :rsa_pkcs1_oaep_padding,
              oaep_hash: :sha256
            )

          IO.inspect(decrypted_aes_key, label: "Decrypted AES Key")
          decrypted_aes_key

        other ->
          IO.inspect(other, label: "Unexpected Private Key Structure")
          {:error, :unexpected_private_key_structure}
      end
    rescue
      e ->
        IO.inspect("Failed to decrypt: #{inspect({e, __STACKTRACE__})}",
          label: "Decryption Error"
        )

        {:error, e}
    end
  end

  def ds5 do
    try do
      private_key_pem = File.read!("priv/flows/private_unencrypted.pem")

      # Remove PEM headers and decode base64
      private_key_der =
        private_key_pem
        |> String.replace(~r/-----BEGIN RSA PRIVATE KEY-----/, "")
        |> String.replace(~r/-----END RSA PRIVATE KEY-----/, "")
        |> Base.decode64!()

      # Decode DER-encoded private key
      {:Ok, private_key, _} = :public_key.der_decode(private_key_der, :RSAPrivateKey)

      IO.inspect(private_key, label: "RSA Private Key Structure")

      m = %{
        encrypted_aes_key:
          "i/j13zUEy37M00eFJYchyIyk+HIlmLs8X6gmJLO9UfiljUmQCX1kYVPYfVRbS+5moWKtkIc0K/KG59CObmT8sMbhSXFEbnCKL7bbLka80NC5lqoill4LKrAtOaWJ/Zxbw7YjWS1+zinkIZZCbJ0OGZjH24XVgx3pVx5PYK53JDSYt6T6kUo1Kt15uc5M7zne1/T46c5YCw2BnSGDvXX+74W7T8xo+dTl6krOzFGuxNXFOYtG3coFK3Ad4eteMIhlsxpoRjFggfIWv9VuA7TT9AOQ9mdF2juN98Hu0hm3kSi5IHHGagTq+UwrspnMl76kw+GZXinjQNSI2ZV8yYYHxQ==",
        encrypted_flow_data:
          "esYLI3l/ZREnkg3EDoipH/TeF8fL6goV1GELcV/Jv+WWVkB/5JcJ5NRoFzfnDZrTKA==",
        initial_vector: "65SPDGDmgHlG4MiRzy5abQ=="
      }

      # Decrypt AES key
      encrypted_aes_key = Base.decode64!(m.encrypted_aes_key)

      decrypted_aes_key =
        :crypto.private_decrypt(:rsa, encrypted_aes_key, private_key,
          rsa_padding: :rsa_pkcs1_oaep_padding,
          oaep_hash: :sha256
        )

      IO.inspect(decrypted_aes_key, label: "Decrypted AES Key")
      decrypted_aes_key
    rescue
      e ->
        IO.inspect("Failed to decrypt: #{inspect({e, __STACKTRACE__})}",
          label: "Decryption Error"
        )

        {:error, e}
    end
  end

I’m at a loss for the errors I keep getting:

 FL.ds4
Private Key Entry: {:PrivateKeyInfo,
 <<48, 130, 4, 190, 2, 1, 0, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1,
   5, 0, 4, 130, 4, 168, 48, 130, 4, 164, 2, 1, 0, 2, 130, 1, 1, 0, 162, 144,
   53, 31, 174, 226, 155, 127, 205, 227, ...>>, :not_encrypted}
Unexpected Private Key Structure: {:PrivateKeyInfo,
 <<48, 130, 4, 190, 2, 1, 0, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1,
   5, 0, 4, 130, 4, 168, 48, 130, 4, 164, 2, 1, 0, 2, 130, 1, 1, 0, 162, 144,
   53, 31, 174, 226, 155, 127, 205, 227, ...>>, :not_encrypted}
{:error, :unexpected_private_key_structure}
15:36:33.174 DB "get_inactive_agents"
15:36:33.193 DB "summarize_1"
15:36:33.227 DB "summarize_2"
15:36:33.423 {:sse, "stats-refresh"}
 FL.ds5
Decryption Error: "Failed to decrypt: {%ArgumentError{message: \"non-alphabet character found: \\\"-\\\" (byte 45)\"}, [{Base, :bad_character!, 1, [file: ~c\"lib/base.ex\", line: 137]}, {Base, :\"-decode64base!/2-lbc$^0/2-0-\", 2, [file: ~c\"lib/base.ex\", line: 642]}, {Base, :decode64base!, 2, [file: ~c\"lib/base.ex\", line: 640]}, {FL, :ds5, 0, [file: ~c\"lib/chatflow/fl.ex\", line: 222]}, {:elixir, :eval_external_handler, 3, [file: ~c\"src/elixir.erl\", line: 386]}, {:erl_eval, :do_apply, 7, [file: ~c\"erl_eval.erl\", line: 750]}, {:elixir, :eval_forms, 4, [file: ~c\"src/elixir.erl\", line: 364]}, {Module.ParallelChecker, :verify, 1, [file: ~c\"lib/module/parallel_checker.ex\", line: 112]}]}"
{:error, %ArgumentError{message: "non-alphabet character found: \"-\" (byte 45)"}}

Any help and insight would be appreciated :palms_up_together:

Also, i have updated the question to include the requirements from META:

Implementing Endpoints for Flows - WhatsApp Flows (facebook.com)

Request Decryption and Encryption
The incoming request body is encrypted, you need to decrypt it first, then you need to encrypt the server response before returning it to the client.

You can find code examples of decryption/encryption in various programming languages in the Code Examples section.

For data_api_version “3.0” you should follow below instructions to decrypt request payload:

extract payload encryption key from encrypted_aes_key field:
decode base64-encoded field content to byte array;
decrypt resulting byte array with the private key corresponding to the uploaded public key using RSA/ECB/OAEPWithSHA-256AndMGF1Padding algorithm with SHA256 as a hash function for MGF1;
as a result, you’ll get a 128-bit payload encryption key.
decrypt request payload from encrypted_flow_data field:
decode base64-encoded field content to get encrypted byte array;
decrypt encrypted byte array using AES-GCM algorithm, payload encryption key and initialization vector passed in initial_vector field (which is base64-encoded as well and should be decoded first). Note that the 128-bit authentication tag for the AES-GCM algorithm is appended to the end of the encrypted array.
result of above step is UTF-8 encoded clear request payload.
For data_api_version “3.0” you should follow below instructions to encrypt the response:

encode response payload string to response byte array using UTF-8;
prepare initialization vector for response encryption by inverting all bits of the initialization vector used for request payload encryption;
encrypt response byte array using AES-GCM algorithm with the following parameters:
secret key - payload encryption key from request decryption stage;
initialization vector for response encryption from above step;
empty AAD (additional authentication data) - many libraries assume this by default, check the documentation of the library in use;
128-bit (16 byte) length for authentication tag - many libraries assume this by default, check the documentation of the library in use;
append authentication tag generated during encryption to the end of the encryption result;
encode the whole output as base64 string and send it in the HTTP response body as plain text.

Marked As Solved

CharlesO

CharlesO

I eventually used NodeJS : {:nodejs, "~> 2.0"} to call the encryption.js file provided in Meta’s examples.

works like a charm!

Also Liked

cmkarlsson

cmkarlsson

Assuming this: Implementing Endpoints for Flows - WhatsApp Flows - Dokumentation - Meta for Developers is the original instructions, it says:

using RSA/ECB/OAEPWithSHA-256AndMGF1Padding algorithm with SHA256 as a hash function for MGF1; being the important part in this case.

private_decrypt defaults to RSA PCKS1 padding (which should no longer be used as it is not secure)

I’m not sure erlang crypto module supports RSA/ECB/OAEPWIthSHA-256andMGF1Padding mode.

You can try with:
:public_key.decrypt_private(cipher, key, [rsa_padding: :rsa_pkcs1_oaep_padding]) but I am pretty sure that uses SHA1 instead of SHA256.

In addition public_key:decrypt_private2/3 and the equivalent in crypto module are deprecated. I have not found what they are replaced with though. It seems that the original intent was not for encrypt/decrypt but for signing and verification. The docs suggest that they have been replaced with :public_key.sign/verify which will not help you much.

Might give you additional options to try.

We used erlang for a crypto heavy application but we used a port (Ports — Erlang System Documentation v27.0.1) to do all the asymmetric crypto operations rather than the built-in and it performed very well with stable latency compared to our java and golang equivalent apps.

voltone

voltone

If you want to use :public_key to decode PEM-encoded keys you need to use :public_key.pem_entry_decode/1 on the data returned by :public_key.pem_decode/1.

If you want to extract the DER entry from the PEM yourself you’re going to have to do a bit more work to remove extra whitespace (e.g. newlines).

P.S. you can save yourself a lot of trouble by using x509:slight_smile:

D4no0

D4no0

Cryptography stuff from scratch is very hard, even if you know the theory, you might still fail to implement it correctly.

If there isn’t a ready solution available, I would always consider first using Ports or another mechanism to borrow the implementation from another ecosystem, as I’ve fallen myself into this trap of spending countless days debugging myself.

zookzook

zookzook

With the help of rustler you can use Rust to decrypt the AES key. I used the node.js example and was able to decode the encrypted AES key using Rust/Rustler:

use base64::prelude::*;
use rustler::{Env, NifResult};
use rsa::{RsaPrivateKey, Oaep, sha2::Sha256};
use rsa::pkcs8::DecodePrivateKey;
use rustler::types::binary::{Binary, OwnedBinary};

#[rustler::nif(schedule = "DirtyCpu")]
fn decrypt<'a>(env: Env<'a>, private_file_path: String, cipher_text: String) -> NifResult<Binary<'a>> {

    let padding = Oaep::new::<Sha256>();
    let content = std::fs::read_to_string(&private_file_path).unwrap();
    let private_key = RsaPrivateKey::from_pkcs8_pem(&content).unwrap();
    let enc_data = BASE64_STANDARD.decode(cipher_text).unwrap();
    let dec_data = private_key.decrypt(padding, &enc_data).unwrap();
    let mut result: OwnedBinary = OwnedBinary::new(dec_data.len()).unwrap();
    result.as_mut_slice().copy_from_slice(&dec_data);
    Ok(Binary::from_owned(result, env))
}

rustler::init!("Elixir.RCrypto");

Here is the output:

iex [09:40 :: 2] > RCrypto.decrypt("./private_unencrypted.pem", "hiHNE0AFZI6pn/Sbo0yPmbm37NQxuCc1rDxotfkeOHjjCO9BEVMtLQKfOE6w2/VQwISqJpAE0D9FlScvEdMgNjtbo5rNRutHFVF5xnvLsXyV228dxt5m8h+6k1LApk5TLOj97+fi4iiYq5Tr3n1E+4jpY0TTl30s/k0rhciOIy7HwFpaRWVSiOZFqx+PTYrJYcRot/DoArd4DLAdlbeoHcyoB4R0LV4DJRUmKwrcP/7J3njC1PpetGjgGiy5g/e/dKoxmcjjyhSwS7QeoCxBAF4ErHWHFfNOeNyCdwLmOtlLIa55nwGQ0Jyn0xp4WQj2xJqoHvCTcqbAxRtCJ5b5zA==")
"<some-key-data>"
iex [09:40 :: 3] >

Of course, the code is just a quick demo. Loading the key for each call should not be used at all.

al2o3cr

al2o3cr

Using those steps, I was able to recreate your observations and make some additional ones:

  • the generated private_unencrypted.pem file has a -----BEGIN PRIVATE KEY----- header, not the one that the code in ds5 was expecting to trim off
  • the RSAPrivateKey details are available by also using pem_entry_decode as @voltone suggested:
    File.read!("path_to/private_unencrypted.pem")
    |> :public_key.pem_decode()
    |> hd()
    |> :public_key.pem_entry_decode()
    
    # result - 11-element tuple
    {:RSAPrivateKey, :"two-prime", <giant number>, 65537, <giant number>, <giant number>, <giant number>, ..., :asn1_NOVALUE}
    

Where Next?

Popular in Questions Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement