n1c0

n1c0

Hackney and client certificates => {:error, :closed}

Hi Guys,

I am trying to do a POST request via hackney (1.14.3).

The curl query is as follow:

curl -v \
--cert /Users/nico/projects/pfx/cert.pem \
--key /Users/nico/projects/pfx/key.pem \
--pass PASSWORD \
--insecure \
--header "Content-type: application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\"" \
--header "Accept: application/soap+xml; charset=utf-8" \
--data "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>" \
https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx

which gives the following results:

* Trying 200.198.239.181...
* TCP_NODELAY set
* Connected to www1.nfe.fazenda.gov.br (200.198.239.181) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; OU=ARSERPRO; OU=RFB e-Servidor A1; CN=www1.nfe.fazenda.gov.br
* start date: Sep 19 16:49:12 2018 GMT
* expire date: Sep 19 16:49:12 2019 GMT
* issuer: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; CN=Autoridade Certificadora do SERPRORFB SSL
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx HTTP/1.1
> Host: www1.nfe.fazenda.gov.br
> User-Agent: curl/7.54.0
> Content-type: application/soap+xml; charset=utf-8; action="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"
> Accept: application/soap+xml; charset=utf-8
> Content-Length: 790
> 
* upload completely sent off: 790 out of 790 bytes
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: application/soap+xml; charset=utf-8
< Server: Microsoft-IIS/8.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Thu, 13 Dec 2018 21:17:00 GMT
< Content-Length: 6380

now when i am trying to do the same with hackney in elixir (1.7.4), i am getting an {:error, :closed}:

url = "https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx"
xml_soap = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>"

headers = 
  [
    {"Content-type", "application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\""},
    {"Accept", "application/soap+xml; charset=utf-8"}
  ]

opts = [
  :insecure,
  recv_timeout: 30_000,
  ssl_options: [
    certfile: "/Users/nico/projects/pfx/cert.pem",
    keyfile: "/Users/nico/projects/pfx/key.pem",
    password: 'PASSWORD',
    versions: [:'tlsv1.2']
  ]
]

leading to:

iex(8)> :hackney.request(:post, url, headers, xml_soap, opts)
{:error, :closed}

I guess the issue must be in the SSL negotiation, but don’t know how to debug this…any idea?

thanks!

Marked As Solved

voltone

voltone

Fix is now available as part of patch package 21.2.1:
http://erlang.org/pipermail/erlang-questions/2018-December/096940.html

Also Liked

voltone

voltone

Another update: I was able to reproduce the issue against OpenSSL. It turns out it only happens if the client sends data prior to the server’s renegotiation request. The test cases included in OTP for renegotiation between an Erlang client and OpenSSL server perform renegotiation prior to any data exchange.

Based on all the information gathered I opened an issue in the Erlang bug tracker:
https://bugs.erlang.org/browse/ERL-803

If anyone wants to reproduce the issue, first launch an OpenSSL test server in one terminal window (you need a certificate and key, for example a self-signed certificate):

$ openssl s_server -cert server.pem -key server_key.pem -msg
Using default temp DH parameters
ACCEPT

In another terminal window, start an Erlang or Elixir shell and connect to the test server and send some data:

Erlang/OTP 21 [erts-10.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :ssl.start
:ok
iex(2)> {:ok, s} = :ssl.connect('localhost', 4433, [])
{:ok,
 {:sslsocket, {:gen_tcp, #Port<0.7>, :tls_connection, :undefined},
  [#PID<0.128.0>, #PID<0.127.0>]}}
iex(3)> :ssl.send(s, "Hello!")   
:ok
iex(4)> 

At this point, type ‘R’ and hit Enter in the OpenSSL window.

voltone

voltone

Here’s what I’ve found so far:

The issue is not Hackney specific: the same thing happens when connecting with Erlang’s ssl API directly.

The issue is not related to the client certificate: the renegotiation only gets as far as the client sending the ClientHello, and the server immediately closes the connection. The server never gets to the point where it asks for the client certificate.

This appears to be a regression in renegotiation handling in recent OTP versions: when I tried the connection on OTP 21.0.9 the renegotiation succeeded (I don’t have a client certificate, but at least I do get the 403 response). On 21.1 and 21.2 it does not work.

voltone

voltone

I don’t have a valid client certificate, so I can’t fully replicate your scenario, but when I connect to that address with Hackney I see the same response. The important thing to note is that the connection gets closed after the TLS handshake completes.

Does the ‘curl’ command actually show a response from the server? The output you quoted only shows curl trying to send a request.

voltone

voltone

Ah, sorry, I just noticed the scrollbar alongside your original curl output, so it wasn’t truncated after all.

The issue is this: in the initial handshake the server does not send a client certificate request, and so the client does not send one (both curl and Hackney). Once the server has received the HTTP request it presumably decides that this particular request requires a certificate-based authentication and it attempts to renegotiate the session. In this second handshake it does request a client certificate:

If the renegotiation succeeds and the client presents a valid certificate, the server sends its response.

With Hackney, the renegotiation fails and the connection is closed.

This is an interesting scenario, I will look into it some more and report back…

n1c0

n1c0

alright thanks for the feedback! I have downgraded to OTP 21.0.9 and run the command again, now getting the 200 response code & the content like in curl:

{:ok, 200,
 [
   {"Cache-Control", "private, max-age=0"},
   {"Content-Type", "application/soap+xml; charset=utf-8"},
   {"Server", "Microsoft-IIS/8.5"},
   {"X-AspNet-Version", "4.0.30319"},
   {"X-Powered-By", "ASP.NET"},
   {"Date", "Sat, 15 Dec 2018 16:27:39 GMT"},
   {"Content-Length", "6380"}
 ], #Reference<0.3983044557.4188012545.22511>}

Do you mind raising an issue for OTP 21.2 ? you must have more details than myself to explain what’s going on ?

Thanks!

Where Next?

Popular in Questions Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

Other popular topics Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New

We're in Beta

About us Mission Statement