n1c0

n1c0

Hackney and client certificates => {:error, :closed}

Hi Guys,

I am trying to do a POST request via hackney (1.14.3).

The curl query is as follow:

curl -v \
--cert /Users/nico/projects/pfx/cert.pem \
--key /Users/nico/projects/pfx/key.pem \
--pass PASSWORD \
--insecure \
--header "Content-type: application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\"" \
--header "Accept: application/soap+xml; charset=utf-8" \
--data "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>" \
https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx

which gives the following results:

* Trying 200.198.239.181...
* TCP_NODELAY set
* Connected to www1.nfe.fazenda.gov.br (200.198.239.181) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; OU=ARSERPRO; OU=RFB e-Servidor A1; CN=www1.nfe.fazenda.gov.br
* start date: Sep 19 16:49:12 2018 GMT
* expire date: Sep 19 16:49:12 2019 GMT
* issuer: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; CN=Autoridade Certificadora do SERPRORFB SSL
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx HTTP/1.1
> Host: www1.nfe.fazenda.gov.br
> User-Agent: curl/7.54.0
> Content-type: application/soap+xml; charset=utf-8; action="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"
> Accept: application/soap+xml; charset=utf-8
> Content-Length: 790
> 
* upload completely sent off: 790 out of 790 bytes
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: application/soap+xml; charset=utf-8
< Server: Microsoft-IIS/8.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Thu, 13 Dec 2018 21:17:00 GMT
< Content-Length: 6380

now when i am trying to do the same with hackney in elixir (1.7.4), i am getting an {:error, :closed}:

url = "https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx"
xml_soap = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>"

headers = 
  [
    {"Content-type", "application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\""},
    {"Accept", "application/soap+xml; charset=utf-8"}
  ]

opts = [
  :insecure,
  recv_timeout: 30_000,
  ssl_options: [
    certfile: "/Users/nico/projects/pfx/cert.pem",
    keyfile: "/Users/nico/projects/pfx/key.pem",
    password: 'PASSWORD',
    versions: [:'tlsv1.2']
  ]
]

leading to:

iex(8)> :hackney.request(:post, url, headers, xml_soap, opts)
{:error, :closed}

I guess the issue must be in the SSL negotiation, but don’t know how to debug this…any idea?

thanks!

Marked As Solved

voltone

voltone

Fix is now available as part of patch package 21.2.1:
http://erlang.org/pipermail/erlang-questions/2018-December/096940.html

Also Liked

voltone

voltone

Another update: I was able to reproduce the issue against OpenSSL. It turns out it only happens if the client sends data prior to the server’s renegotiation request. The test cases included in OTP for renegotiation between an Erlang client and OpenSSL server perform renegotiation prior to any data exchange.

Based on all the information gathered I opened an issue in the Erlang bug tracker:
https://bugs.erlang.org/browse/ERL-803

If anyone wants to reproduce the issue, first launch an OpenSSL test server in one terminal window (you need a certificate and key, for example a self-signed certificate):

$ openssl s_server -cert server.pem -key server_key.pem -msg
Using default temp DH parameters
ACCEPT

In another terminal window, start an Erlang or Elixir shell and connect to the test server and send some data:

Erlang/OTP 21 [erts-10.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :ssl.start
:ok
iex(2)> {:ok, s} = :ssl.connect('localhost', 4433, [])
{:ok,
 {:sslsocket, {:gen_tcp, #Port<0.7>, :tls_connection, :undefined},
  [#PID<0.128.0>, #PID<0.127.0>]}}
iex(3)> :ssl.send(s, "Hello!")   
:ok
iex(4)> 

At this point, type ‘R’ and hit Enter in the OpenSSL window.

voltone

voltone

Here’s what I’ve found so far:

The issue is not Hackney specific: the same thing happens when connecting with Erlang’s ssl API directly.

The issue is not related to the client certificate: the renegotiation only gets as far as the client sending the ClientHello, and the server immediately closes the connection. The server never gets to the point where it asks for the client certificate.

This appears to be a regression in renegotiation handling in recent OTP versions: when I tried the connection on OTP 21.0.9 the renegotiation succeeded (I don’t have a client certificate, but at least I do get the 403 response). On 21.1 and 21.2 it does not work.

voltone

voltone

I don’t have a valid client certificate, so I can’t fully replicate your scenario, but when I connect to that address with Hackney I see the same response. The important thing to note is that the connection gets closed after the TLS handshake completes.

Does the ‘curl’ command actually show a response from the server? The output you quoted only shows curl trying to send a request.

voltone

voltone

Ah, sorry, I just noticed the scrollbar alongside your original curl output, so it wasn’t truncated after all.

The issue is this: in the initial handshake the server does not send a client certificate request, and so the client does not send one (both curl and Hackney). Once the server has received the HTTP request it presumably decides that this particular request requires a certificate-based authentication and it attempts to renegotiate the session. In this second handshake it does request a client certificate:

If the renegotiation succeeds and the client presents a valid certificate, the server sends its response.

With Hackney, the renegotiation fails and the connection is closed.

This is an interesting scenario, I will look into it some more and report back…

n1c0

n1c0

alright thanks for the feedback! I have downgraded to OTP 21.0.9 and run the command again, now getting the 200 response code & the content like in curl:

{:ok, 200,
 [
   {"Cache-Control", "private, max-age=0"},
   {"Content-Type", "application/soap+xml; charset=utf-8"},
   {"Server", "Microsoft-IIS/8.5"},
   {"X-AspNet-Version", "4.0.30319"},
   {"X-Powered-By", "ASP.NET"},
   {"Date", "Sat, 15 Dec 2018 16:27:39 GMT"},
   {"Content-Length", "6380"}
 ], #Reference<0.3983044557.4188012545.22511>}

Do you mind raising an issue for OTP 21.2 ? you must have more details than myself to explain what’s going on ?

Thanks!

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New

Other popular topics Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement