chulkilee

chulkilee

Strange httpc/hackeny behavior on ssl verify

I need to call HTTP API verifying SSL or not. However, I found both httpc and hackney behave strange - ssl option stay across different functions.

See this code:

bad_url1 = "https://expired.badssl.com/"
bad_url2 = "https://wrong.host.badssl.com/"

# Testing httpc

# httpc by default does not check ssl
{:ok, _} = :httpc.request(:get, {to_charlist(bad_url1), []}, [], [])

# verify ssl
{:error, _} =
  :httpc.request(
    :get,
    {to_charlist(bad_url1), []},
    [ssl: [verify: :verify_peer, cacertfile: :certifi.cacertfile()]],
    []
  )

# no verify again
{:ok, _} = :httpc.request(:get, {to_charlist(bad_url1), []}, [ssl: [verify: :verify_none]], [])

# verify ssl... DOES NOT work!
{:ok._()} =
  :httpc.request(
    :get,
    {to_charlist(bad_url1), []},
    [ssl: [verify: :verify_peer, cacertfile: :certifi.cacertfile()]],
    []
  )

# Testing hackney
Application.ensure_all_started(:hackney)

{:error, _} = :hackney.request(:get, bad_url1, [], [], [])
{:error, _} = :hackney.request(:get, bad_url2, [], [], [])
# => :error, as expected

{:ok, 200, _headers, ref} = :hackney.request(:get, bad_url1, [], [], [:insecure])

# before reading the body, other requests without insecure fail
{:error, _} = :hackney.request(:get, bad_url1, [], [], [])

# once the body is read..
{:ok, _body} = :hackney.body(ref)
# => :ok, as expected

{:ok, 200, _headers, ref} = :hackney.request(:get, bad_url1, [], [], [])
{:ok, _body} = :hackney.body(ref)
# WHAT? as insecure is missing, this should fail due to ssl error!

At first I thought hackney keeps some options in pool (I created https://github.com/benoitc/hackney/issues/570 ) - but apparently httpc has the same issue.

Anyone had this issue?

Marked As Solved

LostKobrakai

LostKobrakai

You might want to watch this:

The important point for your issue. Erlang will cache tls verifications.

Also Liked

voltone

voltone

In most applications, the decision whether a given TLS peer is trusted is determined by a global policy: either all is well (known issuer, not expired, correct hostname, etc.) or something is wrong, the decision does not depend on any kind of context. TLS session resumption and HTTP keep-alive are performance optimisations that work very well as long as this assumption holds.

In my talk I mentioned testing (automatic, or manual) as a scenario where these features can lead to unexpected results, and I pointed out how to disable them to get predictable results. But I do believe most users will want to leave them enabled in production.

In the scenario you describe, where some other process establishes unverified TLS connections, the real solution is of course to get that other process fixed.

If this is not possible, you’d have to take control of some of the lower layers yourself, to keep the secure and insecure parts of your application separate. At the HTTP layer you may be able to set up different connection pools (in :httpc, define a custom ‘profile’ and use it instead of the default one; the profile is the ‘reference’ you mentioned, it’s just that there is an implicit default). At the TLS layer you’d have to disable connection reuse, or manage it yourself using the client_reuse_sessions: :save and client_reuse_session options.

Mint only helps in that it does not use a connection pool, but at the TLS layer it uses the standard :ssl application, with the same connection reuse behaviour as other HTTP clients.

chulkilee

chulkilee

Thanks!

I found hackney / httpc reuses http connection across different function calls, even they do not pass same references. This is bad behavior - as your SSL option might not be honored if other process calls the “bad cert” URL with verify_none. Wow.

I really hope mint get adopted. No state MUST be shared without explicit passing references!

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement