thousandsofthem
Issue with http clients and some https certificates
Please help me debug the issue with https certificates, need some ideas what to check next
When fetching data using http clients, some https:// urls returning “Certificate Expired”, despite being perfectly valid, e.g. https://investing.com/, https://api.amplitude.com/ .
It looks like it doesn’t depend on erlang version, elixir version, http library version.
Base system is (a) OSX 10.14.5 (b) Docker, Alpine Linux
test app #1
elixir 1.9.4
erlang 22.3.4.1 (also 22.1.8)
mojito 0.6.4
mint 1.1.0
httpoison 1.5.0
hackney 1.15.2 (also 1.16.0)
iex(8)> Mojito.request(:get, "https://api.amplitude.com/")
[info] TLS :client: In state :certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired
{:error,
%Mojito.Error{
message: nil,
reason: %Mint.TransportError{
reason: {:tls_alert,
{:certificate_expired,
'TLS client: In state certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired\n'}}
}
}}
iex(9)> HTTPoison.get("https://api.amplitude.com/")
[info] TLS :client: In state :certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired
{:error,
%HTTPoison.Error{
id: nil,
reason: {:tls_alert,
{:certificate_expired,
'TLS client: In state certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired\n'}}
}}
iex(10)> opts = [ssl: [cacertfile: :certifi.cacertfile(), verify: :verify_peer]]
iex(11)> HTTPoison.get("https://api.amplitude.com/", [], opts)
# same error - CLIENT ALERT: Fatal - Certificate Expired
test app #2
elixir 1.3.4
erlang 19.3.6.5
hackney 1.15.0
httpoison 1.0.0
iex(6)> HTTPoison.get("https://api.amplitude.com/")
[error] SSL: :certify: ssl_handshake.erl:1609:Fatal error: certificate expired
{:error, %HTTPoison.Error{id: nil, reason: {:tls_alert, 'certificate expired'}}}
Most Liked
kip
According to curl the certificate has expired for at least https://api.amplitude.com/. There was an issue over the weekend with an expired root certificate from Comodo/Sectigo CA for https://openexchangerates.org as well. Perhaps these sites are using the same certificate chain. Have you checked their support site?
kip@Kips-iMac-Pro cldr_dates_times % curl "https://api.amplitude.com/"
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
voltone
Exactly. The :ssl application treats the certificates sent by the server as a linear, unbreakable chain, ignoring any self-signed certificates that may have been sent because of misconfiguration. (Recent versions do allow for servers that send the chain out-of-order). Chain validation then tries to find a root certificate in the trust store that matches the issuer of the last certificate in that chain (furthest from the end-certificate).
This approach is problematic with cross-signed certificates, as this example shows. Most other implementation treat the intermediate CA certificates sent by the server as a pool from which to draw missing elements in the chain, and they try multiple paths until a valid one is found.
The most reliable way to resolve this is to reconfigure the server to not send the cross-signing intermediate CA once the issuing root CA is invalidated. If the CA trust store used on the Erlang/Elixir side includes the (self-signed) equivalent root CA, then verification of the shorter chain will succeed.
There is no easy way to fix this on the client side in a generic way. A workaround for specific cases should be possible by passing a ‘partial chain fun’.
kip
I ran the check on https://ssllabs.com which I think is well respected. It shows the intermediate and root certificates sent by the server as expired.
However, it also shows that there are three paths to establishing trust, one that includes the expired certificates and two that do not. As I understand it from some reading, the :ssl app is unable to resolve alternative trust paths and therefore will fail to validate the peer in more situations than other clients. (I know that is not precise, mostly because I only understand a very small amount of this topic).
It also suggests why you are seeing different outcomes from different clients.
al2o3cr
This is an issue with older OpenSSL versions - we’re running 1.0.2g in production (
) and encountered the same thing. Erlang depends on OpenSSL via libcrypto.so (at least on Ubuntu)
Long writeup here: https://calnetweb.berkeley.edu/calnet-technologists/incommon-sectigo-certificate-service/addtrust-external-root-expiration-may-2020
voltone
Actually it seems removing the expired root CA from the CA trust store seems to work, at least for some endpoints and some OTP versions. I don’t have time to dig into this further right now, but that’s something you may want to try…
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Courses that'll move you from confusion to "Aha, now I get it!"
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"