brundozer

HTTPS on ECS with docker, Let's Encrypt with no http server

Hi,

I’ve been desperately trying to make my phoenix app work with HTTPS and despite the huge amount of resources available on this forum and other websites about this, I haven’t yet been able to do so.

The website works just fine in HTTP, on port 80, but I keep getting an ERR_CONNECTION_RESET when trying to access it on HTTPS. I am not using any reverse proxy, just Cowboy’s http server.

Here is what I get when running curl https://www.allremotedevjobs.com/ --verbose:

*   Trying 3.104.92.194...
* TCP_NODELAY set
* Connected to www.allremotedevjobs.com (3.104.92.194) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.allremotedevjobs.com:443 
* stopped the pause stream!
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.allremotedevjobs.com:443

I have no idea what this error means and it seems quite hard to find anything meaningful from this but it seems obvious that it is linked to the ERR_CONNECTION_RESET in my browser.

I have a phoenix 1.5.1 app running on ECS/EC2 with docker, released with distillery. The way it works is pretty simple: my release is packaged in a docker container and pushed to AWS. My release docker-compose file is mapping the following ports:

ports:
   - '80:4000'
   - '443:5000'

To have the simplest possible example, I have decided to not use any environment variable so my http and https config are in prod.exs instead of releases.exs. Here is the part that I think is relevant to my problem:

  http: [
    port: 4000
  ],
  https: [
    port: 5000,
    keyfile: "/etc/letsencrypt/live/www.allremotedevjobs.com/privkey.pem",
    certfile: "/etc/letsencrypt/live/www.allremotedevjobs.com/cert.pem",
    cacertfile: "/etc/letsencrypt/live/www.allremotedevjobs.com/chain.pem"
  ]

In AWS, my security groups allows access to ports 80 and 443.

I have generated my certificate on my EC2 instance where the docker container runs using certbot, with certbot certonly --standalone. The certificates are successfully generated, and it seems that my phoenix app can read them as it doesn’t complain that the any of the files is not existing or cannot be read when deploying. I am using the full path as shown in the config to reference the cert.pem, privkey.pem and chain.pem files. I thought that I could just used fullchain.pem for the certfile and omit the cacertfile, but as all examples I have seen seem to use the 3 files, I’ve decided to do like everybody else. These 3 files are linked to the same path inside my container using volumes in docker-compose:

volumes:
  - /etc/letsencrypt/live/www.allremotedevjobs.com/privkey.pem:/etc/letsencrypt/live/www.allremotedevjobs.com/privkey.pem
  - /etc/letsencrypt/live/www.allremotedevjobs.com/cert.pem:/etc/letsencrypt/live/www.allremotedevjobs.com/cert.pem
  - /etc/letsencrypt/live/www.allremotedevjobs.com/chain.pem:/etc/letsencrypt/live/www.allremotedevjobs.com/chain.pem

I have tried a lot of different things in the config, tried, to regenerate my certificates a few times, and nothing seems to work. Worse, I have absolutely no idea what I could possibly do to make it work. I do not want to use nginx or apache as it is supposed to work without them.

If anyone has any idea on what I could try, that would be really amazing!

Thank you for your help.

8 comments

#https

3 4474 8

2021-06-07 19:03:15 UTC

Marked As Solved

brundozer

I’ve managed to solve the issue. It was simply a permission issue on the files generated by Let’s Encrypt. The only error I got was the one that I described in my first post. It is quite frustrating that there is no clear error message saying that the certificates could not be read.

I have found this post describing a similar issue, but as the author solved the problem by using nginx, which runs as root and is therefore allowed to read the certificates generated by Let’s Encrypt, I have missed the second reply which would have fixed my issue:

Anyway, thanks for the help!

Post #8

Also Liked

ntilwalli

It was simply a permission issue on the files generated by Let’s Encrypt.

Seeing this line was critical for solving my issue. The /etc/letsencrypt/live/somedomain directly has soft links to the ../../archive/somedomain/ directory. The file permissions in the archive directory are what matter. I initially didn’t investigate past the perms of the soft links in the live directory. The privkey1.pem in the archive directory is initially created (by certbot) as only readable by the owner and needs to be be chmod 644’d so it’s readable by the Elixir server process which does not run as root. Def frustrating that there is no clear error message.

Anyway, thanks!

Post #9