We have posted an incident report that explains why the incident happened and what we are working on to prevent it in the future: 2025 07 30 hex outage | Hex.

You inspired me to cobble something together (it’s in Bash, sorry).

It’s a script that checks the certificate expiration date for a bunch of domains, then uses notify-send to send a notification for each domain that has a certificate that expires in 3 days or less. Then I made it run in a cronjob once per hour.

/home/user/scripts/network/cert-checker.sh

#!/bin/bash

# List of domains to check
DOMAINS="elixirforum.com erlangforums.com hex.pm hexdocs.pm"

# # Log file (optional, for debugging)
# LOG_FILE="/tmp/cert_check.log"

for domain in $DOMAINS; do
  exp_date=$(echo | openssl s_client -connect $domain:443 -servername $domain 2>/dev/null | openssl x509 -noout -enddate | cut -d= -f2)
  
  # Check if we got a valid date
  if [ -n "$exp_date" ]; then
    exp_time=$(date -d "$exp_date" +%s) && 
    current_time=$(date +%s) && 
    diff_seconds=$((exp_time - current_time)) && 
    diff_days=$((diff_seconds / 86400)) && 
    diff_hours=$(((diff_seconds % 86400) / 3600)) && 
    if [ $diff_days -le 3 ] && [ $diff_days -ge 0 ]; then 
      notify-send "Certificate Alert" "$domain expires in $diff_days days and $diff_hours hours"
      # # Optional: log to file
      # echo "$(date): $domain expires in $diff_days days and $diff_hours hours" >> $LOG_FILE
    fi
  fi
done

Make it executable with chmod +x /path/to/script, then add it to a cronjob to check the certs once per hour:

0 * * * * DISPLAY=:0 /home/user/scripts/network/cert-checker.sh

Seems to be working so far.

I put this together with a lot of help from the new Qwen3-Coder model.

That was the sentiment of my comment. I know I can set up a cron, I know when it can be done (without asking AI), but I just didn’t… partially because in a weird way I enjoy doing it manually (I only have one freelance client as I work work a full time job, if I had more I would have nipped this in the bud long ago). So I was delighting at the idea that maybe (and it probably isn’t even the case) that an owner of hex is doing the same

I’m following this for updates so I don’t have to just keep refreshing

Wait, is someone manually updating hex.pm’s letsencrypt certificate based off of the email reminders that recently stopped? I was doing this with a client and they emailed me today about this very issue because I kept telling myself every three months “Oh right, I gotta set up a cron for this, ugh… I’ll do it this afternoon…”