Howdy! I’m trying to do a ton of concurrent outbound HTTP efficiently. I’m doing this:

    task = fn url ->
      Req.new(
        url: url,
        finch: MyFinch,
        max_redirects: 2,
        retry: false
      )
      |> Req.Request.put_header("connection", "close")
      |> Req.head()
      |> case do
        {:ok, response} ->
          response.status

        {:error, response} ->
          :error
      end
    end

    Task.Supervisor.async_stream_nolink(
      {:via, PartitionSupervisor, {MyApp.TaskSupervisors, self()}},
      urls,
      task,
      max_concurrency: 1000,
      on_timeout: :kill_task,
      ordered: false,
      timeout: 60000
    )
    |> Enum.to_list()

At first, processing output looks good, but after ~5 seconds, I start ending up with a ton of:

%Mint.TransportError{reason: :nxdomain}
%Mint.TransportError{reason: :nxdomain}
%Mint.TransportError{reason: :timeout}
%Mint.TransportError{reason: :nxdomain}
%Mint.TransportError{reason: :nxdomain}
%Mint.TransportError{reason: :nxdomain}
%Mint.TransportError{reason: :timeout}
%Mint.TransportError{reason: :timeout}
%Mint.TransportError{reason: :timeout}

Anything bad leap out? The error seems to indicate Mint failing in connect…but…why. At lower volume or the start of a big batch, everything works fine. It’s just as it cranks up heavy it goes sideways.