garrison
Safely performing external HTTP requests (avoiding SSRF)
If you want to perform arbitrary external HTTP requests you have to be very careful to avoid Server-Side Request Forgery (SSRF). Certain IPs may respond with privileged data (e.g. cloud servers often have a special metadata endpoint). There could also be privileged internal services that a server has access to, some of which may use HTTP.
The Phoenix Security Guide mentions this but provides no guidance beyond “avoid making requests with user input”. This Paraxial guide provides similar advice (I suspect the former adapted the latter).
This is unhelpful; there are entire classes of applications which need this behavior, such as: anything which renders link previews, proxies images, reads RSS feeds, fetches web content, and so on. This is not a niche use-case.
As a start you can attempt to filter out malicious user input (e.g. a link to a private IP address), but this is insufficient as a DNS response can still point a malicious domain to a sensitive address. I have seen this referred to as “DNS rebinding” but I think this is an abuse of terminology (though there is overlap).
It is tempting to think you can get around this by first resolving the name with :inet_res and then connecting to it as normal, but this is also insufficient. There is no guarantee the connection will receive the same DNS response. An attacker could simply toggle back and forth until they get lucky.
We can find the correct solution in go-camo, an image proxy written in Go. One must first resolve the name, check the address, and then make a request to that address. It seems that in Go this can be done via some callback (I don’t know Go).
So back in Elixir, we can resolve the name with :inet_res but we need to pass the resolved IP directly into our HTTP client. The problem is that we still need to pass in the original hostname, not only for the Host header but more importantly for HTTPS. The hostname is needed to validate the cert.
Mint’s connect/4 actually supports this via a hostname option, but as far as I can tell Finch and therefore Req do not take advantage of this. Finch allows conn_opts but only when you start a pool, which is unhelpful. As a result, I don’t see any viable way to protect against SSRF in the application layer using our standard tools (Finch/Req).
This seems like a problem. Am I missing anything?
Marked As Solved
joram
You can pass the hostname to Req as part of connect_options:
Req.get(
"https://123.456.789.123",
connect_options: [hostname: "www.example.com"]
)
Req will dynamically start or reuse a Finch pool with those options.
Pools don’t shut down by default so you may also want to specify a pool_max_idle_time, and you probably also want to disable redirects:
Req.get(
"https://123.456.789.123",
connect_options: [hostname: "not.actually.example.com"],
pool_max_idle_time: :timer.minutes(5),
redirect: false
)
Also Liked
LostKobrakai
From a quick look the custom :hostname option seems to exist because of this:
On newer OTP versions it might be safe to manually set the Host header and use the IP on the address.
garrison
A blame shows that the option was added in this PR which adds support for connecting to IPs (and sockets). The hostname is actually required when connecting to an IP. (Actually, what are you supposed to do if you’re really connecting to an IP, pass it in as a string?)
The docs claim the hostname value is used for the Host header and for HTTPS: to validate the cert and also for SNI (to request the right cert). It also says “and so on”, whatever that means lol.
This is all fine and good; it’s exactly what I want to do. Resolve the domain myself and then connect to the IP directly (to avoid malicious DNS replies) while preserving the original host for HTTPS.
The problem is that as far as I can tell Finch (and therefore Req) provides no way to pass this value in. Apparently it can be passed in with conn_opts when creating the pool, but I want to pass it in when making the request. I have a very poor understanding of how Finch’s pools actually work so maybe there actually is some way to do this? But it’s certainly not obvious, and this is a serious security issue for which we need to provide a solution and clear guidance.
Are you suggesting the Host header will be used for HTTPS by Mint when connecting to an IP? I haven’t found anything to this effect in the docs or code but I also didn’t try it either.
As far as I can tell the code you linked is related to Mint verifying cert hostnames itself because they don’t like the OTP behavior or it was not previously available (not immediately clear to me).
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"