Here be dragons. The free version of nginx will only do DNS resolution on a backend hostname at startup, the paid version will do periodic lookups.

They do mention this further down the page, but in 8 months when it randomly breaks you have to hope you remember it needs to be periodically restarted to keep working.

This is by far the stupidest paywalled feature ever, because it amount to downtime extortion.

There are hacky ways around it, though. The method here is something I've used: https://tenzer.dk/nginx-with-dynamic-upstreams/

Having to hack around the nginx paywall made me briefly consider going back to Apache.

Have you tried Caddy server? No affiliation just a happy user. It’s open source.

It may or may not be able to replace Nginx depending on your use case. For me Caddy has replaced everything I used to use Nginx for and more.

https://caddyserver.com/

I'm quite interested in Caddy. The last time I checked, things were in a rough spot with the v2 transition, but it looks like the documentation has improved.

Apache Traffic Server (no relation to Apache itself) would be an excellent option: https://trafficserver.apache.org/

Kong Gateway - which is built on top of NGINX - provides frequent DNS lookups for free in the open source version, and we have implemented this feature a very long time ago (2017?) to overcome this limitation.

So if you need need this capability for free, check it out. Not only that, but SRV record resolutions too.

It’s open source, what’s keeping you from patching out that limitation?

I've always been curious to see how project owners respond to someone re-implementing portions of paid features in an open source project.

Assuming the patch is valid, do they decline it citing the paid feature or do something like making a straw man argument against it?

I haven’t tried to pitch something to nginx, but as long as you did it as a clean implementation, “We’re declining to merge, since this is duplicative of code in our paid offering” is the general approach. And then you’re able to maintain your patch set alongside their upstream source.

My patches removing spyware and phone-home in open source have been universally rejected.

Nothing. For instance, the Debian package nginx-extras includes implementations of some closed-source nginx features. But in my experience the patches are not particularly well-maintained (since they obviously won't be merged by nginx, there's already an official paid version, and the features are named differently from the closed-source ones so they're harder to find).

Dumb question - do AWS S3 endpoints change DNS that much? Is the DNS resolution limit an issue with this specific workload, or just a general issue?

Run `dig s3.amazonaws.com` a few times. It's got like a 5 second TTL and the IP changes every time.

Use openresty local resolver with

set $proxy_url xxx; proxy_pass $proxy_url;

Use HAProxy, this is what it is designed for.