> Having complete control of TCP/IP in userland like this, with so little code, is so valuable I feel like there needs to be some special name for the technique.
Yes! Userspace TCP/IP is how we implement firewall for Androids (which don't expose iptables on non-root devices but let you setup TUN interfaces via VPN APIs). Right now, we rely on LwIP (wrapped in golang) and it has worked wonderfully well; especially since it is light-weight without any locking-overheads (single-threaded) and that bodes well for battery-powered devices.
> The whole thing is kind of a vindication for Go's standard library network interface, which I have always hated.
The Fuchsia team at Google is re-implementing netstack3 in Rust (and hence you're probably right to call it "gVisor netstack") due to what I presume are performance and efficiency reasons (which is of interest to us because we develop for smartphones). Of course, flyctl doesn't need that, but since you wrote about pulling in heavy dependencies, I am interested in your take on it.
Don't want to go OT but I'm super curious what your experience developing a network application for non-root Android devices has been?
As a non-Android developer, I've been working on a project the last few months that involves running an HTTP server on the device and tunneling out so it can receive requests from the outside world, and the platform feels nerfed at every level from filesystem access to keeping your server from being battery-killed.
Android development is a bit tedious relatively compared to iOS due to having to support multiple API levels and having to account for subtleties across OEM implementations, but things have drastically improved in the last few years, especially after Oreo (Android 8).
Process reaping is also, I believe, a problem on iOS? One way to keep a process out of OutOfMemory/LowMemoryKiller's reach is to make it a foreground service (what stuff like Music Players do) and generally be very stringent with resource use. It is easy to profile for resource usage thanks to Android Studio's built-in profiler and tools like https://perfetto.dev/
Oh iOS is way worse from what I can tell. I don't consider it a viable computing platform so haven't bothered trying to make my software run there.
But Android seems to be working hard to "catch up" to iOS.
I'm mostly comparing to native Linux development. Obviously you may need to make some changes for security, but I feel like they've gone way overboard with things like forcing the storage access framework/media storage APIs, killing even foreground services (doze mode etc), and so on.
At the end of the day, if you're using software to purposefully limit what hardware is capable of, I think that's wrong. Even if you're worried about security, add a simple escape hatch for power users.
Yes! Userspace TCP/IP is how we implement firewall for Androids (which don't expose iptables on non-root devices but let you setup TUN interfaces via VPN APIs). Right now, we rely on LwIP (wrapped in golang) and it has worked wonderfully well; especially since it is light-weight without any locking-overheads (single-threaded) and that bodes well for battery-powered devices.
> The whole thing is kind of a vindication for Go's standard library network interface, which I have always hated.
The Fuchsia team at Google is re-implementing netstack3 in Rust (and hence you're probably right to call it "gVisor netstack") due to what I presume are performance and efficiency reasons (which is of interest to us because we develop for smartphones). Of course, flyctl doesn't need that, but since you wrote about pulling in heavy dependencies, I am interested in your take on it.