Could you expand on why you don't like Apache? I'll grant you that it can be confusing to get used to configuring, especially if you're not really familiar with how the web works, but as far as I can tell it's a pretty damn good server.
My fundamental beef with Apache is that people wrap way too much functionality into it, to the point where their entire end-to-end web stack is two sets of processes: Apache and a database server.
On top of that it's configuration is awful, the modules are all awful compared to their domain-specific alternatives, the architecture is awful (especially wrt concurrency), the development process is molasses, and the Apache Software Foundation has all but abandoned httpd for IBM-focused all-Java astronaut architecture with as much bureaucracy as they can possibly fit into a public process.
I hate Apache because I'm intimately familiar with how the web works, and the ASF is responsible for so much web-hostile WS-* garbage.
For a while (~4 years ago) I thought lighttpd was worthwhile as a total replacement, and did a project that hacked it's WebDAV implementation for userspace filesystems (pre-MacFUSE), but fundamentally its architecture is still Apache just with sensible concurrency.
I've come to be really fond of the reverse-proxy model, and really like Nginx running in front of independent app processes using whatever HTTP abstraction is native to the language (WSGI, Rack, Servlets, etc.) along with a nice native high-level spec-focused HTTP server (twisted.web, mongrel, ???). The last web application I wrote from scratch was on Google AppEngine, and I really like their version of WSGI.
> My fundamental beef with Apache is that people wrap way too much functionality into it, to the point where their entire end-to-end web stack is two sets of processes: Apache and a database server.
Fine, that's a technical point, and fair, but that doesn't make it a 'dog'. It works well for many things, and not so well for some others.
Most of the rest of what you write is vitriol and hyperbole ... "awful", "molasses", "hate", "garbage", and so on and so forth. Your "liberal with what you accept and conservative with what you output" (which I voted up, it's a great quote) goes for interaction with other people, as well.
As a member of the ASF, I'd also like to point out that the foundation goes where people want it to. Sure, there are lots of Java projects, but there are some reasons for that. Java tends to be used by people (and especially companies) who also like to be sure of the provenance of their software, and Apache projects can be very sure of that. That process does introduce a bit of bureaucracy, but it's not all that bad, really. Overall, yes, the ASF has gone a little bit 'enterprisey', but that's no reason to get bitter about it. If you don't like it, don't use it. (The FSF also has similar kinds of bureaucracy too, in order to ensure that they have the copyright for their software).
There are also numerous non-Java projects, like CouchDB, Perl, Python and Tcl modules, and 'Harmony' which is Java, but an implementation of the language itself, which requires some fairly interesting hacking, and of course the web server itself. Most of these projects are worked on by different people.
The concurrency model (models, actually), like many things, is of course a tradeoff, with no absolute 'right' answer (although I think Yaws does a pretty good job).
BTW, I do the Apache Tcl stuff, or did... I don't have much time for it these days.
> I hate Apache because I'm intimately familiar with how the web works, and the ASF is responsible for so much web-hostile WS-* garbage.
Presumably, there are a few people at Apache that know a thing or two about how the web works, too. Even "intimately". Like, say, Roy Fielding, who wrote part of the RFC this whole article is about.
I'm not familiar with what you're talking about. Could you clarify something for me? Why would you need Nginx and twisted.web? Wouldn't you just call your WSGI application from Nginx?
Nginx's architecture doesn't really mesh well with WSGI's interface protocol... I believe it has to do with blocking the serving Nginx process whilst the WSGI request is being processed.
Nginx prefers to "pass off" the request to a web-app server to do the heavy lifting and take care of fast, easy serving in the fastest way possible.
"select" based web servers and their modern (poll/kevent/whatever) brethren, for serving purely static content, are faster and smaller. But that's always been the case.