I used to like Xfce in its old 4.2 days. It used to be the Cholesterol-Free Desktop Environment.
XFCE seems to turn into KDE 3 - start from design of the old Unix Common Desktop Environment and add an eclectic mix of features from all possible desktop interfaces while catering to users. Not necessarily a bad thing, a lot of people like it.
XFCE itself is still written to be modular and lightweight (although the definition of lightweight has changed over the years). Most of its components can be installed separately with few dependencies. Distributions like Xubuntu and Fedora pile on tons of packages to fill up the CD images and present a nice, full-featured desktop to new users. After the Gnome3/Unity upgrades I switched to Xubuntu and just rip out about 50% of the packages after doing a clean install.
KDE 4 and GNOME 3 happened in the meantime. Which means there's been a lot of demand but a dearth of supply for a conservative but feature-rich desktop environment.
Since XFCE was probably the closest to meeting that demand, it's not surprising that it gained a lot of features. Meanwhile, it looks like LXDE is going to fill the niche that XFCE used to fill.
I have only used XFCE since 4.6 so you could be right. XFCE 4.8 is still way less of a mess than GNOME and KDE 3. Configuring and using XFCE was very straightforward.
It gets a lot of hate right now, but its a really stress-free environment for me. Taking a look at the GNOME 3 cheat-sheet will fix most of the power-user complaints.
I think when GNOME 3 get more features and applications it will quickly regain user share because of its superior design philosophy. Like what happened to GNOME 2 - it got flamed hard in the reviews, Linus blasted it; but its developers kept making it better without changing their design decisions and it was adopted by major distributions and most users stuck with it.
XFCE seems to turn into KDE 3 - start from design of the old Unix Common Desktop Environment and add an eclectic mix of features from all possible desktop interfaces while catering to users. Not necessarily a bad thing, a lot of people like it.
I find it a confusing mess now.