You can get an Airport Extreme for $179 and a 500GB USB external on Newegg for $60, the combination of which would provide all the features of a Time Capsule for $239. Granted you have to deal with two devices and a decreased throughput to the disk, but for casual use it may be enough for some people.
(you can also get 1TB externals on Newegg starting at $95, for a total of $274, a significantly lower price than the $499 1TB Time Capsule)
I believe so. Just make sure that the Airport Extreme is running at least version 7.3.1 of the firmware. I personally use an Extreme and a WD MyBook as my backup solution and while I wouldn't say it's fast I've never had issues where it wouldn't work.
(you can also get 1TB externals on Newegg starting at $95, for a total of $274, a significantly lower price than the $499 1TB Time Capsule)