I worked at Placester for a couple of years and built the system that imports data from real estate agencies. When I left last year, we had coverage with around 90% of the MLS's in the US. Most of what you say is right, but some clarifications and context:
You don't need to be a brokerage to get access to the MLS feed. Each MLS has their own policies for how you can display the data, what logos, size and text needs to be shown on the page with their listings though. Which means it unrealistic to build Zillow/Trulia site off of MLS data. Placester builds them for individual real estate agents which is significantly easier for keeping the MLS happy.
Some MLS's are great and will give you access to the data without much hassle, others are not and you have to pay a lot of money. Even once you get access, you will get almost no technical help or useful documentation on integrating with them. Since MLS's are almost never related, you still need to talk to 300+ different companies in order to get coverage of the US.
There is a standard that most MLS's follow for their data, which is called RETS [1]. I would say about 80% of MLS's use RETS, the problem with RETS is that it's a standard in the same sense that CSS was a standard 10 years ago. The original library I wrote for RETS [2] is open sourced, and is littered with examples [3], [4] and [5] (to name a few), of inconsistencies across RETS servers.
If you can work through all of that, you're golden. It took us about 1.5-2 years to get the experience of seeing how MLS's work in order to simplify the integration process down to 1-2 days, with RETS typically requiring no (or minimal) engineering work.
Thanks for all your work on ruby-rets! It works like a champ for pulling in listings from MLSPin and CCIAOR. As you mentioned, dealing with all the "certified" RETS vendors is a nightmare. For the uninitiated (and fortunate), RETS has its own querying language called DMQL which is inconsistent across versions and MLS vendors. Even trivial tasks like importing photos are handled with vast difference across MLS vendors.
Despite all the technical hurdles, given their resources, I would be shocked if Zillow and Trulia DIDN'T import the majority of their Mlsdata via RETS. Most MLS providers allow 3rd party access to the data feeds. There is no way all the listing data is re-entered via agents.
Seconded! RETS is a goddamn nightmare. For years we've used librets and it's the worst. When we found your Ruby-only library we got it working with our feeds within the day and I can't tell you how relieved we are not have to deal with librets compilation.
You don't need to be a brokerage to get access to the MLS feed. Each MLS has their own policies for how you can display the data, what logos, size and text needs to be shown on the page with their listings though. Which means it unrealistic to build Zillow/Trulia site off of MLS data. Placester builds them for individual real estate agents which is significantly easier for keeping the MLS happy.
Some MLS's are great and will give you access to the data without much hassle, others are not and you have to pay a lot of money. Even once you get access, you will get almost no technical help or useful documentation on integrating with them. Since MLS's are almost never related, you still need to talk to 300+ different companies in order to get coverage of the US.
There is a standard that most MLS's follow for their data, which is called RETS [1]. I would say about 80% of MLS's use RETS, the problem with RETS is that it's a standard in the same sense that CSS was a standard 10 years ago. The original library I wrote for RETS [2] is open sourced, and is littered with examples [3], [4] and [5] (to name a few), of inconsistencies across RETS servers.
If you can work through all of that, you're golden. It took us about 1.5-2 years to get the experience of seeing how MLS's work in order to simplify the integration process down to 1-2 days, with RETS typically requiring no (or minimal) engineering work.
[1] http://en.wikipedia.org/wiki/Real_Estate_Transaction_Standar... [2] https://github.com/zanker/ruby-rets [3] https://github.com/zanker/ruby-rets/blob/master/lib/rets/htt... [4] https://github.com/zanker/ruby-rets/blob/master/lib/rets/htt... [5] https://github.com/zanker/ruby-rets/blob/master/lib/rets/htt...