Your Guide to Licensing Listing Data for Your Residential Real Estate Tech Startup
Week #8
Hello! This Sunday newsletter explores startups, short-term rentals, or whatever random thing has entered my mind this last week. I pick one topic weekly to go deep on and have some disparate quick hits at the end.
Several times a year, I get connected to a startup founder who has questions on how to get real estate listing data into their fantastic new real estate app. I have a 5-15 minute response that attempts to boil down the complexities but still misses a lot of nuance and context that could be helpful.
Today’s post is driven by a personal motivation to have a resource to point back to for those conversations. If you’re a newsletter subscriber with zero interest in 3,000+ words on licensing listing data, scroll to the Quick Hits at the end or tune in next week.
For ease of reading, I’ve broken this topic into the following sections:
MLSs Defined
Standard MLS Data Licenses
Non-Standard MLS Data Licenses
Listing Data Standards
Data Aggregation Companies
Other Sources of Listing Data
Questions to Ask Yourself Before Starting
Key Considerations Based on Those Questions
tl;dr Summary
MLSs Defined
Before we talk about listing data or how to license it, it’s essential to know what an MLS is and why it exists. And no, we’re not talking about soccer here. The best definitions of terms like “MLS Organization,” “MLS System,” and “MLS Vendor” can be found in Greg Robertson’s writeup on Vendor Alley entitled “Speaking MLS.” Please go read that before proceeding.
I’ll add one more definition. “MLS Subscribers” are individuals who pay the MLS a membership fee. Membership includes various benefits, the primary one being access to the MLS System. Not anyone can become an MLS Subscriber. This is typically restricted to licensed real estate agents, brokerage firms, appraisers, or other similar entities.
In the US and Canada, there are approximately 650 MLS Organizations, which collectively represent close to 2 million MLS Subscribers. This is down from around 900 when I started in the industry 12 years ago – due to consolidation. And yet… Why so many? Each MLS Organization generally serves all of the MLS Subscribers in a specific geographical area. As a few examples:
NorthstarMLS serves ~22,000 subscribers around Minneapolis/St. Paul, surrounding suburbs, and now much of greater Minnesota.
CRMLS (California Regional MLS) serves 100,000+ subscribers primarily around southern California, including LA, Orange County, and many surrounding areas.
Norfolk Board of REALTORs has an affiliated MLS serving ~100 subscribers near Norfolk, Nebraska.
Occasionally one MLS has little geographic overlap with other MLSs. Rarely MLSs directly compete with each other in the same geographic area. In such cases of overlap, MLSs sometimes opt into data share arrangements with each other. Such data shares allow MLS Subscribers to join only one MLS Organization but see all of the listings near them.
Typical 80/20 power laws apply to MLS Organizations. Of the ~650, the top 50 make up ~50% of listings and total subscriber count, and the top 150 make up ~80% of listings and total subscriber count.
It’s important to know why MLS Organizations exist before you go asking them for a data license. They exist to serve their subscribers. A key way they do that is to provide a means for them to share listings with offers of compensation with each other. In other words, sellers’ agents offer buyers’ agents a specific offer of compensation (e.g., 2.7% of the listing price) if the buyers’ agent brings a buyer for the listing. MLS Subscribers pay a monthly, quarterly, or annual fee for access to an MLS System, amongst other things.
Standard MLS Data Licenses
Most, but not all, MLS Organizations are owned by one or more local REALTOR Associations. This is important because REALTOR-owned MLSs generally have to follow specific rules imposed by the National Association of REALTORs (NAR). NAR requires that their MLSs license data to their subscribers (and their vendors) in at least each of the following ways:
IDX (“Internet Data Exchange”). This is the most common/classic method of licensing data. It covers licensing data for brokerage and agent websites (or apps) to publicly display listings to consumers. Such sites generate buyer leads for the brokerage and can effectively show all listings in an area. Typically a third-party vendor powers a white-labeled website or app on behalf of a brokerage or an agent. (Examples: EdinaRealty.com or the Edina Realty app.)
VOW (“Virtual Office Website”). A means of licensing data that was institutionalized as part of a 2008 Department of Justice settlement with NAR. Basically, suppose a brokerage puts up a VOW. In that case, consumers may register for more complete access to data than they would otherwise see on an IDX website. Such access is behind a gated login and not publicly available. In such cases, consumers may see information like listings that are now off-market and some additional fields or listings that aren’t available for display under IDX rules.
AVM (“Automated Valuation Model”). In 2014, NAR instituted a policy change requiring all MLSs to license data to their subscribers (and their vendors) to create automated valuations. This was partially in response to the Zestimate becoming popular and wanting to ensure that any brokerage could create its own AVM to compete.
BBO (“Brokerage Back Office”). New as of a policy change in late 2021, NAR now requires MLSs to provide their subscriber brokers a Brokerage Back Office Feed. This feed allows brokerages (and their vendors) to access all listing data for their own backend data processing needs. Such a path is typically used for cases where the data isn’t displayed to end consumers but may power other backend systems a brokerage needs or uses. (Note: this is a superset of a policy instituted just one year earlier. That policy ensured that MLSs must give brokerages back their own listing data to presumably use however said brokerage wants to. The BBO policy takes this further to include all brokerages’ listings.)
These are the only four paths with which every REALTOR-owned MLS Organization must license its data to its subscribers (and their vendors). Therefore, they represent the only semi-quick/guaranteed path to obtain data from many MLSs at scale.
Each of those four paths has standard paperwork available for it. Many of these are ‘three-party agreements’ that include the MLS Organization, the brokerage, and a technology vendor. As long as the brokerage and technology vendor agree to follow the rules outlined in the agreements, they generally won’t be denied access by the MLS.
Sometimes an MLS requires a technology vendor to go through a pre-approval process before the vendor can sign data access agreements or go live with data.
Sometimes smaller MLSs refrain from talking to technology vendors directly. They require their own subscribers to request data access agreements for their vendors through one of the four paths above. This makes it all the harder for new vendors to make inroads into the industry.
Non-Standard MLS Data Licenses
Most MLS Organizations have additional ways to license their data to others outside the standard four IDX, VOW, AVM, and BBO paths above.
Every single MLS Organization handles this differently. Here’s an example of how ARMLS (Arizona Regional MLS) addresses this in their Content Access Policy (emphasis added as my own):
All individuals and entities that are granted access to MLS Content are required to enter into a Licensing Agreement. At the sole discretion of ARMLS, Agreements may be customized as needed to address circumstances not addressed in the standard agreements.
There are four (4) standard agreements:
A. ARMLS Subscriber Agreement – an access agreement between the MLS and a Subscriber to the MLS Content and MLS System. This agreement is agreed to electronically via the MLS System.
B. Content License Agreement – Participant – a license for access to and use of the MLS Content by a Participant for the purposes of IDX, VOW, BBO or AVM. This agreement will be signed by the Participant and all parties who have access to or will be working with MLS Content.
C. Content License Agreement Brokerage Firm for its own Proprietary Data – a broad license for ARMLS to deliver only the Participant’s brokerage MLS data back to the Participant. The Participant has no contractual restrictions on the use of their own data.
D. Content License Agreement - Vendor – an enterprise license for access to and use of MLS content by a vendor, that allows the vendor to sell a product or products exclusively to Subscribers. This agreement will be signed by all parties who have access to MLS Content.
Bullet (D) above is the primary way a vendor would access ARMLS data outside the standard IDX, VOW, BBO, or AVM paths. You’ll note, however, that ARMLS intended this for use by vendors that want to sell a product or products exclusively to MLS Subscribers. Not for building “the next Zillow.”
Speaking of Zillow, most MLSs have, over time, negotiated bespoke agreements with the likes of Zillow, Homes.com, and REALTOR.com. Such contracts allow these consumer portals to display data directly outside of the standard IDX agreements. (Although Zillow has recently migrated to displaying data under IDX rules over the last two years as it became a broker.)
Listing Data Standards
The Real Estate Standards Organization (RESO) sets standards intended to be used by all MLS Organizations and their MLS Vendors. A Data Dictionary standard is intended to standardize how all MLS Organizations identify similar fields. There is a Web API standard for how to consume data, replacing an older RETS standard being sunset.
I’m glad RESO exists, and I’m happy standardization keeps improving.
It’s also true that even with standards, there is still a lot of variation from a technology standpoint across MLS Organizations and MLS Vendors.
Data Aggregation Companies
A small number of technology companies exist that provide services to other technology vendors to streamline processes around licensing MLS data. These companies’ primary service is to normalize how data is consumed from MLS Organizations and MLS Vendors. They provide a single API or single shared database for a technology vendor, allowing them to avoid learning the nuances of every MLS feed.
The field of what companies provide this service continues to evolve. Constellation 1, Zillow’s Bridge Interactive, and iHomefinder are some examples that come to mind. Other industry mainstays – like WolfNet – have stopped providing such services.
Even when leveraging these companies’ services, a technology vendor still needs a contract with every MLS Organization a vendor needs data from. Sometimes these companies have services to help with that. But that help only goes so far.
Other Sources of Listing Data
Outside of licensing listing data directly from the ~650 MLS Organizations, there are three other options that technology vendors sometimes consider:
ListHub. ListHub was the original source for listing data for Zillow and Trulia (before REALTOR.com bought ListHub). ListHub serves as a middleman between an MLS and a Publisher. It allows each brokerage to decide which sites they might want to publish listings to. This is the fastest way for a new entrant to reach a nationwide scale. It has a higher cost than many early-stage startups can absorb (low six figures annually for a national license). It usually requires that new Publishers agree to send generated leads to Opcity. And, due to the opt-in nature, it doesn’t include all the listings in a given area. Some brokerages auto-opt-in to publishing their listings to all nationwide publishers, while others do not. As a result, you will see some areas with very few listings and others with most (but not all) of the listings.
Scraping Data. I’m not advocating for this, haven’t done it myself, and my strong advice is – don’t do it. New vendors will nevertheless consider this approach as they’re getting started. Word has it that some of the national portals started out this way. (I can’t say if this is true or not.) But the biggest reason not to do this is that if you display scraped listing photos to consumers, you will get sued. You can look up past lawsuits that companies like NeighborCity were involved in. The long and short of it is that listing photos are obviously copyrighted. If you try to display these without the explicit permission of the listing brokerage (or its MLS Organization or its authorized third party like ListHub), you will eventually get sued.
Other Sources. Other companies do exist that will license listing data to other vendors on a near-nationwide footprint, often without listing photos. Sometimes these companies get access to data in an above-board way – by paying MLS Organizations for the data under clearly defined contractual terms. Other times these companies are themselves scraping the data. Often, these companies will expressly prohibit the display of this data on any consumer-facing websites or apps. As such, this may only be a fruitful path for vendors servicing other use cases, like analytics for other parties around the real estate transaction.
New vendors entering the space might do a Google search for something like “Zillow listings API,” and they’ll get optimistic that Zillow provides such a thing. Initial search results are deceiving. Zillow doesn’t provide such an API – or at least not in the way you were probably hoping for. They do have some APIs to get access to rental listing data. And as mentioned in the Data Aggregation Companies section, their Bridge Interactive division offers a streamlined means for accessing a limited number of MLS Organizations’ data. But even that requires a signed contract with every MLS Organization you want to license data from.
Questions to Ask Yourself Before Starting
Some of the questions that will influence which paths are available to you for licensing listing data that you should think about ahead of time are:
Will you display listings on a public-facing website or app to consumers without a login?
If yes on #1, do you need national coverage (US and/or Canada) quickly for your idea to work?
If yes on #1, will the site/app be primarily branded for a brokerage or agent? (e.g. EdinaRealty.com or the Edina Realty app)
If yes on #2, is this sold to a national franchise brand, or is it sold to individual brokerages? This isn’t an obvious distinction if you’re new to the industry. For example, there is a critical difference between RE/MAX, the international franchisor, and RE/MAX Results, a large brokerage franchisee in Minneapolis/St. Paul.
Are you providing a SaaS product for purchase/use by brokerages/agents?
Are you providing a SaaS product for purchase by MLS Organizations themselves?
Are you hoping to include listings that aren’t on the MLS? (e.g. rentals, for sale by owner, or other “off-market” listings)
Are you hoping to have international listings outside of the US and Canada?
Key Considerations Based on Those Questions
You Want to Become a Nationwide Search App?
If you want to create a consumer-facing nationwide search website or app that isn’t primarily brokerage branded, you’ve got an arduous path ahead. There’s no sugarcoating it.
If you’ve got seed funding, you can jumpstart this path by partnering with ListHub. But you’re long-term beholden to ListHub (owned by REALTOR.com).
Suppose you’ve got a path to a lot of quick venture funding. In that case, you might consider negotiating agreements with all of the MLS Organizations you want data from. However, the path to negotiating such agreements is treacherous. You must convince an MLS Organization’s CEO and Board of Directors that it’s in their collective best interest to license you data. They’re generally too busy to consider this outside one of the four standard paths (IDX, VOW, AVM, and BBO). That’s hard enough to do once with one MLS Organization. Doing it 400-600+ times has only been achieved by a small handful of giant companies that have been in the industry for many years.
In addition to lots of funding, you’ll need to be patient on having lots of ramp-up time, and you’ll need to hire numerous MLS Industry Relations individuals on staff.
Want more certainty, to take less time, and do so for less money? Become a real estate brokerage in all 50 states. That sounds like … a chore that will still take a lot of money. And oh, by the way, you’ll draw the ire of almost everyone in the industry.
The only other path I’m aware of is that apps like Estately seemed to get off the ground by partnering with brokerages. There was an arrangement where a local brokerage sought a data license. Then, there was co-branding between Estately and the local brokerage on their national go-to-market. I haven’t seen many do this, and I have suspicions about why.
You Want to Sell Brokerages or Agents Tools?
Hopefully, your use case fits neatly into the IDX, VOW, AVM, or BBO path. If so, all you need are paying customers! And, patience in filling out paperwork and waiting for credentials with each new MLS you add. And, you’ll also need development time or a Data Aggregation Company to assist you.
This was the path we took at HomeSpotter, and we built connections to 300+ MLS Organizations over our journey. We didn’t work with a Data Aggregation Company, mainly because we wanted to control our own destiny.
Suppose, for some reason, your use case doesn’t fit into the IDX, VOW, AVM, or BBO boxes. In that case, I’d encourage you to reconsider how to adjust your service to make it fit into one of them. Especially now that BBO is a more universal option within the last year.
You Want to Sell to MLS Organizations?
If an MLS Organization buys a service from you to provide a “Member Benefit,” they’ll typically give you their data at no additional cost to fulfill the service. Just be prepared for really long sales cycles.
You Want to Sell to National Franchisors?
Cool. I have so much to say about this, but I’m already at 3,000+ words. For today, I’ll just say, start by selling tools to brokerages or agents, show some demonstrated successes, and then some other day I’ll bore you with all of the MLS rules compliance issues that working at a national franchise level introduces.
You Want to Include Non-MLS Listings?
Most MLS Organizations do not allow co-mingling non-MLS listings with MLS listings. Where you source these other listings is outside the scope of this post.
You Want to Do This Outside of the US and Canada?
The international real estate market is almost entirely different than what we have in the US and Canada. In general, the seller doesn’t pay a buyers’ agent commission. Therefore, there often aren’t buyers’ agents at all. Without this fundamental business model in place, MLS Organizations also don’t exist internationally in the same way they do in the US and Canada. If you’re curious, Australia seems to have the most robust listing portal market outside North America.
Most residential real estate technology companies solve for the US and Canadian markets, or they solve for international markets. They rarely do both due to business model differences.
tl;dr Summary
Licensing listing data and scaling to a nationwide footprint is really, really hard. There are proof points in the market that show it can be done. But hopefully, the long writeup above allows you to go in with eyes wide open on what it will take.
This Week’s Quick Hits
(If you showed up to read about licensing listing data, these quick hits have nothing to do with that topic. They’re just random thoughts appended to the end of my weekly newsletter.)
I switched to using Arc browser earlier this week. It will be a more significant productivity boost in total for me than Superhuman was for email. I have a limited number of invites remaining. Hit me up if you want one.
My son’s First Lego League team made it to state yesterday and also won an award for their innovation project. This is the second year in a row my son has gone to state. I’m so proud of him. This year, I’ve been more actively involved as an assistant coach, often helping his team two days a week. First Lego League is doing such a good thing for grade school kids. Beyond coding and problem-solving, it also helps them to work on presentation skills. I’m so grateful for my son’s teacher/coach, who has gone above and beyond to invest in the kids.
Photo by Cytonn Photography on Unsplash