Short answer: Real estate sites break in ways brochure sites don't: an IDX feed can generate tens of thousands of near‑identical listing URLs, faceted search creates infinite crawlable combinations, sold listings go stale, and the same MLS data appears on hundreds of agent sites. The job is to let Google crawl and index the pages that can rank (your area pages, your featured listings) and keep it away from the ones that can't.
The IDX crawl problem
An IDX integration can expose:
- One URL per active listing (fine, but thin and duplicated across agents)
- Filter combinations:
— effectively infinite - Pagination that goes 200 pages deep
- Sold/expired listings that 404 or stay live forever
Left alone, Google wastes its crawl on junk and never gets to your good pages.
How to control it
Faceted / filtered URLs
- Decide which facets deserve indexable pages (e.g.
— real search demand) and which don't (price sliders, sort order). - Non‑indexable facets:
+ let them be crawled, or block withif crawl budget is a real issue. Don't do both to the same URL. - Canonical filtered pages to the clean parent where appropriate.
Individual IDX listings
- Keep them indexable if your feed lets you add unique content (agent notes, neighbourhood context); otherwise expect them to rank weakly — they're duplicated market‑wide.
- Add
/schema. - When a listing sells, either 301 to a relevant search page or keep it as a "sold" page with a clear status (good for "sold prices in [area]").
Pagination
- Self‑referencing canonicals on each page; make sure page 2+ is still crawlable so listings are discovered.
- A sitemap of listing URLs helps discovery more than deep pagination.
Sitemaps
- Split:
,,,. Keepaccurate. - Submit all in Search Console; watch the indexed‑vs‑submitted ratio.
Speed at scale
Listing and detail pages are image‑heavy and often server‑rendered from a feed. Cache aggressively, lazy‑load galleries, use a CDN, and don't block rendering on the IDX widget's JavaScript.
Duplicate content across agents
You can't out‑unique the MLS data itself. You win with the pages around it: area guides, market reports, school pages, "homes under $X in [area]" — content the portals and other agents don't bother writing.
Key takeaways
- IDX feeds create crawl‑budget and duplicate‑content problems; manage which facets get indexed.
- Give sold listings a deliberate afterlife (301 or a "sold" page), not a 404 or a stale live page.
- Split sitemaps by type and watch indexed‑vs‑submitted in Search Console.
- Your rankings come from the unique local content around the listings, not the listings themselves.
FAQ
Should I noindex all IDX listing pages?
How do I stop Google crawling filter URLs?
Can you audit my IDX site?
Alamin Sultan
Web developer for real estate and e‑commerce. WordPress · Next.js · MERN.
