How Volatile Are ChatGPT's Vacation Rental Recommendations?

More guests and homeowners are asking ChatGPT who the best vacation rental company in town is. If you manage vacation rentals, the obvious questions are: what is ChatGPT reading when it answers, and will it give the same answer tomorrow?

This study was inspired by Glen Allsopp's AI volatility research at Detailed[1], which found that exact repeat answers are rare but the most prominent brands keep coming back. We wanted to see how that plays out in our industry, so we ran 7 prompt templates (“best vacation rental management company in…”, “most trusted…”, “manage my Airbnb in…” and so on) across 42 coastal markets on Sep 3, Sep 10, Sep 11 and Sep 19, 2026. We captured every answer along with the web pages ChatGPT retrieved and the map cards it showed.

The short version: the top pick is fairly stable, but the rest of the list isn't. Across any two days, ChatGPT kept the same #1 company 78% of the time, but only about half of the companies it named (49% overlap) showed up again. The websites it read changed even more — only 33% overlap on average.

We also found that ChatGPT's map cards often show Yelp ratings, not Google. Where a company had both profiles, ChatGPT showed the Yelp numbers 64% of the time — a median of 2.5 stars from 23 reviews, compared with 4.6 stars from 358 reviews on the same companies' Google profiles.

We picked 42 coastal vacation rental markets along the Gulf Coast, the Southeast and the Mid-Atlantic, plus San Diego, and wrote 7 prompt templates that a guest or homeowner might realistically type. Each template was run in every market on four days. For every answer we saved the full response, the companies it named (and their order), the web pages ChatGPT retrieved, which of those pages it cited, the search queries it ran, and any map-pack cards it showed.

One important caveat: the model changed under us. ChatGPT decides for itself whether to answer with its full model or a smaller “mini” model. On Sep 3 and Sep 10 almost every answer came from the mini model; by Sep 19, 95% came from the full model. Some of the change we measured between days is therefore a change in model, not just normal randomness — which is also what a real user would experience.

Sep 19 also covered fewer markets (31 of 42), so comparisons involving that day use only the markets captured on both days. We compared days in pairs, and when we say a company was “named every day” we only count days where at least 4 of the market's 7 answers were stored.

To add context, we matched the companies ChatGPT named against Nearsight's database of vacation rental managers and pulled in their Google, Yelp and BBB data, OTA dependency, portfolio size, web traffic and more. We matched 426 of the 838 company–market pairs.

Before answering, ChatGPT searches the web and reads a batch of pages — a median of 20 to 37 per answer, depending on the day. Setting aside vacation rental companies' own websites, we wanted to know which sites it relies on in every market.

The answer is a very short list. Only 5 third-party sites were retrieved in at least half of markets on at least 3 of the 4 days: bbb.org, reviews.birdeye.com, comparent.com, airbtics.com, onefinebnb.com. Three are review or rating sites and two are short-term rental software companies that publish “best property managers in [city]” pages.

The mix changed sharply on Sep 19. BBB and Birdeye, which had been retrieved in roughly 60% of prompts, almost disappeared (about 2%). In their place, ChatGPT read the companies' own websites far more often, plus Facebook groups, travel magazines like Condé Nast Traveler and Southern Living, and — oddly — dictionary pages defining the word “best” and Wikipedia articles on national brands.

When ChatGPT recommends local businesses, it often shows a row of map cards with a star rating and review count. ChatGPT doesn't say where those numbers come from, so we worked it out: for each of the 4,110 cards, we compared the rating and review count against the company's Google and Yelp profiles in Nearsight's database (Google data as of September 28, 2026).

Because our Google data is more recent than the study, we allowed Google review counts to have grown by up to 30% (or 15 reviews) since the card was shown. Yelp counts rarely move, so we used a tight match there. We could confidently place 59.8% of cards; the rest belonged to companies we couldn't match or whose numbers fit neither profile. The results didn't change whether we allowed 20%, 30% or 50% Google growth.

42% of the cards we could place showed Yelp data. Looking only at companies with both a Google and a Yelp profile, ChatGPT showed Yelp 122 out of 192 times (64%). That matters because vacation rental companies usually look much worse on Yelp: those Yelp cards had a median of 2.5 stars from 23 reviews, and 93.4% were under 4 stars. The same companies' Google profiles had a median of 4.6 stars from 358 reviews.

We compared every pair of days on the same prompts in the same markets. The results split into two layers: a stable top and a churning middle.

• The #1 pick held up. ChatGPT named the same top company 78% of the time, ranging from 66.7% (Sep 3 to Sep 19) to 85% (Sep 3 to Sep 11).
• The rest of the list churned. Only 49% of the companies named in a market on one day were named on the other day of the pair.
• The sources churned even more. The overlap in websites retrieved averaged 33%, falling to 19.7% between Sep 3 and Sep 19.
• The searches were almost never repeated. ChatGPT's behind-the-scenes search query was identical only 6% of the time.

Even the most similar pair — Sep 10 and Sep 11, one day apart — kept only 57.1% of companies and 44% of websites. What was stable between those two days was the pages themselves: when a website came back for the same prompt, it was the same page 90.8% of the time. For pairs involving Sep 19, when the full model had taken over, that dropped to about 56.6%.

Across markets with at least 3 days of data, we tracked 705 company–market pairs. 197 (28%) were named every day, 109 were named on most days, and 399 (57%) were named only once. Every market had a core of 4 to 9 companies ChatGPT always came back to, surrounded by a long tail it mentioned only now and then.

Pick a market to see its leaderboard. Click a company to see its data in Nearsight.

We compared companies named every day with companies named only once, using their Nearsight data. Among local operators, the consistent group had about 3x the Google reviews (443 vs 150) and about 3x the organic web traffic. They were twice as likely to have had a map-pack card (80.6% vs 40%), and less likely to be highly dependent on OTAs like Airbnb and Vrbo (18.2% vs 25.2%).

Portfolio size mattered much less. The consistent group was only modestly larger (median 145.5 vs 115 properties), and across all matched companies there was essentially no correlation between portfolio size and how often ChatGPT named them (0.01). National brands are shown separately because only a handful matched our database — treat that view as anecdotal.

• Don't judge your AI visibility on one screenshot. With only about half the list carrying over between days, a single check tells you very little. Track the same prompts over time.
• Check your Yelp profile. Most managers ignore Yelp, but ChatGPT showed Yelp ratings on 64% of map cards for companies that have both. A 2-star Yelp profile can sit right next to your name in the answer.
• Be present on the handful of sites ChatGPT reads everywhere. BBB, Birdeye, Comparent and a few STR software directories showed up in most markets. A complete, accurate profile on these is cheap insurance.
• Keep your own website clear and crawlable. Company homepages and owner/management pages made up a large share of what ChatGPT cited — and on Sep 19 they became its main source.
• Reviews still matter. The companies ChatGPT named every day had far more Google reviews and stronger ratings than the ones it mentioned once.