“Never saw a pool but didn’t look for it”: reviewing the reviews

The Apple Tree Inn has 4.1 out of 5 stars on Google Maps, 3.5 out of 5 on TripAdvisor, 3.7 out of 5 on Yelp, an 8.2 out of 10 on Expedia, and a 7.8 out of 10 on Booking.com.

This puts it around the 23rd percentile of the first 40 hotels that come up in the area on Google Maps, i.e., quite low. It crushes the Berkshire Travel Lodge (2.7) and Econo Lodge (3.0), edges out Seven Hills (3.9), ties Life House (4.1), trails Quality Inn and Holiday Inn (4.2), is overshadowed by Miraval and the Red Lion Inn (4.4), is humbled by the Courtyard by Marriott (4.6), and can’t hold a candle to the Devonfield (which somehow has a 5.0 after 323 reviews).

As someone who knew nothing about hotels before Claire, I’ve often gone by these numbers. Rating and price is certainly my first filter. Now that I’m getting to know this market, I know that Seven Hills is utterly charming (though currently closed) and the Red Lion is an absolute institution, and I would never choose Quality Inn or Courtyard over them. What does it even mean that Miraval and the Red Lion have the “same rating”? Miraval recently underwent a $60 million renovation and its listed room rate on Google is $1,708; the Red Lion’s is $186. Miraval is like a whole campus with sports arena–level parking; the Red Lion is one big building on a street corner in town. They are very different things.

But these ratings, and the reviews behind them, are part of what you’re buying when you buy a hotel, and also your best look at what the hotel is. You can tour the property and inspect it very diligently, but you are only looking at one time of year. It’s hard to evaluate how it’ll feel in the winter, how many rooms smell or get stuffy, or how often things break.

While putting together her offer, Claire spent a lot of time reading Apple Tree reviews. But, weirdly, I think my friend Aaron spent more. He had just quit his big tech user research job but some sick compulsion compelled him to continue researching users. Without anyone asking (or really knowing what he was doing), he put together an Airtable covering 456 guests and breaking their reviews down into 2,179 sentiments. Plenty are positive:

• “the main house and the rooms are finely decorated in a traditional manner which made us feel immersed in a bucolic atmosphere, it was pretty romantic i must say” — Rating: 5; Tags: Main house, Rooms, Decoration, Feel/Ambiance, Common areas; Reason for stay: Getaway; Room type: Main house; Guest type: Couple; Sentiment: Positive; Date: July 2022.

But I feel especially drawn to the negative ones:

• “when we got there and checked us in no one said anything to us about the ostritch room being closed and the event cancelled…” — Rating: 2; Tags: website, communication, open/closed confusion, check-in/check-out; Reason for stay: Music/Entertainment; Room type: Unknown; Guest type: Family; Sentiment: Negative; Date: September 2023.

• “i was really embarrassed to bring my girlfriend here, it has a reputation of being nice that it absolutely does not meet.” — Rating: 1; Tags: Mismatched expectations, Lodge, Rooms, Reputation, Website, Communication, Feel/Ambiance; Reason for stay: Location; Room type: Lodge; Guest type: Couple; Sentiment: Negative; Date: August 2022.

• “i did get email which stated that for $110 i could delay departure and also for $34 i could go into my room sooner than 4pm! also had a $25 dollar ‘resort’ fee added, allegedly to use one of 2 old, tiny tennis courts tucked into the woods” — Rating: 1; Tags: Nickel-and-Diming, Communication, Emails, Mismatched expectations, Early check-in / Late check-out, Resort Fee; Reason for stay: Location; Room type: Lodge; Guest type: Solo traveler; Sentiment: Negative; Date: August 2022.

You can learn a lot about what matters to guests from Aaron’s well-chosen list of 163 tags.¹ For now, I’ll just pull out seven personal lessons.

1. Reviews are relative to expectations.

Most negative reviews are due to mismatched expectations because the information online is incomplete or out of date. Room photos don’t show you where you are in the building, or even what building you’re in (see below). They don’t show you the steep stairs to the third floor, or the noise coming from the live performance down in the tavern. You can’t tell there’s no phone or TV or minibar in the room, or what the view is really like. Out of date is worse. The pull-out bed has been removed; someone said there was once a clawfoot bathtub; the event you came for has been canceled; the gelato printed on the menu has been gone for so long that the server laughs and says “I forgot about that!” (June 2023).

I don’t blame the server! It is surprisingly hard to maintain an accurate world-model across signage and websites and such, especially when information propagates to third parties you have no control over. Scrapers and aggregators and old reviews and social media posts can lag by years, and the age of the average post only increases. Some hot new hotel management company (Life House) was advertising using AI to set prices; I’d rather use it to spot contradictions between legible assertions and the observed state of the premises. (Though I don’t think I’d really want that either.)

Sometimes the surprise went the other way, which Aaron called “Unexpected touches” rather than “Mismatched expectations”. People were pleasantly surprised to find real fireplaces; live music; birthday champagne; arrival cocoa; a dog. And they were grateful when the hotel accommodated their unexpected circumstances: helped fix their car, packed a breakfast to go, found substitute evening plans, or accommodated very late or early check-ins.

When you see 4.4 for the Red Lion and 4.4 for Miraval, obviously those are relative to $186 and $1,708, respectively, and prospective guests will naturally factor that in. It’s the surprises that aren’t captured by the price that make you look good or bad for the money. I.e., you get no credit in your rating for things that are obviously connected to how much the guest paid; you get credit for the less-legible residuals of that model. What should the staff do if the power goes out? It’s not defined in (informal) contract of buying a room; thus, it affects your rating.

2. The main house and the lodge are practically separate hotels.

When Claire first toured the property it was obvious that there are two things here, so she didn’t realize that people didn’t realize there were. But how would they? There’s one name; one URL; one address on Google Maps; one infobox on Google; one listing on Booking.com; one average rating of 4.1 stars. Nobody’s really studying the map and literature; they’re relying on the abstraction of “a venue”, “an establishment”, which wraps the physical plant into a point mass.

The result is among the most common (and deeply structural) frustrations: “[I]t was touted as a b&b but make sure you do not stay in the lodge. more like a rundown motel. outdated and musty.”² We considered renaming the lodge the “Summer Annex”, to designate its purpose as seasonal overflow, but it’s a bit of a pain to change across all booking sites so it’s still the lodge for now. And we’ve considered listing them totally separately on booking sites, but that comes with its own overhead, since the service and amenities are all shared.

Before Claire took over, we thought the dissatisfaction was in one direction: main house good, lodge bad. But we have found that the frustration goes both ways. Some people are here for yoga at Kripalu and really just want the most basic quiet place to rest their head and are upset if they accidentally wind up in a quaint “luxe” room above the noisy tavern!

3. One man’s entertainment is another man’s noise.

The last structural frustration was that there are two buildings; this one is that there is one building serving two purposes at odds.

“I am sure that if people had to choose between living where the noise of children never stopped and where it was never heard, all the good-natured and sound people would prefer the incessant noise to the incessant silence,” wrote George Bernard Shaw. That quote hung in my brother’s bedroom growing up; my parents certainly aligned themselves with happy tolerance of chaos. The framed version did not mention it comes a chapter entitled “Children as Nuisances”, which is not an endorsement of incessant noise, but takes a moderate stance on discipline.

Some guests come for a meditative retreat, and some come for live music. Those goals are opposed and it is hard to separate them in space and time. The Ostrich Room can never be a late-night party bar when there are rooms over it, and the hotel can never guarantee a peaceful retreat when it has a live music program. As ever, a lot of this comes down to communication. We now say on the website which rooms are over the bar so the people who want the music can choose the room it leaks out into.

If you sell microwaves or something, your customers don’t interact much; it doesn’t matter to one what another is microwaving. But in establishments like bars, restaurants, social networks, and hotels, part of what you’re selling to people is the other people. But you can’t pick them, and some of their goals are directly at odds. And your margins are low enough that it’s tough to alienate any faction.

The previous owners kept the bar open and raucous until like 2 a.m., and in doing so built an absolutely beloved local institution — and elicited a bunch of negative reviews. Claire’s approach so far has been the opposite: close the bar at 10 and protect the guests’ sleep first. Some patrons came into the bar on Saturday, having heard there’d be live music, and having been to the old incarnation many times; but they came in past 9, missed almost everything, and had to leave. They politely said hi and bye and thanks and good luck to us on their way out; I pried and they confessed they think we need to be open later. Claire responded honestly: the margins on the rooms are a lot better. But we’ll have to keep figuring out this balance.

4. A lot can be forgiven if you establish at the start that you’ll be available to listen.

The previous owners (who took over around peak Covid) went for a futuristic contactless experience. You’d get emails telling you to do mobile checkin so when you showed up there’d just be an envelope with your key waiting for you. The curtains of the front desk were clamped shut.

There is certainly a boom in all sorts of things where people pay extra to not have to talk to people. It seems to work for many Airbnbs. But it seems to depend on a level of predictability and clarity that a quirky rural hotel with various buildings and shifting services can’t attain and shouldn’t attempt. Pretty much the first thing Claire did was open those curtains and take up residence behind the front desk.

And so many people come in frustrated or confused by something but immediately soften with attention. The decisive turn in so many reviews is: a problem arose; did the hotel respond?

Sometimes no: “we lugged all our bags inside before being told we were in a different building. staff on duty didn’t offer any assistance, even though i was alone with two kids and clearly struggling”.³ Sometimes yes: “Our car broke down on their driveway the night we arrived and we were pretty much stranded for the weekend but thanks to James, Daniel and especially Memo who even went to town to get items for the car, we were really well looked after. These people care about their guests and go over and beyond and we are so thankful to them and we look forward to returning!”⁴

Obviously it’s best when the staff can fix your problem, but many reviews talk about mere tone and availability. Does it seem like anyone wants to help? If you don’t feel heard by the establishment, it spills over to a review where you’ll feel heard by the objective reality of seeing your words stuck there, indexed and factored into an average. But the hotel can prevent that spill. Listening is not just a means to provide a service; it also seems to be a service itself.

Tamina serves coffee and calvados to the customers (there aren’t all that many, the room being always half empty) and then goes back behind the bar. Almost always there is someone sitting on a barstool, trying to talk to her. Everyone likes Tamina. Because she knows how to listen to people. But is she really listening? Or is she merely looking at them so attentively, so silently? I don’t know, and it’s not very important. What matters is that she doesn’t interrupt anyone. You know what happens when two people talk. One of them speaks and the other breaks in: “It’s absolutely the same with me, I…” and starts talking about himself until the first one manages to slip back in with his own “It’s absolutely the same with me, I…” The phrase “It’s absolutely the same with me, I…” seems to be an approving echo, a way of continuing the other's thought, but that is an illusion: in reality it is a brute revolt against a brutal violence, an effort to free our own ear from bondage and to occupy the enemy’s ear by force. Because all of man’s life among his kind is nothing other than a battle to seize the ear of others. The whole secret of Tamina’s popularity is that she has no desire to talk about herself. She submits to the forces occupying her ear, never saying: “It’s absolutely the same with me, I…”
— Milan Kundera, The Book of Laughter and Forgetting

5. Politely clustering near four stars gives negative reviews more weight.

Suppose the average guest will give you 4 stars, which is nice of them. The happiest guest will give you 5 stars; the unhappiest will you 1. If you have one 4-star review so far, a 5-star review would raise your average by 0.5, whereas a 1-star review would lower it by 1.5. The negative scale is 3× wider than the positive scale.

This dynamic seems to generalize beyond ratings. In sharp contrast to my main job at a tech startup (where all that matters is becoming the best ever, and anything else is worthless), in a hotel, the payoff curve flattens out faster at the high end of service such that avoiding the bad outweighs pursuing the excellent.

If you go out of your way to stock rare books and special coffee and fresh flowers and take time to do wild one-off favors and errands for people, your reward (at least in short-term business terms) is relatively meager: sometimes people notice, some fraction of them leave a good review, some fraction recommend you to a couple people, some small fraction of that will ever turn into business.

But if you ever slip up — leave the desk unattended, get breakfast out late, let an air conditioner to break, run out of ice, fail to communicate that some furniture or event has changed, run out of chairs — dealing with the complaints and fallout and remediation can quickly consume far more time than you spent on that special mitzvah for that happy guest.

This doesn’t mean that you don’t pursue excellence; it just means that pursuing excellence requires a certain supra-rational faith.

6. A hotel’s static real estate masks its dynamic service.

All of Aaron’s analysis, and all these quotes, come from reviews before Claire took over. How much of it still applies to the hotel Claire operates today?

Architect Frank Duffy’s shearing layers are site, structure, skin, services (in the wiring/plumbing/HVAC sense), space plan, and stuff, of which only stuff has changed: some new furniture and curtains and a near-total turnover of the wall art in common spaces. But he was talking about buildings; a hotel is also a booking model, a food & beverage operation, a concierge service, a staff. Our maintenance, housekeeping, and booking staff have not changed; the rest has.

A hotel is a durable object fixed in space with enormous inertia, but it’s also a bundle of services that are only as good as their most recent day. The real estate wraps Hagai’s chicken schnitzel in a package with the illusion of endurance. But even within a single owner’s tenure, the service level can change dramatically month to month. There are periods when the hotel is vibrant, abuzz, sparkling with effort; and you can tell when it’s up for sale, a lame duck, running on a skeleton crew. Everyone wants to be judged on their best work, when they were really trying — but reviews keep coming in either way, and they all go into the same average.

Although, if you change the name of the hotel, Google wipes the slate clean.

7. The people clamor for ice.

@goldisacks: i'm gonna be real with you. I do not understand ice dispensers in hotels. I have never needed ice in a hotel. Why is it so important for there to be ice in a hotel. Why are ice dispensers mentioned in hotel reviews. What are people doing with ice in a hotel. Am I losing my mind

Well:

• “it would be nice if they had an ice machine in the building for the guest to use”

• “only 1 bar of soap, no ice, no cups”

• “no ice machine in the lodge, had to get that in the inn”

• “an ice machine in the lodge would be nice”

• “there was an ice bucket in the room but no where to get ice, no glasses just paper cups no coffee pot, no tv. when you come into the building there was a small entry room where there could’ve been a ice machine and a comfortable couch maybe a tv there at least”

• “i tried to call to find out where i could get ice without having to walk over to the main building, and was put on hold and hung up on”

• “there was no ice machine so no easy access to ice”

• “no ice machine”

• “all of the staff went out of their way to be warm, welcoming, and helpful, helping us get ice from the kitchen”

• “There was no glassware so we had to go downstairs to get ice and glasses for our room”

• “there was always ice and coffee and tea available”

• “no ice/water in that building”

• “the lodge is not so convenient or well-appointed (no ice machine, dated finishes)”

The lack of ice in the lodge summer annex lodge seems to really make it feel cut off, like a town without a post office. In the absence of a machine, ice becomes a service issue, and staff is limited. What can I say? The people love ice. It makes your drink cold.

---

The average review since Claire took over is a 4.8. I guess I should figure out what the average was in the first two months of the prior owners. But yesterday our overall average went from 4.1 to 4.2.

And yes, we promise there is a pool. It’s wonderfully sited atop a hill; it’s the best view we have. But, indeed, it is not visible from most of the property, and isn’t on the way to anything but a Tanglewood parking lot.

---

Changelog

I haven’t sent a newsletter for a month because there’s been so much “real work” happening.

On June 1, the Ostrich Room opened to the public for drinks. Nobody came, but we had a great time with the new staff (see our intro video). Giovanna had struck up a conversation with Claire at Brava and became our first bartender; she pulled in her friend Nic with whom to alternate. Kari is a veteran of local powerhouse Red Lion who alternately buses tables and wrangles point-of-sale contracts. Finn and Rory are our sassy brother–sister barbacks. At the time, Claire stated she’d have food and music by June 19, the eve of the Tanglewood season.

On June 8 we had our first test run of food and got our first great little crowd. On June 9 we put up a TV in there — the only one on the premises. On June 13 we did a second test run of food — with Hagai, who signed up to stick around.

On June 19 we opened with the full program of drinks and food (Wed – Sat) and live music (Wed & Sat), hitting Claire’s self-imposed deadline on the dot.

We’ve gone through a lot of menu and signage iterations. We added a guest book and a “Draw something banal from work” book. Danny has fully revamped the coffee station. The lodge lobby looks better every time I go in (once a month). Memo hung string lights over the fire pit. And they tore down the circus-tent fabric in the roof of the round room.

We started over from scratch on the marketing website and relaunched it, also on June 19. Claire wrote brutally straightforward bullet points for each room and Christian re-took all the room photos to match the updated decor — a crucial element of setting guest expectations to improve reviews! As of today the website shows the Ostrich Room’s hours and live music calendar.

Claire has eliminated many of the nickel-and-diming fees that reviews complained about, including early check-in and late check-out fees, and I think she folded the resort fee into the base price. We do still have this leftover “mobile pre-check-in” thing that doesn’t work and makes every guest feel like they’ve already done something wrong before they’ve even entered the building.

This morning we hosted our first after-wedding brunch. And with the start of Tanglewood we’ve had our first totally sold-out nights. When last I wrote, we were preparing for the season; now it’s racing by. It feels good taking over a hotel at a time when business will tend to grow week-over-week anyway; now I’m already dreading the feeling of peaking. It’ll take all summer to learn what I wish I’d done for the summer!

Whatever. Tonight I watched the desk and the crowd for the Brandi Carlile concert was fun. I gave an old arthritic couple a ride just down the driveway and I’ve never seen anyone so grateful. Two young women in matching shirts came in hoping the bar was open and left with my bag of potato chips. Everything seemed less stiff. Claire and Danny actually got to go to the concert, and both were beaming about how amazing it feels to walk across the street to something world-class. We live closer than many people park. They were aglow. It was like they finally personally realized the truth of their marketing.

1. In decreasing order of prevalence, Aaron’s tags are: Mismatched expectations, Rooms, Staff, Lodge, Feel/Ambiance, Main house, General Feedback, Communication, Breakfast, Website, Location, Ostrich Room, Decoration, Cleaning, Grounds, Dinner/Lunch, Amenities/Guest essentials, Views, Common areas, Room features, Price, Tanglewood, Bathroom, Broken items, Music/Entertainment, Pool, Furniture, Beds, Reception/Front Desk, Noise, Drinks, Accessibility, TV, Travel motive, Coffee/Tea, Check-in/Check-out, Hotel comparison, Open/Closed Confusion, Booking, Room type confusion , Price comparison, Room comparison, Complaint, Smell, Renovation in progress, Safety, Lenox, Room size, Common area fire place, Neighbors, HVAC, Linens, Unexpected touches, Porch, Menu, Reviews, Weather, Nickel-and-Diming, Paint/Wallpaper, Pests/Bugs, Phone, Switching rooms, Restaurant Comparison, Staff inefficiency, Windows, Ice Machine, In-Room Fireplace, Lighting, Refrigerator, Housekeeping frequency expectation, Emails, Luggage, Room upgrade, Security, Wifi, Bellflower, Fire pit, Shower/Tub, 3rd Party Booking Sites, Fees, Kripalu, Nearby restaurants, Refused to stay/Left before end of stay, Request, Rug, Water cups, Elevator, Hiking, Key Pick-up/Drop-off, Microwave, Outlets, Air quality, Early check-in / Late check-out, Exercise, Family, Keys, Refund, Transportation, Carpet, Reputation, Room balcony, Cancellations, Discount, Stairs, Toilet, Parking, Resort Fee, Shopping, Stockbridge, Tall people, Tennis, Baby, COVID, Children, Exhaust Fan, Food delivery, Older adults, Place for toiletries, Shampoo/Conditioner/Bodywash, Sink, Snacks, Takeout, Tissues, Wedding, Clock, Deck, Driving, Meal Reservations, Meal Substitution, Museums, Outdoor activities, Plowing, Radio, Soap, Temperature, Wayfinding, Bottled water, Cell reception, Coat hangers, Colonial Theater, Deposit, Drain, Electricity, Great Barrington, Hairdryer, Ice bucket, Incidentals, Ironing, Jacob's Pillow, Leak, Locked buildings, Luggage rack, Mass MoCA, Mirrors, Newspapers, Nightlight, Pets, QR code, Return guest, Scenery, Shower cap, Tap, Vegetarian.

2. More: “there are two buildings. i was placed in the building away from the main inn. the common area in the building was substandard”; “our room was in one of the other buildings - very dirty - dust everywhere - the ey probably forgot to clean from dust since fall or summer”; “i think you get what you paid for, but main building is night and day with this room”; “did not know that we would not be in the main house when we booked the stay”; “it was difficult for us going back and forth up the second hill to the annex houses as we are all senior citizens. would just recommend finding out where your room is located before your book”; “rooms in the separate building are motel style, not as advertised on site”; “the main house is a gorgeous piece of century-old architecture but the contrast in character between the main house and the lodge is glaring. the main house is like staying in a piece of history while the accommodations at the lodge are bested by motel 6. i would love to come back to this property , but only if the room was in the main house or the lodge is significantly improved.”

3. More: “We couldn’t notify anyone in the building. there were NO STAFF anywhere and no phone number”; “no staff at reception desk”; “but it was hard to find staff at check-in. having some clear directions on how to get help near the key return and maybe having people wear a uniform top or name tag so that the staff doesn't blend in with the guests would be a bit better”; “when i got there + went in, i made 3 laps of the first floor looking for someone to speak with. (there is no registration desk)”; “the staff person seemed to notice me look in the window but didn’t offer to let us in and have a look.”

4. More: “the staff did their best to accommodate me, switching my room to a less musty one and calling the people above me to be more quiet with running and jumping”; “the manager took my allergies into account and were able to make things safe for me to enjoy”; “we had a quite late check-in but they were super accommodating and did their best to receive us”; “we were later than expected, but staff (ive forgotten her name) stayed late, was very kind and got us settled”; “when the electricity went out saturday night, the bar in the main lodge fortunately was open to purchase drinks. the candle light created a festive atmosphere until the lights came on.”