Three Harvard Business School professors ran GPT through a set of willingness-to-pay surveys the same way a consumer insights team would. Some of the numbers that came back werent just off. They were wrong-signed, meaning the model predicted people would pay more for a feature when real human respondents said the opposite.

That result comes from James Brand, Ayelet Israeli, and Donald Ngwe, whose working paper “Using LLMs for Market Research” (HBS Working Paper 23-062, revised 2026) is one of the more careful academic tests of a question a lot of executives have already answered for themselves without any data: can an LLM just tell you what your customers want.

The paper says: sometimes. Not reliably. And the failure mode isnt random noise, its a specific, structural blind spot.

What the model got right, and exactly where it broke

The researchers found baseline LLMs could reasonably approximate willingness-to-pay for features that already existed in a product category. Ask the model about a feature customers have been buying for years, and it does an okay job guessing what they’ll pay for it.

New features are where it fell apart. Ask the same model to predict demand for something genuinely novel, a feature with no real-world pricing history for it to pattern-match against, and the estimates got shaky. In some cases the direction flipped entirely, the model said “customers will pay more” when they would actually pay less.

The fix the professors found wasnt more prompting cleverness. It was fine-tuning the model on prior human survey data from comparable products and customer groups. Once grounded in real human responses, accuracy improved. Without that grounding, the model was guessing with confidence, which is the most dangerous kind of guessing.

This isnt really a market research story

Market research is just the domain where someone bothered to run the controlled test. The underlying assumption the professors were probing is much bigger: that an LLM’s output on a question of human preference or judgment can stand in for actually asking humans.

That assumption shows up everywhere executives make calls today. Pricing decisions. Positioning a new product line. Reading “signal” out of a chatbot transcript about whether a hire is a culture fit. Ranking which market to expand into first. None of those are market research surveys, but all of them lean on the same premise the HBS paper just stress-tested and found breaks under novelty.

Some willingness-to-pay estimates from the models werent just inaccurate. They pointed in the wrong direction entirely.

Where this actually bites a company

A pricing team that fine-tunes a model on five years of real customer data, then asks it to forecast demand for a feature adjacent to what it already knows, is doing something close to what the HBS paper validated works. A team that opens a chat window, describes a brand-new product concept, and takes the model’s confident answer as market signal is doing the thing the paper found fails, and fails silently. No error message. No confidence interval. Just a wrong number that looks exactly like a right one.

Thats the gap most companies havent mapped yet: which of their AI-assisted decisions are grounded in real prior data, and which ones are just a model pattern-matching on vibes and calling it judgment. Figuring out that distinction across pricing, hiring, and strategy, not just marketing surveys, is exactly what an AI Assessment for companies is built to surface.

The professors’ recommendation is blunt: AI should augment human research, not replace it. Every executive currently treating a model’s output as a finished answer instead of a first draft is running the experiment the HBS paper already ran. They just dont have the paper’s control group.

Key conclusions:

• Plausible is not the same as accurate. LLM estimates sometimes resembled human willingness-to-pay, but were often materially wrong—and occasionally had the wrong sign.
• There is no reliable warning when an answer is wrong. Without a human benchmark, researchers cannot distinguish a good synthetic estimate from a convincing failure.
• Fine-tuning works mainly within a narrow neighborhood. Human survey data improved predictions for new features in the same product category and population.
• The gains do not transfer broadly. Fine-tuning on one category did not reliably improve another and sometimes made performance worse.
• LLMs reproduce averages better than market segments. The models failed to recover meaningful differences by income, gender, and political affiliation, even after augmentation.
• Model choice and prompting materially affect the result. Different LLMs generated substantially different preference estimates, so findings cannot automatically be transferred between models or releases.
• The strongest workflow is “human data first, AI extrapolation second.” Use genuine research to establish the market, then use a tuned model to screen adjacent concepts or features before commissioning another study.
• The business value is speed and cost—not replacement. Synthetic studies can run in hours at low cost, making them useful for prioritization and hypothesis generation, but not for final pricing, segmentation, or launch decisions.

My sharpest interpretation: LLMs do not eliminate the need for market research. They make existing market research more reusable. Their value increases when a company already owns relevant, high-quality customer data.

One important caveat: this is an April 2026 working-paper revision, but most principal experiments used GPT-3.5 Turbo in 2024. The precise performance of today’s models may differ; the paper’s deeper warning about validation and generalization remains highly relevant. Read the HBS working paper.