Can AI replace user research interviews?

AI-moderated interviews can uncover customer needs at a much lower cost; a 2026 pre-registered study found comparable depth and more needs found on the same budget. That isn’t the same as replacing every interview. Human moderators still matter where rapport and sensitive disclosure count.

What are synthetic users?

Synthetic users are AI-generated stand-ins for research participants. Studies in 2026 found they often reach the same headline conclusions as real people but miss much of the variation, which is where unexpected insights usually come from.

What is the risk of using AI to synthesize user research?

The main risk is treating a confident AI summary as a validated finding. AI can help find themes and patterns, but someone still has to judge how strong the evidence is and own the conclusion.

How should research teams use AI responsibly?

Use AI to expand reach, run interviews and help with synthesis. Keep people accountable for study design, checking who actually responded, and deciding what the findings justify and how certain they are.

AI Can Ask the Questions. It Can’t Tell You What the Answers Mean.

AI has made it cheap to ask more people more questions, and research teams should use it. It can’t take responsibility for what the answers mean.

AI-moderated interviews are moving from experimental tools into established research platforms. Remesh launched its AI interviewer last month, and Maze now runs AI-moderated studies alongside its human-moderated ones. AI synthesis usually sits right next to them, ready to turn a stack of transcripts into findings.

AI is entering user research generally in three distinct ways: interviewing people, generating synthetic respondents, and analyzing research data. The first expands your access to human evidence. The second creates simulated evidence. The third helps you interpret evidence you already have. They get discussed as one thing, but they raise different questions about what a team can legitimately conclude. AI can help with all three, but it can’t take responsibility for what the evidence implies, even when its output makes that responsibility tempting to hand off.

What AI changed

The advantage is significant. Researchers at Columbia and Wharton ran a pre-registered study of 317 people comparing AI-moderated interviews, human-moderated interviews and fixed questionnaires (arXiv). Interview for interview, the AI moderator uncovered as many customer needs as experienced human moderators, at about a fifth of the cost. On the same $5,000 budget, AI interviews surfaced 238 distinct customer needs. Human interviews surfaced 109.

It’s the same for day-to-day research work. Synthesis that used to take a researcher days can now take hours. Reading 500 customer comments instead of 50 stops being a heroic effort and becomes a Tuesday project. AI lets you ask more people more questions, more often, for less. Teams that refuse it on principle are missing a very advantageous tool.

Where it breaks

The same Columbia study had a second half to it. The researchers used the interviews to build “digital twins” of each participant. These were AI models meant to predict how that person would react to marketing they hadn’t seen yet. The richer AI interviews didn’t produce better quantitative predictions than the simple questionnaires did. And the twins reasoned differently from the people they modeled. Most real people reacted to images and video on instinct, while the twins explained their reactions analytically, like robots.

Nielsen Norman Group found a related problem from a slightly different angle (NN/g). They gave 120 AI-generated respondents a question they’d already asked 604 real human professionals. Both groups reached the same headline answer. But the humans used more than 50 distinct terms in their answers, and the synthetic respondents stayed around a dozen. The unusual answers, the ones that might point to something that could give unique and interesting insights, were largely missing.

Neither study shows that AI can’t help you analyze real data and give meaningful insights. It often can by clustering themes, pulling supporting quotes, comparing segments, spotting complicated patterns you’d miss. What the studies show is that AI output can look like a complete picture while leaving out the range of different answers real people gave. In product research, those differences are often where the interesting decision gets made. The common answer can lead to confirmation bias, but the outlier can reveal where your assumptions about the product are wrong. Or not. Either way, it takes judgment to decide which differences matter.

The respondents are using AI too

There’s a newer, more frustrating problem that’s been popping up lately as well. AI isn’t only asking the questions. It may be answering them.

This summer I ran two small unmoderated rounds of research for a product I’m developing for independent financial advisors. The panel was targeted at financial advisors. I built in a set of questions designed to check whether respondents actually were who they said they were. I asked how they described their practice, which software platforms they currently used, how a real account transfer worked.

Of seven respondents, one ended up plausibly being a real advisor. The rest included adjacent professions, one who answered as a custodian and one who described their tools as “Microsoft.” Two participant answers were phrased so similarly that I questioned whether they’d been written independently. AI assistance was one possibility. I suspected it, but I couldn’t establish it, and I didn’t need to. Without those checks, I would have had seven “advisors” and a confident-sounding synthesis of useless data.

A finding is only as credible as the people, source and evidence behind it, whether or not AI was involved. Cheaper participant data doesn’t just bring you more signal. It can bring you more noise that looks like signal.

What stays human

If AI is good at reach and increasingly capable at analysis, the human role doesn’t need to disappear. It concentrates on knowing which conversations need a person, and owning the conclusions.

For example, in the previously mentioned Columbia study, people sounded measurably more positive, more energized and more assertive talking to a human interviewer than to the AI, even though the content and questions were comparable. The authors noted that human moderation keeps its advantage where rapport and sensitive subjects matter. In fintech, wealth management and estate planning, this is highly relevant. People don’t tell a random interviewer the real story about their money until they trust the person interviewing them.

As for owning conclusions, User Interviews surveyed 150 researchers in May about synthetic users (User Interviews). Eighty percent use AI regularly in their work. Their top concerns were the quality and accuracy of the insights (88%) and stakeholders over-trusting AI-generated findings (79%). And 63% said their team had no guidance at all on using synthetic users or how to integrate them into their workflows. In other words, most organizations haven’t made clear decisions on how much to trust what AI produces and how it gets integrated into product design.

What owning the conclusion looks like

The rule I like to work by at a high level is pretty straightforward. AI can inform the judgment, but a person owns the conclusion. It helps to separate three things that can get blurred:

A finding describes what the evidence supports.
An interpretation explains what it might mean.
A decision determines what to do about it.

AI can contribute to each, but deciding which is which is still a person’s job. In practice, I think that ownership comes down to a few habits:

Decide what “done” means beforehand. Before I did a second round in my study I mentioned above, I wrote down what would count as a “gate” to move forward. It needed to include at least two credible respondents describing the problem I was trying to solve, and at least one repeating pattern. Then, if the second round contradicted the first, the contradiction would mean I needed to reassess my hypothesis.

Check who you actually heard from. A few questions that test whether people are who they say they are cost almost nothing to add. In my study, I found a problem that no amount of synthesis would have.

Label the strength of every claim. At the end of my research, the pattern repeated, but I had one credible respondent, not two. I closed that phase of discovery anyway, on my own judgment, and wrote it down as a decision, not a finding. I still can’t say for sure the problem is real, and I haven’t tested whether anyone would pay to solve it. There’s more to do with the testing, but both are written down exactly that way to avoid bias debt later.

Write down the judgment calls. AI synthesis will happily turn thin evidence into a confident paragraph. A person accountable for the conclusion has to explain why the evidence supports it.

Where this argument doesn’t hold

AI interpretation will get better. It probably will, quickly. But the case for ownership doesn’t depend on AI being bad at analysis. Even a perfect summary needs someone accountable for acting on it.
Humans interpret badly too. We do. Researchers hear what the roadmap wants and ask leading questions. The difference is that a person can be questioned, challenged and held to their reasoning. That’s an argument for checking human judgment, not for removing it.
Most teams can’t afford human research at all. True, and for them AI research beats no research. A solo founder reading 200 AI interviews is still the person who owns what they conclude.
The evidence is thin. The Columbia study covered one product category, one AI platform and only 24 human-moderated interviews, and the authors say so. The User Interviews survey is a self-selected sample. My own study was seven respondents and is an illustration, not proof. But the sources point the same way from different methods, which is why I thought this was an article worth writing.
People are sometimes more candid with AI. Some research suggests participants may disclose sensitive things more readily when they don’t fear judgment. That’s an argument for choosing the interviewer deliberately for each study, and that’s a decision a person should make.

Scale the research. Own the conclusions.

Use AI to talk to more people than you ever could before. Let it help with the synthesis. Then do the part AI can’t: decide what the evidence justifies, say how sure you are, and put your name on it.