QA Testing vs. Game Onboarding Research: Know Which One to Run and When
Photo by on Unsplash
QA catches bugs. Onboarding research catches confusion — and confusion is what kills your Day 1 retention. Most studios are disciplined about quality assurance, yet they ship with a first-time user experience that loses 40 to 60 percent of new players inside the first session. The reason is almost never broken code. It's a tutorial that assumes too much, a mechanic explained too late, or a moment where a new player simply doesn't know what they're supposed to do next.
These are two genuinely different problems that require two genuinely different approaches. Getting clear on which one you need, and when, saves money, saves time, and keeps your launch from turning into a post-mortem.
What QA Testing Actually Does (and Doesn't Do)
Quality assurance is about verification. Does feature X work as specified? Does the game crash on this hardware? Does the save system behave correctly across edge cases? QA testers follow defined test plans, look for deviations from intended behavior, and log reproducible bugs.
It's indispensable work. According to Sentient Gaming's analysis of functionality testing, consistent QA coverage directly contributes to player retention by ensuring features perform as expected. Players don't forgive crashes and corrupted saves. Those bugs will tank your reviews.
But QA has a firm boundary. It tells you whether the game works. It cannot tell you whether the game makes sense to someone who has never played it before. A QA tester following a test plan already knows what the mechanic is supposed to do. A brand-new player on launch day does not.
What Onboarding Research Actually Does
Onboarding research, sometimes called first-time user experience (FTUE) testing, is about observation. You put real players who match your target audience in front of the game for the first time, with no hand-holding, and you watch what happens. Where do they hesitate? What do they misread? Where do they give up? What do they try that you never expected?
The goal isn't to find things that are broken. It's to find things that are unclear. Those are completely different failure modes, and they require different people running the session with different goals in mind.
A useful framing from Antidote's breakdown of QA versus playtesting: QA asks "does this work?" while playtesting asks "does this feel right to someone playing it for the first time?" Both questions matter. Neither one answers the other.
The Onboarding Moment That Actually Matters
The first five to ten minutes of your game are doing more work than any other segment. A player is forming their first impression of your controls, your tone, your genre conventions, and whether they personally belong in your game's world. If anything in that window creates friction, most players will not push through it. They'll just stop.
This is where onboarding research earns its keep. Some specific things it surfaces that QA will never find:
- Assumed knowledge. Your team has played this game for months. New players haven't. Mechanics that feel obvious to you are genuinely confusing to a fresh set of eyes.
- Tutorial sequencing problems. Information delivered in the wrong order creates confusion even when every individual element is correct.
- Interface hesitation. Players pausing or clicking around without direction signals a navigation problem that doesn't show up in any bug log.
- Emotional mismatch. A player who expected one tone and got another will disengage fast. You can only catch this by watching real reactions.
Screen recordings, think-aloud sessions, and post-session interviews with players who genuinely fit your target persona will show you things that internal testing simply cannot replicate. Your team is too close to the product.
When to Run Each Type of Testing
The short answer is: run both, but on separate tracks, starting earlier than feels comfortable.
Usability research consistently shows that the cost of fixing a problem rises steeply the later it's discovered. A confusing FTUE sequence found in alpha takes a few days to redesign. The same problem found the week after launch is already embedded in a thousand one-star reviews.
A rough framework by stage:
Concept and pre-production: This is too early for formal QA, but it's a good time for early concept feedback sessions. Can players understand the core loop from a prototype? Do the genre signals land?
Alpha: Start targeted onboarding research here, even with rough builds. Watch a handful of players attempt the first session cold. You'll learn more in two hours of observation than in a month of internal debate.
Beta: QA ramps up in earnest. Onboarding research should be running in parallel, not wrapping up. This is when you validate the fixes you made from alpha findings. Did the revised tutorial actually solve the confusion, or just shift it?
Pre-launch: Final QA regression cycles. One more onboarding research pass with fresh participants who haven't seen any previous version. What you find here still has time to ship as a Day 1 patch if needed.
The Recruiting Problem Nobody Talks About
One reason studios skip proper onboarding research isn't laziness. It's the friction of finding the right participants. QA can use internal testers who know the game. Onboarding research requires people who have never seen the game and who match your actual player profile. If your game targets mid-core mobile players who prefer strategy, you need those people specifically, not just anyone willing to sign up.
This is exactly where generic recruiting falls apart and why studios often default to running sessions with whoever is available, which produces misleading results. A participant who doesn't match your audience will have friction for the wrong reasons, and you'll chase false problems.
Getting the recruiting right is half the value of the research. If building that pipeline in-house feels like too much overhead on top of an already full production schedule, it's worth considering a research partner who already has those networks in place. The VGM research services team specializes in finding and vetting specific player audiences so studios can focus on acting on findings rather than chasing down participants.
The Practical Takeaway
You don't have to choose between QA and onboarding research. You have to stop treating them as the same thing. QA keeps your game from being broken. Onboarding research keeps your game from being abandoned.
Run them on parallel tracks. Start both earlier than feels necessary. Recruit real players who match your target audience for the onboarding work, and don't let proximity bias from your own team substitute for genuine first impressions.
Day 1 retention problems are almost always visible in your onboarding data weeks or months before launch. The studios that catch them early ship with confidence. The ones that don't spend the first two weeks post-launch trying to understand why players aren't coming back.
If you want to run proper onboarding research without the recruiting headache, reach out to the VGM team. We'll help you set up sessions with the right players at the right stage, so you get findings you can actually act on before it's too late to act on them.
FAQ: QA Testing vs. Onboarding Research for Games
What is the difference between QA testing and onboarding research in game development?
QA testing verifies that game features work as specified and finds bugs. Onboarding research observes how new players experience the game for the first time to identify confusion, friction, and points where players disengage.
When should a game studio start onboarding research?
Alpha is a practical starting point, even with rough builds. Early sessions with a handful of target-audience players will surface structural onboarding problems while they're still cheap to fix.
Can internal testers run onboarding research?
Partially, but with significant limitations. Internal testers already know the game, which means they can't replicate the genuine first-time experience. Onboarding research requires participants who have never seen the product before and who match the target player profile.
What does FTUE mean in game development?
FTUE stands for First-Time User Experience. It refers to everything a player encounters during their initial session, including tutorials, UI introduction, control onboarding, and early narrative framing.
How does onboarding research reduce Day 1 churn?
By identifying the specific moments where new players hesitate, get confused, or quit before the first session ends. Fixing those moments before launch directly improves early retention metrics.