Tester Kit: agents run the tests, people decide

The problem

The Tester team tested 100% manually. Most members are not strong technically, so they could not write automation the traditional way. Each person finished at most 3 to 4 tasks a day, with not enough time to re-run old cases, while the Dev team was already using AI and shipping features faster than they could be tested.

Before Tester KitOld bugs came back because no one had time to re-run old cases

• Recurring bugs
• New bugs

• Bugs in one moduleabout 30% were recurring bugs

  3070

Context and role

• A Tester team of four, serving several projects in parallel.
• Test cases were kept in spreadsheets, with many cases repeated between releases.
• I designed the process, built the skills, trained the team and tracked adoption, alongside my BA role.
• Constraints: testers do not write code; agents must not decide pass or fail on their own; every result must have evidence.

Step 1: Classifying the test cases

Not every case should go to an agent.

ClassificationWhich cases go to agents and which are tested by hand

	Example	Given to agent	What the person does
API and data checks	Endpoint returns the right code, database is in the right state	Yes	Verifies the report
Repetitive, stable UI flows	Sign in, create a record, filter a list	Yes	Verifies the evidence
Multi-step business flows with calculations	Partial payment, then check receivables	Yes	Verifies each step and the figures
Look and feel, user experience	Layout, colours, messages	No	Tests by hand
Third-party integration, real payments	Webhooks, payment gateway	No	Controlled manual testing
Exploratory	Finding defects outside the scripts	No	A skilled tester

Step 2: Designing Tester Kit

The kit has 8 skills, packaged so that a tester only needs to install it in the IDE to run the automated cases, with no code to write. The representative skills form a chain:

Tester KitOne job per skill, one output per job

1. 1Generate test casesfrom stories, acceptance criteria, business rules
2. 2Pre-run checkenvironment, sample data, test accounts
3. 3Run a case seton staging, screenshots, logs
4. 4Regressionre-run old cases for the module that changed
5. 5Draft bugsteps to reproduce, expected, actual
6. 6Collect evidencefolder to attach to the ticket

RulesApplied to every skill

• The agent gives only a preliminary result: looks like a pass, looks like a fail, or not sure.
• Every result must come with evidence: screenshots, logs, actual values.
• Do not touch production data. Use test accounts only.
• The final pass or fail decision belongs to the tester.

Step 3: Running in parallel

Each tester opens 10 to 15 IDE agent tabs (Claude Code, Codex, Antigravity), and each session runs one group of cases. The agents run first while the tester watches the summary sheet.

A tester's day

• Tester
• Agent

1. 1TesterSplit cases into groups
2. 2AgentRun in parallelone group of cases per session10–15
3. 3AgentPreliminary resultswith evidence
4. 4TesterVerify the evidence
5. 5TesterManual testingfailed cases, cases that cannot be automated
6. ?TesterQA decision

Step 4: The “looks like a pass” situation

The simulated example below shows why a person still has to verify. Case: record VND 1,000,000 against an invoice of VND 3,000,000; receivables must go down by exactly VND 1,000,000.

Step 5: The tester’s new role

The tester goes from doing everything by hand to verifying the results of what AI has done.

New role

1. 1Design test casesAI produces the draft
2. 2Choose cases for agents
3. 3Coordinate agent sessionswatch the summary sheet
4. 4Verify the evidencetest failed and uncertain cases by hand
5. ?Decide pass or fail
6. 6Bug report with evidence

Results

By internal measurement:

• Before
• After

• Recurring bugs in the same module✓ 30% → 5%

  30%
  5%

On the US client project, counted from the project documentation:

The kit has been handed over to testers, including one on Ubuntu, with a guide for non-technical users.

What I learned

• People who do not write code can still direct automated testing if the tool speaks their language: test cases and evidence.
• The agent is not the one who decides pass or fail. The tester remains responsible for verifying the test results.
• Automation is most valuable where it frees up time for the work that needs people: hard cases and exploratory testing.