Playground
How to Test an AI Agent in KriyaOS
Testing is how you learn whether your agent is actually behaving the way you expect.
In KriyaOS, the playground is the place where you try questions, observe replies, and decide what needs improvement.
Testing Mindset
The goal is not to prove your agent is perfect. The goal is to notice what is already working, what still feels weak, and what you should improve next.
What the KriyaOS Playground Is
The playground is your testing space.
In plain language, it is where you talk to your agent before deciding whether the current version feels ready.
It helps you move from:
- “I configured the agent”
to:
- “I know how the agent actually responds”
Why Testing Matters
Testing matters because an agent can look complete in setup and still respond in ways you did not expect.
For example, the agent might:
- answer clearly
- miss an important fact
- sound too generic
- misunderstand the role you wanted it to play
Testing helps you catch these things early.
When To Test
Testing is useful both before and after publishing.
Before Publishing
Test before publishing when you want to:
- check the draft version
- review important answers
- catch mistakes before the agent goes live
After Publishing
Test after publishing when you want to:
- confirm the live version still feels right
- review how the agent behaves after updates
- make sure the published experience matches your expectations
How Testing Usually Works
The basic loop is simple:
- Ask the agent a question
- Read the response carefully
- Notice what feels correct or incorrect
- Improve the agent
- Test again
This is the most helpful mental model for the playground:
test → observe → improve → repeat
What To Look For When Testing
When you test your agent, focus on a few practical things.
Correct Answers
Ask:
- did the agent answer the question correctly?
- did it stay on topic?
- did it sound like the kind of assistant I wanted?
Missing Knowledge
If the agent gives vague or incomplete answers, it may not have the right knowledge yet.
This is especially common when:
- no knowledge was added
- uploaded knowledge is still processing
- the information provided was too limited
Unexpected Responses
Sometimes the agent answers, but not in the style or direction you expected.
That often points to:
- prompt or instruction issues
- unclear role definition
- missing context
How Agent Testing Connects to Knowledge
Testing is one of the easiest ways to notice whether knowledge is helping.
If the agent cannot answer specific questions about your business or content, test results may be showing that:
- the right knowledge was not added
- the knowledge is not ready yet
- the question is outside the information you provided
How Testing Connects To Agent Configuration
Testing also helps you check whether the agent setup is working as intended.
For example, it can reveal whether:
- the system prompt is clear enough
- the agent role is too broad or too vague
- the response style feels right
Even when the knowledge is good, the configuration still shapes how the agent behaves.
What To Do If Responses Are Wrong
If responses feel wrong, do not panic and do not assume the whole agent failed.
Start with simple checks:
- Was the answer factually correct?
- Did the agent have the knowledge needed?
- Was the knowledge ready to use?
- Does the configuration need refinement?
If the issue seems knowledge-related, read Troubleshooting Knowledge.
A Good Way To Test
A practical testing habit is:
- start with a few simple expected questions
- then try realistic user questions
- then test one or two edge cases
This gives you a clearer picture than only testing one perfect example.
A Helpful Mindset
Testing is not about proving the agent is perfect.
It is about learning:
- what already works
- what still feels weak
- what to improve next
That is how agents become more reliable over time.