Decide what you need to learn
Write down three to five questions the prototype must answer. Do users understand what it can help with? Does it find the right information? Can specialists check the output without substantial extra work? What do users do when an answer is incomplete?
Do not try to resolve user needs, model quality, every integration and the complete production architecture in one test. Choose the questions that determine whether the idea deserves the next investment.
Build only what the test requires
If you want to test whether customers understand the service, a clickable prototype with prepared responses may be enough. To test whether a model retrieves the right facts, you need a technical prototype with a limited set of sources. To measure time spent, the system must more closely resemble the tool employees use today.
Tell participants what is real and what is simulated. Otherwise they may assess technical qualities the prototype does not have or spend time on details that are outside the purpose of the test.
Include both routine and difficult cases
Show the complete process for a few selected cases, from the user's first action until the output is checked or passed on. Test what can go wrong, not only the example that works best.
- A common case the system should handle well.
- A question that lacks necessary information.
- Sources that conflict or are out of date.
- A case that must be handed to an employee.
- A way to correct, reject or start again.
Test with the people who will use it
Let employees or customers try the prototype on tasks and situations they encounter in practice. Ask them to use it as they normally would while you observe. Watch where they become uncertain, which information they need and what they have to check before proceeding.
Measure both quality and effort. A draft that looks good but needs extensive fact-checking and rewriting may be worse than the current method. Record how long the task takes, the errors that occur and how much of the output can be used.
Use the results to improve the idea or stop
After the test, you should be able to explain what worked, what needs to change and which questions remain. The next action may be improving the sources, changing the interface, trying a different technical approach, running further tests or ending the trial.
I can design and build the prototype, plan the test and improve the system with your team. You will then have a sound basis for further development, rather than a demonstration designed only to look promising.