Automated tests
Set up and quickly test inputs or training data using Automated tests in NLX
An application build is required before any Automated test may be used. See Deploy.
What are Automated tests?
Automated tests allow you to easily test your application's performance with recognizing user intent and matching to the correct flow (based on your provided descriptions or training phrases), or they allow you to quickly execute saved conversation inputs when running frequent tests with your conversation workflows.
AI tests (intent recognition)
To test your application's ability to recognize user intent and invoke the appropriate flow from provided training phrases, create an AI test on your application:
Click Automated tests tab of your application > Under AI tests section, select Create test
Enter a test name > Click Save
Select + Add test
Select ✨Generate to ingest training data of a flow(s)*
Alternatively, you may skip this option and manually enter an utterance(s) to test
Review or specify the flows you want to test each utterance against
Select Save
*If your flow's training data includes custom Slots, the Generate combinations toggle ensures all variations of those slot values are simulated in the test.
Once satisfied and ready to run your test, select the Test executions tab to process your test.
Execute test
To execute a saved intent recognition test, select the test:
Choose Test executions tab > Click Execute test
Once the test completes its run, you may select it to view detailed results
Failed phrases display the expected flow to be matched alongside the actual matched flow:
Automated test Settings
Manage custom settings for all AI intent recognition tests created and executed for your application. Select or create an AI test saved on your application, and choose the test's Settings tab:
Execute during build: Automatically triggers your test after each new successful build
Failure threshold: Indicates the percentage of tests allowed to fail in the whole test suite before a build fails
Minimum confidence score: Minimum confidence score indicates the NLP confidence value each test must achieve before it fails. This applies to every individual test
To delete an automated flow recognition test, select the Delete application test option under Danger zone.
Conversation flow test
To create an automated conversation flow test, begin by entering Test mode on a flow:
Select a flow
Toggle mode to Test on the bottom of your Canvas builder
Provide inputs in the test panel to converse with your application
To save your sequence of inputs, select the play button ▶️ icon > Choose + New automated test
Enter a test name
Click Save
Your named automated flow test will now be visible to replay. You may proceed from the point of the save and continue testing the conversation or generate additional automated tests.
Last updated