Automated tests

Set up and quickly test inputs or training data using Automated tests in NLX

What are Automated tests?

Automated tests allow you to easily test your application's performance with recognizing user intent and matching to the correct flow (based on your provided descriptions or training phrases), or they allow you to quickly execute saved conversation inputs when running frequent tests with your conversation workflows.


AI tests (intent recognition)

To test your application's ability to recognize user intent and invoke the appropriate flow from provided training phrases, create an AI test on your application:

  • Click Automated tests tab of your application > Under AI tests section, select Create test

  • Enter a test name > Click Save

  • Select + Add test

  • Select Generate to ingest training data of a flow(s)*

    • Alternatively, you may skip this option and manually enter an utterance(s) to test

  • Review or specify the flows you want to test each utterance against

  • Select Save

*If your flow's training data includes custom Slots, the Generate combinations toggle ensures all variations of those slot values are simulated in the test.

Execute test

To execute a saved intent recognition test, select the test:

  • Choose Test executions tab > Click Execute test

  • Once the test completes its run, you may select it to view detailed results


Automated test Settings

Manage custom settings for all AI intent recognition tests created and executed for your application. Select or create an AI test saved on your application, and choose the test's Settings tab:

  • Execute during build: Automatically triggers your test after each new successful build

  • Failure threshold: Indicates the percentage of tests allowed to fail in the whole test suite before a build fails

  • Minimum confidence score: Minimum confidence score indicates the NLP confidence value each test must achieve before it fails. This applies to every individual test


Conversation flow test

To create an automated conversation flow test, begin by entering Test mode on a flow:

  • Select a flow

  • Toggle mode to Test on the bottom of your Canvas builder

  • Provide inputs in the test panel to converse with your application

  • To save your sequence of inputs, select the play button ▶️ icon > Choose + New automated test

  • Enter a test name

  • Click Save

Your named automated flow test will now be visible to replay. You may proceed from the point of the save and continue testing the conversation or generate additional automated tests.

All saved automated flow tests will be visible from the test panel in the flow or from an application's Automated tests tab where the flow is attached.

Last updated