LogoLogo
  • GETTING STARTED
    • Welcome to the NLX platform
    • How NLX works
    • Guides
      • Analytics dashboard
      • Chat
      • Generative Journey (Slots)
      • Model Context Protocol
      • Voice
      • Voice+
      • Touchpoint components
        • Carousel modality
        • Video modality
    • Terms & concepts
    • Generative AI
    • Developer
  • Build
    • Workspace setup
    • Flows
      • Intro to flows & variables
      • The Canvas
      • Flow settings
      • Nodes
      • Flow appearance
    • Resources
      • Actions
        • Implementation
        • Request model
      • Analytics tags
      • Context variables
      • Data requests
        • Implementation
        • Response model
        • Request model
      • Knowledge bases
        • Ingest content
        • Add metadata (beta)
        • Apply KB
      • Lifecycle hooks
        • Implementation
      • Modalities
      • Secrets
      • Slots (custom)
        • Adding values
        • Translating slots
      • Voice+ scripts
        • Add + download script
        • Deploy script + install SDK
        • Create Voice+ flow
    • Integrations
      • Channels
        • Alexa
        • Amazon Chime SDK
        • Amazon Connect
        • AWS End User Messaging SMS
        • AWS End User Messaging Social
        • Bandwidth
        • Genesys
        • Twilio
        • Zendesk Sunshine
      • LLM services
        • Amazon Bedrock
        • Anthropic
        • Azure OpenAI
        • Cerebras
        • Cohere
        • Google Vertex AI
        • Groq
        • NVIDIA
        • OpenAI
        • xAI
      • NLP
        • Amazon Lex
        • Google Dialogflow
        • Google Dialogflow CX
        • Custom NLP
    • Translations
  • Test & Deploy
    • Applications
      • Attach flows
      • Assign default behavior
      • Add channels
        • API
          • REST API
        • Alexa
        • Amazon Chime SDK
        • Amazon Connect
        • AWS End User Messaging SMS
        • AWS End User Messaging Social
        • Genesys
        • Genesys SMS
        • Messenger
        • Microsoft Teams
        • Slack
        • SMS via Bandwidth
        • Twilio SMS
        • Twilio Voice
        • WhatsApp via Twilio
        • Zendesk Sunshine
      • Deploy
      • Optional: Set lifecycle
      • Optional: Set languages
    • Test
      • Test a conversation
      • Automated tests
      • Test an external integration
  • Analyze
    • Conversations
    • Analytics
      • Creating dashboards
      • Formulas & multi-metrics
      • Canvas analytics
    • Training
  • Workspace Settings
    • Escalation channels
    • Resource tags
    • Audit
  • Admin
    • Access control
      • Roles & permissions
    • Notifications
    • FAQ
    • Contact support
Powered by GitBook
On this page
  • What's a Modality?
  • Checklist
  • Pre-setup: Integrations
  • Step 1: Set up a modality
  • Step 2: Construct a flow
  • Step 3: Deploy an application
  • Step 4: Set up a webpage
  • Step 5: Test the modality
  1. GETTING STARTED
  2. Guides
  3. Touchpoint components

Video modality

From start to finish, set up a chat-based application ft. a custom video modality with NLX

Last updated 26 days ago

What's a Modality?

A modality allows for information to be transmitted to a user that goes beyond a standard message. Images, audio, structured text, haptic signals, videos, and more can be modalities you may want to provide users during conversation.

In chat applications, videos are useful for providing users with rich visual information that can explain, demonstrate, or enhance a user's search.


Checklist

You'll complete the following to successfully use a custom video modality:

You may add the following to an existing in your workspace, if desired.


Pre-setup: Integrations

Est. time to complete: ~5 minutes

A one-time integration of a Natural Language Processing (NLP) engine must be completed in your workspace:


Step 1: Set up a modality

Est. time to complete: ~1 minute

  • Select Resources from workspace menu > Choose Modalities

  • Select New modality

  • Enter "Video" as the name name

  • Select Schema and change the Type to Object

  • Select Properties and then + Add property

  • Enter "src" as the Field name

  • Click Create modality


Step 2: Construct a flow

Est. time to complete: ~1 minute

You'll now enable the video modality you created on an end node and provide it with a video URL:

  • Select Flows from workspace menu > Create or choose an existing flow

  • Toggle Enable on the Video modality you created in the previous step

  • Enter dQw4w9WgXcQ as the value for "src"

  • Click Save

Repeat for any other flows where you would like to enable your video modality.


Step 3: Deploy an application

Est. time to complete: ~10 minutes

  • Select Applications from workspace menu > Choose New application

  • Enter a descriptive name > Click Save

  • Click Flows tab of application > Select Attach flows > Attach one or more flows you've created to make available to your application > Click Attach selected

  • Select Default behaviors tab of application > Assign any attached flows to the application's behaviors > Click Save

  • Select Channels tab of application > Expand API option > Click + Create channel

  • Click Create channel

  • Select Deployment tab of application > Click Create or Review & build

  • Wait for validation to complete > Select Create build*

  • When satisfied with a successful build, click Deploy


Step 4: Set up a webpage

Est. time to complete: ~10 minutes

  • Resuming from Step 3, click on the Details link next to the Deployed status of your application

  • Under Setup instructions, expand the API row

  • Copy the Application URL and API key

  • Copy the HTML file below and replace REPLACE_WITH_APPLICATION_URL and REPLACE_WITH_API_KEY with the values you just copied

<html lang="en">
  <head>
    <title>NLX Widget Sample HTML</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <script defer src="https://unpkg.com/@nlxai/chat-widget/lib/index.umd.js"></script>
    <script defer src="https://cdnjs.cloudflare.com/ajax/libs/htm/3.1.1/htm.js" integrity="sha512-RilD4H0wcNNxG2GvB+L1LRXCntT0zgRvRLnmGu+e9wWaLKGkPifz3Ozb6+WPsyEkTBLw6zWCwwEjs9JLL1KIHg==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <script>
      window.addEventListener("DOMContentLoaded", () => {
        // EMBEDDABLE COMPONENT SETUP
        // Destructure dependencies
        const React = nlxai.chatWidget.React;
        const useConversationHandler = nlxai.chatWidget.useConversationHandler;

        // Use the https://github.com/developit/htm package as a JSX alternative
        const html = htm.bind(React.createElement);
 
        Video = ({ data }) => {
          const videoSrc = `https://www.youtube.com/embed/${data.src}`;
          return html`
            <iframe
              width="264"
              height="160"
              src=${videoSrc}
              allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
              allowFullScreen
            ></iframe>`;
        };

        const widget = nlxai.chatWidget.create({
          config: {
            applicationUrl: "REPLACE_WITH_APPLICATION_URL",
            headers: {
              "nlx-api-key": "REPLACE_WITH_API_KEY"
            },
            languageCode: "en-US"
          },
          titleBar: {
            "title": "Modalities"
          },
          // CUSTOM BEHAVIOR SNIPPET
          onExpand: (conversationHandler) => {
            const checkMessages = (messages) => {
              if (messages.length === 0) {
                conversationHandler.sendWelcomeIntent();
              }
              conversationHandler.unsubscribe(checkMessages);
            };
            conversationHandler.subscribe(checkMessages);
          },
          customModalities: { Video },
          // CUSTOM BEHAVIOR SNIPPET END
          theme: {
            "primaryColor": "#2663da",
            "darkMessageColor": "#2663da",
            "lightMessageColor": "#EFEFEF",
            "white": "#FFFFFF",
            "fontFamily": "-apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, Oxygen-Sans, Ubuntu, Cantarell, \"Helvetica Neue\", sans-serif",
            "spacing": 12,
            "borderRadius": 8,
            "chatWindowMaxHeight": 640
          }
        });
      });
    </script>
  </body>
</html>

Step 5: Test the modality

Once you can access the page, click on the NLX widget at the bottom right of the page and converse with your application until you reach the node in the conversation where the video modality is set.

Click on any end node (a or node that is unconnected thereafter) > Select Add functionality > Choose Modalities

🧠 Looking for more? See

Click + Add domain under Whitelisted domains > Add your webpage's URL and

Using the , you can quickly build an HTML page to test out your video modality. You'll need to copy the application URL and API key from the application's Deployment tab.

Save the HTML file you just edited and serve the HTML file with your preferred command line tool such as

To test the modality, either host your HTML file or serve it locally using a command line tool like .

chat application
Steps for NLP integration
Construct flows
https://developers.nlx.ai
NLX Touchpoint SDK
serve
serve
user-type node