Video modality

From start to finish, set up a chat-based application ft. a custom video modality with NLX

What's a Modality?

A modality allows for information to be transmitted to a user that goes beyond a standard message. Images, audio, structured text, haptic signals, videos, and more can be modalities you may want to provide users during conversation.

In chat applications, videos are useful for providing users with rich visual information that can explain, demonstrate, or enhance a user's search.

Checklist

You'll complete the following to successfully use a custom video modality:

Create a Modality
Construct a Flow
Deploy an Application
Set up a webpage

Pre-setup: Integrations

Est. time to complete: ~5 minutes

A one-time integration of a Natural Language Processing (NLP) engine must be completed in your workspace:

Steps for NLP integration

Step 1: Set up a modality

Est. time to complete: ~1 minute

Select Resources from workspace menu > Choose Modalities
Select New modality
Enter "Video" as the name name
Select Schema and change the Type to Object
Select Properties and then + Add property
Enter "src" as the Field name
Click Create modality

Step 2: Construct a flow

Est. time to complete: ~1 minute

You'll now enable the video modality you created on an end node and provide it with a video URL:

Select Flows from workspace menu > Create or choose an existing flow
Click on any end node (a user-type node or node that is unconnected thereafter) > Select Add functionality > Choose Modalities
Toggle Enable on the Video modality you created in the previous step
Enter dQw4w9WgXcQ as the value for "src"
Click Save

Repeat for any other flows where you would like to enable your video modality.

🧠 Looking for more? See Construct flows

Step 3: Deploy an application

Est. time to complete: ~10 minutes

Select Applications from workspace menu > Choose New application
Enter a descriptive name > Click Save
Click Flows tab of application > Select Attach flows > Attach one or more flows you've created to make available to your application > Click Attach selected
Select Default behaviors tab of application > Assign any attached flows to the application's behaviors > Click Save
Select Channels tab of application > Expand API option > Click + Create channel
Click + Add domain under Whitelisted domains > Add your webpage's URL and https://developers.nlx.ai
Click Create channel
Select Deployment tab of application > Click Create or Review & build
Wait for validation to complete > Select Create build*
When satisfied with a successful build, click Deploy

Step 4: Set up a webpage

Est. time to complete: ~10 minutes

You can quickly build an HTML page to test out your video modality. You'll need to copy the application URL and API key from the application's Deployment tab.

Resuming from Step 3, click on the Details link next to the Deployed status of your application
Under Setup instructions, expand the API row
Copy the Application URL and API key
Copy the HTML file below and replace REPLACE_WITH_APPLICATION_URL and REPLACE_WITH_API_KEY with the values you just copied
Save the HTML file you just edited and serve the HTML file with your preferred command line tool such as serve

<html lang="en">
  <head>
    <title>NLX Widget Sample HTML</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <script defer src="https://unpkg.com/@nlxai/chat-widget/lib/index.umd.js"></script>
    <script defer src="https://cdnjs.cloudflare.com/ajax/libs/htm/3.1.1/htm.js" integrity="sha512-RilD4H0wcNNxG2GvB+L1LRXCntT0zgRvRLnmGu+e9wWaLKGkPifz3Ozb6+WPsyEkTBLw6zWCwwEjs9JLL1KIHg==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <script>
      window.addEventListener("DOMContentLoaded", () => {
        // EMBEDDABLE COMPONENT SETUP
        // Destructure dependencies
        const React = nlxai.chatWidget.React;
        const useConversationHandler = nlxai.chatWidget.useConversationHandler;

        // Use the https://github.com/developit/htm package as a JSX alternative
        const html = htm.bind(React.createElement);
 
        Video = ({ data }) => {
          const videoSrc = `https://www.youtube.com/embed/${data.src}`;
          return html`
            <iframe
              width="264"
              height="160"
              src=${videoSrc}
              allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
              allowFullScreen
            ></iframe>`;
        };

        const widget = nlxai.chatWidget.create({
          config: {
            applicationUrl: "REPLACE_WITH_APPLICATION_URL",
            headers: {
              "nlx-api-key": "REPLACE_WITH_API_KEY"
            },
            languageCode: "en-US"
          },
          titleBar: {
            "title": "Modalities"
          },
          // CUSTOM BEHAVIOR SNIPPET
          onExpand: (conversationHandler) => {
            const checkMessages = (messages) => {
              if (messages.length === 0) {
                conversationHandler.sendWelcomeIntent();
              }
              conversationHandler.unsubscribe(checkMessages);
            };
            conversationHandler.subscribe(checkMessages);
          },
          customModalities: { Video },
          // CUSTOM BEHAVIOR SNIPPET END
          theme: {
            "primaryColor": "#2663da",
            "darkMessageColor": "#2663da",
            "lightMessageColor": "#EFEFEF",
            "white": "#FFFFFF",
            "fontFamily": "-apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, Oxygen-Sans, Ubuntu, Cantarell, \"Helvetica Neue\", sans-serif",
            "spacing": 12,
            "borderRadius": 8,
            "chatWindowMaxHeight": 640
          }
        });
      });
    </script>
  </body>
</html>

Step 5: Test the modality

To test the modality, either host your HTML file or serve it locally using a command line tool like serve.

Once you can access the page, click on the NLX widget at the bottom right of the page and converse with your application until you reach the node in the conversation where the video modality is set.

Last updated 16 days ago