Video modality

From start to finish, set up a chat-based application ft. a custom video modality with NLX

What's a Modality?

A modality allows for information to be transmitted to a user that goes beyond a standard message. Images, audio, structured text, haptic signals, videos, and more can be modalities you may want to provide users during conversation.

In chat applications, videos are useful for providing users with rich visual information that can explain, demonstrate, or enhance a user's search.

Checklist

You'll complete the following to successfully use a custom video modality:

Create modality
Create flow
Deploy
Install

Step 1: Create modality

Est. time to complete: ~1 minute

Select Resources from workspace menu > Choose Modalities
Select New modality
Enter "Video" as the name name
Select Schema and change the Type to Object
Select Properties and then + Add property
Enter "src" as the Field name
Click Create modality

Step 2: Create flow

Est. time to complete: ~3 minutes

You'll now enable the video modality you created on an end node and provide it with a video URL:

Select Resources from workspace menu > Choose Flows > Click New flow or choose existing
Click on any end node (a user-type node or node that is unconnected thereafter) > Select Add functionality > Choose Modalities
Toggle Enable on the Video modality you created in the previous step
Enter dQw4w9WgXcQ as the value for "src"
Click Save

Repeat for any other flows where you would like to enable your video modality.

🧠 Looking for more? See Flows

Step 3: Deploy

Est. time to complete: ~2 minutes

Select Applications from workspace menu > Choose New application > Select Custom > Provide a name
Under the Functionality section of your app's Configuration tab, attach the ExampleCarouselFlow
- Select Default behavior > Assign any attached flows to the application's behaviors
- From the Delivery section > Select API channel > Whitelist your webpage's URL
- Click Update channel > Click Save
A build now constructs a package of your application with a snapshot of current state of flows, languages, and application setup. A deployment then pushes a successful build to the communication channels where your app is installed:
- Click deployment status in upper right > Select Build and deploy
  - Review Validation check for critical errors or detected UX issues in custom flows. Before each new build initiates, a validation check is run to provide a preview of potential errors that may cause failed builds. Detected issues are listed with descriptions and potential solutions
  - You may provide a Description of notable build edits as a changelog
  - Click Create build
- Click deployment status in upper right > Select All builds > Choose Deploy on successful build > Click Create deployment

Step 4: Install

Est. time to complete: ~5 minutes

You can quickly build an HTML page to test out your video modality.

From your application's Configuration tab in NLX, click the API channel
Select the Setup instructions tab in the modal > Save your Application URL and API Key
Copy the HTML file below and replace REPLACE_WITH_APPLICATION_URL and REPLACE_WITH_API_KEY with the values you just copied
Save the HTML file you just edited and serve the HTML file with your preferred command line tool such as serve

<html lang="en">
  <head>
    <title>NLX Widget Sample HTML</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <script defer src="https://unpkg.com/@nlxai/chat-widget/lib/index.umd.js"></script>
    <script defer src="https://cdnjs.cloudflare.com/ajax/libs/htm/3.1.1/htm.js" integrity="sha512-RilD4H0wcNNxG2GvB+L1LRXCntT0zgRvRLnmGu+e9wWaLKGkPifz3Ozb6+WPsyEkTBLw6zWCwwEjs9JLL1KIHg==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <script>
      window.addEventListener("DOMContentLoaded", () => {
        // EMBEDDABLE COMPONENT SETUP
        // Destructure dependencies
        const React = nlxai.chatWidget.React;
        const useConversationHandler = nlxai.chatWidget.useConversationHandler;

        // Use the https://github.com/developit/htm package as a JSX alternative
        const html = htm.bind(React.createElement);
 
        Video = ({ data }) => {
          const videoSrc = `https://www.youtube.com/embed/${data.src}`;
          return html`
            <iframe
              width="264"
              height="160"
              src=${videoSrc}
              allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
              allowFullScreen
            ></iframe>`;
        };

        const widget = nlxai.chatWidget.create({
          config: {
            applicationUrl: "REPLACE_WITH_APPLICATION_URL",
            headers: {
              "nlx-api-key": "REPLACE_WITH_API_KEY"
            },
            languageCode: "en-US"
          },
          titleBar: {
            "title": "Modalities"
          },
          // CUSTOM BEHAVIOR SNIPPET
          onExpand: (conversationHandler) => {
            const checkMessages = (messages) => {
              if (messages.length === 0) {
                conversationHandler.sendWelcomeIntent();
              }
              conversationHandler.unsubscribe(checkMessages);
            };
            conversationHandler.subscribe(checkMessages);
          },
          customModalities: { Video },
          // CUSTOM BEHAVIOR SNIPPET END
          theme: {
            "primaryColor": "#2663da",
            "darkMessageColor": "#2663da",
            "lightMessageColor": "#EFEFEF",
            "white": "#FFFFFF",
            "fontFamily": "-apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, Oxygen-Sans, Ubuntu, Cantarell, \"Helvetica Neue\", sans-serif",
            "spacing": 12,
            "borderRadius": 8,
            "chatWindowMaxHeight": 640
          }
        });
      });
    </script>
  </body>
</html>

Step 5: Test

To test the modality, either host your HTML file or serve it locally using a command line tool like serve.

Once you can access the page, click on the NLX widget at the bottom right of the page and converse with your application until you reach the node in the conversation where the video modality is set.

Last updated 2 days ago