Overview
Goal: To onboard developers onto chatbot the XState-Chatbot code base so that they can modify existing flows or create new ones
Prerequisites
...
.
This document sticks to explaining the chatbot's core features and does not dive into the use cases implemented by the chatbot. There is another document dedicated to it.
Prerequisites
...
NodeJS
...
Github
...
SOA
...
Checkout code
...
Install …
Recommended IDE: VSCode
Postman
...
Run code
using react-app
using node.js
When to use one vs other
...
Overall philosophy
No NL
Simple form fill
State machine approach
...
Introduction to x-state
Required reading ….
...
Developing
Introduction to project structure (quick)
Channels
Session manager
Machines
...
Understand current dialog flow -
Have the new developer interact with existing modules using react-app
How to use visual representation and IDE to understand dialog state including hierarchy / Understand how that flow is provided in the code
General patterns
Hierarchical structure
Typical structure 1 - preprocess -> question -> process -> error or transition
onEntry, onEvent, invoke-onDone, always ...
When to queue messages vs when to call toUser()
Scaffolding
Understand how session manager works
Channels
envconfig
Telemetry
Localization
Handling location - Google APIs
What is Postgres used for
What is ElasticSearch used for. Sanctifying user info
Best practices
Naming convention
Testing
How to handle large lists
Escape to the web, e.g., city & locality
Two-level structure, e.g., complaint type
Always monitor DROP OFF / COMPLETION for three months when introducing new dialog in the field
Future thoughts
...
Developer - Test
Changing prompt and response
Modifying dialog flow
Rearranging questions
Adding a new question
Adding a conditional question
Developing new dialog
Build ELK dashboard to understand drop off funnel
...
Operations
What do we have deployed today. Systems diagram ….
ELK
Initial Setup
Git - In macOS, most probably, it comes preinstalled. Git Installation Guide
Visual Studio Code (VSCode) - Great for Node JS Project Management
Install NodeJS
If you have HomeBrew, an easier way to install would be “brew install node”
After installing you should be able to run
node -v
npm -v
GitHub Repository - XState-Chatbot
Open this folder in the VSCode
Terminal could be opened within the VSCode (Ctrl + `)
For the first time installing dependencies: “npm install”
To run the app: “npm start”
To run the nodejs-server version of the code:
brew install cask postgresql
Install PostgreSQL (username - postgres, password - postgres, host - localhost, port - 5432). If the values differ, please modify those in the app.
Create a database named “chat”
Run the .sql files present in /db/migration/main folder
NodeJS Project Convention
All file and folder names in kebab-case
Class names in PascalCase. Variable names in camelCase
Development Path
...
Dummy service
...
eGov service
...
In-Memory database
...
react (1) - initial dialog dev, external developers
nodejs
...
react (2) - dialog hardening and testing
nodejs
...
Postgres database
(NA for React)
...
nodejs
...
nodejs (3) - staging and production
Message Format
...
Text
User input
Location
Geocode (lat, long) pair
Street address
Binary
...
Image
...
Overall Philosophy of Chatbot
This chatbot solves the basic form filling aspect of a chat flow. By collecting the information from the user, an API call can be made to the rainmaker backend services to fulfill what the user wants to do. It uses the concept of StateCharts (similar to State Machines) to maintain the state of the user in a chat flow and store the information provided by the user. XState is a JavaScript implementation of StateCharts. All chat flows are coded inside the XState framework.
This chatbot does not have any Natural Language Processing component. In the future, we can extend the chatbot to add such features.
Basic Introduction to XState
XState is a JavaScript implementation of StateCharts. There is detailed documentation available to study XState. Few of the concepts of XState that are used in Chatbot are listed below. Basic knowledge of these concepts is necessary. It can also be learned while going through the chat flow implementation of pilot use cases of PGR and Bills.
Few tips about using XState. These have been followed throughout the pilot chat flows.
If we want to move to any state which is not at the same hierarchical level, then we should assign it a unique id value. If it has an id value, we can address it using the # qualifier in the target attribute.
As id should be unique, please make sure there aren’t multiple states with the same id value. If there is a duplicate, the machine won’t function as expected.
Any actions(like onEntry) should be surrounded by assign.
Code Block assign( (context, event) => { ... } )
This would include almost all functions except the guard condition code snippets.
Understanding the Chat Flow
All the interactions with the user - sending a message to the user and processing an incoming message from the user is coded as a state in the State Machine. It would be a nice start to test any chat flow with the supplementary react-app provided for the developers to execute the state machine locally. (Please follow the guidelines in the README of the react-app.)
We have followed few standard patterns to code any chat interaction. Please try to follow these patterns to code any new chat flow. These patterns are explained below. You can also study those by browsing through the code of the pilot use cases of PGR and Bills.
The chat states would only include dialog specific code. Any code related to backend service should be written as a part of a separate …-service.js file.
Any code that doesn’t include any asynchronous API call can be written as a part of the onEntry function or an action.
If the function needs to make an API call, that would have to be written with the invoke-onDone pattern. The asynchronous function should be written as a part of the service file. The consolidated data returned by it can be processed in the state of the dialog file.
Helper functions are written in
dialog.js
file. It is advised to use those functions as much as possible rather than writing any custom logic in dialog files.
Scaffolding
Apart from the chat flow and its backend service API calls, few other components are present in the project. These components do NOT need to be modified to code any new chat flow or changing an existing chat flow. These components with a short description for each are listed below:
Session Manager: It manages sessions of all the users on a server. It will store the user’s state in a datastore, update it, and read it when any new message is received on the server. Based on the state of the user, it will create a state machine and send the incoming message event to the state machine. It sanctifies the state (any sensitive data like the name and mobile number of a user are removed) before storing the state to the datastore.
Repository: It is the datastore where the states of the users get stored. To reduce dependency, an in-memory repository is also provided, which can be used by configuring an environment variable. So to run the chatbot service, PostgreSQL isn’t a hard dependency, but it is advisable to use the PostgreSQL repo provider.
Channel Provider: There can be many different WhatsApp Providers. Any one of the providers will be configured to be used. A separate
console
WhatsApp Provider is present for the developer to test the chatbot server locally. Postman collection to mimic receiving messages from a user to the server is present in the project directory.Localization: Every message to be sent to the user is stored within the chatbot. Localization service is not being used. These messages are present near the bottom of the dialog files. A separate localization-service.js is provided to get the messages for the localization codes for the messages that are not owned by the chatbot. For example, the PGR complaint types data is under the ownership of the PGR module, and the messages for such can be fetched from the egov-localization-service using the functions provided in the localization-service.js.
Service Provider: To ease the initial dialog development, instead of the coding API calls to the backend services, we can configure the chat flow to use a dummy service. This can be configured using an environment variable and modifying the
service-loader.js
file.Telemetry: Chatbot logs telemetry events to a kafka topic. (Any sensitive data will get masked before indexing the events onto ElasticSearch by egov-indexer.) Following events get logged:
Incoming message
Outgoing message
Transition of state