100 Days to RASA Chatbot — Tech
I think I can safely assume that you have come here to know what lies ahead in the remaining 90 days of our RASA based Chatbot development.
And in case if you have not, then I’d request you to have a look at the Product & Business side of this RASA based Chatbot Development. The first 10 days of our Chatbot development were really important as we were focusing on understanding the problem statement right and drawing out the expected flow basis the discussions with our Business Unit.
Click here to know what went around in the first 10 days.
Day 10 to Day 15
Designs | New API Development | Events, Analytics & UTM Tracking Setup
I will refrain from sharing the designs here. How about check the new bot yourself? Just go to https://cashify.in and scroll to the bottom of the webpage and click on “Chat with Us”. Voila!
Talking of New API Development, there were a couple of new APIs that we got created to serve some of our use-cases.
(i) Field Agent Logistics API: This API basically provided us the Field Agent’s customer leads data for a day, which was basically also required to calculate the Agent ETA for showing it to the customer who enquired for it. There are a couple of important data attributes that we get from this API. Agent’s shift start time, his current lat-long coordinates, last lead served time etc.
(ii) Generic API for is_Logistics information: This was a generic API that was created to basically provide us a True/False value against the “is_Logistics” key through this API. This API basically tells us whether a certain Buyback order will be served by Logistics or from Partner?
“Events” was another important thing to take care of. As the name suggests, “Events” are basically the instances of user engagement, mostly in the form of clicks, messages typed, user’s selection and various other types of interactions.
What do we do with these events? — These events importantly need to get saved in our databases so as to draw insights from them.
What kind of insights do these events majorly give? :-
- What are the dialogues that the customer is majorly executing or interacting with?
- At what all instances are fallbacks occurring?
- Which stories (use-cases) are mostly executed?
- Let’s say if we have buttons in our dialogue flow, then which ones are those that were majorly clicked by the customer?
- Similarly, we also get to look at all those use-cases/stories that were relatively less performed/executed by the customers.
Why do we exactly need these events?
The insights that these events provide, essentially create the scope of improvement for us that we can later go and include in our Product Roadmap.
How did we plan to track/save the events?
We are simply pushing our event logs into MongoDB. We are maintaining a dedicated collection for all the conversations that are now taking place on our Chatbot. Every single conversation (chat session) has been given a unique object ID.
Let’s show you how a sample event looks like :-
{
“_id” : ObjectId(“*********************”),
“sender_id” : “default”,
“active_form” : {},
“events” : [
{
“event” : “action”,
“timestamp” : *****************,
“name” : “action_session_start”,
“policy” : null,
“confidence” : null
},
{
“event” : “session_started”,
“timestamp” : *****************
}]
}
I’ve masked the critical information for some compliance reasons. However, you get a decent idea of how RASA provides event logs in general. You can explore more about RASA’s Event Broker.
We also did a bit of UTM tracking. In one of our flows of the Chatbot, when we convey it to the user whether we provide Sell/Repair service in the user’s location or not, we also provide a link, clicking which the user can land on the Cashify’s product listing page, and search for their particular device variant / model that they want to sell or get repaired.
Day 15 to Day 75
NLP Model Creation : Stories | Intents | Actions | API Endpoint Integrations | Front End (ReactJS) Component Creation
Now comes the part, where I feel a lot of RASA enthusiasts would have quickly fast forwarded to. This anyway basically defines the most integral part of any RASA based chatbot development.
Here, we (primarily, the Developer) define Intents, Stories, Domain, Config, Credentials files, etc.
Let’s take a quick tour through each of these files and their significance:-
Intents file: This is where the developer defines all the possible intents of the user that the bot can be subjected to. For instance, in the case of our Chatbot, we can majorly identify two broad level intents. One is “Awaiting Pickup Specialist” and the other is “Service Availability”. What is done in an Intents file is that the user provides some sample messages/utterances that the bot can receive from the customer, for each of the defined intents.
A sample intents file would look like:-
## intent:awaiting_pickup_specialist
- When will the Pickup Specialist come for pickup?
- I have other concerns.## intent:service_availability
- I want to sell my mobile phone
- I want to get my phone repaired
What’s shown above is obviously a sample intents file. Our actual intents file looks more verbose for obvious reasons. Can’t share the details here for compliance reasons.
Stories file: This is the file where the developer defines all the stories (flows) anticipated in the chatbot, to happen. A story is basically a representation of a conversation between a user and an AI assistant, converted into a specific format where the user inputs are defined as intents (and entities where necessary) and the assistant’s responses and actions are expressed as action names.
*Please assume the indents at your end.
## fallback
- utter_default## awaiting_pickup_specialist_story
* awaiting_pickup_specialist
- utter_awaiting_pickup_specialist
- return## service_availability_story
* service_availability
- utter_service_availability
Domain file: This is the file where the developer defines the 5 key parts for RASA NLU and RASA Core to function, namely, slots, entity, intent, actions & templates. You can read more about them here. A sample domain file looks like:-
*Please assume the indents at your end.
slots:
PERSON:
type: text
entities:
- sell
- repairintents:
- awaiting_pickup_specialist
- service_availabilityactions:
- action_restart
- utter_service_availability
- utter_awaiting_pickup_specialist
- utter_bye
- utter_defaulttemplates:
utter_service_availability:
- text: Hey {PERSON}! Do you want a Sell or Repair service?
- text: Hello {PERSON}! May I know what kind of a service are you looking / for?utter_awaiting_pickup_specialist:
- text: Hey {PERSON}! Please let me know what’s your pickup specialist / related query?
Similarly, there are config and credentials files too.
Config file is a yaml-formatted file that contains environment variables and command line arguments.
Credentials file contains the API endpoints, auth/login credentials of most of the voice/text chatbots, or other third party platforms or message streaming services that we want to integrate with our Chatbot.
You may visit RASA’s official website and read their documentation to know more.
Day 75 to Day 80
Bot Simulation on Local | A/B [Audience Split] Test Setup
This was the time when we were regularly testing the bot on our developer’s local machine and suggesting any flow improvements, right there. A lot of google meet/phone calls happened, emails were exchanged, just to ensure that we move forward with a stable chatbot build, when we proceed to the staging environment.
A/B (Audience Split) Testing was another important piece that was done during this time. We used GTM to control the audience split as the plan was to expose this new RASA Chatbot to a limited set of users and gauge the performance on this limited audience.
As we progressed, we ended up using GTM to also control the size/dimensions of the iframe that used to load on the website. GTM is a swiss-army-knife, I tell you 😎
Just to give you a quick idea of how a normal GTM script looks like (assume the indents at your end):-
<script>
function getRandomInt(max) {
return ######;
}
function testVariant() {
var ##### = document.getElementById(“########”);
var ##### = document.getElementById(“########”);
variantElement.children[0].innerText = “#######”
var styling = document.createElement(“style”);
styling.innerHTML = “div######## > div{width:##### !important;height:##### !important;”;
document.body.appendChild(styling);
########.style.display = “none”;
########.style.display = “block”;
window.localStorage.setItem(‘######’, ‘true’);
}
</script>
Day 80 to Day 90
Bot Testing on Staging | BugFixes | Pre-production API & Flow Tuning
We didn’t do anything departing from the normal testing theories of various companies. Just like other companies, it was important for us to deploy our Chatbot on our staging servers and test the end-to-end flows.
It is generally this stage where the Product Manager conducts thorough testing of his product feature, with the help of the QA. Smoke Testing, Unit Testing, Integration Testing, Automation Testings, etc. are some of the popular QA techniques that you normally would be accustomed to.
As expected, there were a couple of bugs that were discovered and fixed on time.
There were also some API level changes that were done, for a larger fine-tuning objective.
Day 90 to Day 95
Bot Testing on Beta | Load / Stress Testing | Server Consumption Analysis
Once, the stable stage Chatbot build was confirmed, we pushed it to our beta environment. The idea was to publish our chatbot on the beta website of Cashify which essentially used production DB and APIs to serve the front end. We placed real orders and performed chatbot conversations with a real production-like sentiment.
Tech Lead also conducted load & stress testing via Jmeter & Postman.
Day 95 to Day 100
Crowd Testing on Beta | Pre-prod final cosmetic & logic fixes
The chatbot was now offered to the internal audience for crowd testing.
As already anticipated, we had a lot of entries from the co-PMs and other internal staff for cosmetic improvements and logic enhancements.
We tried to cater the most critical ones and parked the remaining for future releases.
Day 100 :
And, we go LIVE! — NOV 20, 2020
Let me not share any images of how our RASA bot looks like. Rather, I shall ask you to visit https://www.cashify.in and quickly scroll down to the bottom of the page. Clicking “Chat with Us” will take you to our new RASA Chatbot.
And yes, don’t forget to sell your old phone!
Now that you will be visiting India’s leading consumer <> seller second hand gadgets marketplace, check out some really cracking price quotes for your old phones.
And our repair services are niche in the domain too, FYI :)
Thanks for reading!
