I built An Offline ChatBot Companion Raspberry Pi Robot

 

This post also has a YouTube version.

Meet Katherine my personalized offline conversational companion chatbot, which can answer questions, give facts on anything, and can remember and comprehend. 

I made this chatbot without any large language models. So no-GPT essentially.

After years of working on my own conversational software as a hobby, I finally have a prototype I can use to demonstrate.

This is Katherine my offline chatbot. I named it after Katherine Johnson - A human computer that worked for NASA. Katherine is powered by my own python code , some public libraries, and local files. I’ve spent a few years learning different strategies for closed looped conversational system. Thanks to Katherine here, I think I’m on the right track.

Why did I make Katherine?

 Well aside from having my own personalized chatbot, l wanted to be able to have a companion robot that I could talk to no matter the situation. Whether it be a Power outage, or a camping trip, my companion should be able to be useful anywhere. Privacy also became important to me as I began noticing all of our cell phone carriers starting to use our voice creepily to deliver ads.

How does Katherine Work?

Katherine works by taking in commands by listening to my speech. This is the speech to text component. Then when it wants to talk, you’ll see the animated mouth start to move, signaling a response, which it does vocally thanks to a text to speech component. I also added blinking for more character. 

Great now our robot can talk and listen what’s next? Next is the fun part. Commands and a knowledge base . Katherine needs information to talk about. You can’t have a great conversation without some kind of subject at hand.

For my project I searched the net and found the perfect knowledge base solution.  Wikipedia Enters the chat….

I know you all may be thinking, “oh no Wikipedia? That’s not accurate” and you may be right, however, Wikipedia has aggregated a massive amount of unrelated information. Where else can I get facts on Komodo dragons and all the satellites NASA has sent into space . It was the best solution for my problem.  

So cool I want to use Wikipedia but won’t that need the internet? And the answer is no. Thanks to libraries such as Kiwix I can save the entirety of a website as a file to be viewed offline. I use this component to get information about anything, with great speed.

Katherine is also equipped with the ability to answer questions thanks to the Bert model. The algorithm and acronym Bert stands for (Bidirectional Encoders Representations from Transformers) and is also the name of the model. Bert works by pointing it to a snippet of text and then you can ask questions about that text. 

My goal has been to try to couple the BERT and Wikipedia, for quick question answering, once I can figure out my hurdles I’ll come back to this.

The last major feature that makes Katherine so unique, is that Katherine can remember and even comprehend the things I tell it past our daily session. For example, I can Tell Katherine my brother likes firetrucks and not have to tell her again tomorrow.

 But this feature is a little deeper than just saving a record. My code comprehends allowing me to ask more complicated questions like. Which little boy likes firetrucks and ice cream. A query of this nature requires a little more than a simple database in order to respond accurately. This Concept is called Natural Language Understanding or NLU. 

So yeah Katherine has some pretty heavy features under the hood but that’s not all it can do. Katherine can tell time. Update the calendar, Get weather forecasts and the latest news updates. Katherine can give me definitions and it also has an extensive set of dad jokes.

So as you can see Katherine is a full feature chatbot companion, but I’ve missed another large part of being a companion. Actually accompanying, the user. So essentially mobility. 

For Katherine’s mobility, I chose a simple self balancing strategy. A two wheeled self balancing robot gives the impression of being up right like a human as well as gives me room to stack components. It’s been one of my favorites in robotic mobility design. The circuitry of the robot allows me to control it using my phone.

So that’s it everyone. I’ve introduced you to my state of the art Offline Chatbot system that can talk ,remember and even comprehend. I hope you’ve enjoyed this post. Happy Chatting!