News

Open Source Chat Bot

An open-source chat bot for rapid implementation of domain specific knowledge. Use your own docs to provide the data.

Jeremy Branham

Jul 3, 2023 • 3 min read

In the world of conversational AI, chatbots have become an essential tool for businesses to engage with their customers. Savantly's BASE-BOT is software project that takes advantage of the LangChain chatbot framework, offering easy reusability and customization options. In this blog post, we will delve into the features, technical aspects, and future roadmap.

Overview

Savantly BASE-BOT is a chatbot, forked from the LangChain chatbot platform.
Its primary purpose is to provide a streamlined and parameterized solution for developers looking to quickly incorporate chatbot functionality into their applications. By leveraging LangChain's streaming support and async API, it can update pages in real-time for multiple users, enhancing the interactive user experience.

Getting Started with BASE-BOT

Running Savantly's BASE-BOT is easy thanks to its Docker containerization. To begin, make sure you have your OPENAI_API_KEY as an environment variable. Then, execute the following command to run the Docker container:

docker run -p 9000:9000 -e OPENAI_API_KEY=${OPENAI_API_KEY} savantly/base-bot

Customization Made Easy

BASE-BOT focuses on customization, allowing developers to tailor the chatbot according to their specific requirements.
To get started, you can copy the provided docker-compose.yml file to your project folder and create a data directory for your own domain.
You'll find starter data in the ./data folder. After setting up the necessary directories, run docker-compose up to launch the chatbot with your custom settings. Feel free to override other settings as needed to fine-tune the behavior.

Developing with BASE-BOT

If you wish to extend the functionality of Savantly's BASE-BOT or contribute to its development, the following steps will guide you:

Install the project dependencies using pip install -r requirements.txt.
Run ingest.sh to ingest LangChain docs data into the vector store. This step is only required once.
Utilize different Document Loaders to load your own data into the vector store, allowing the chatbot to access the relevant information.
Start the application by running make start.
To enable tracing, ensure that langchain-server is running locally, and pass tracing=True to the get_chain function in main.py.
Access the chatbot by opening localhost:9000 in your web browser.

Configuring BASE-BOT

To customize the chatbot parameters, modify the settings.py file according to your preferences. Additionally, you can create a .secrets.py file to store any sensitive data. Both of these files can be mounted in place if you are using the pre-built Docker image, simplifying the deployment process.

Technical bits

BASE-BOT consists of two main components: ingestion and question-answering.

Ingestion:

Scan the ./data/docs folder for content to load, excluding files starting with a dot.
Load the documents using LangChain's DirectoryLoader.
Split the documents into segments using LangChain's TextSplitter.
Create a vectorstore of embeddings using LangChain's vectorstore wrapper, utilizing OpenAI's embeddings and FAISS vectorstore.

Question-Answering:

Given the chat history and user input, determine a standalone question using GPT-3.
Given that standalone question, look up relevant documents from the vectorstore.
Pass the standalone question and relevant documents to GPT-3 to generate a final answer.

Roadmap

DONE - Parameterize prompts and ingestion settings
DONE - Enable loading docs from a folder
DONE - Create Docker image
DONE - Create docker-compose for rapid testing
Create parameters and code to substitute the pickled vector store for Pinecone or another implementation
Create Helm chart
Create web-component for dropping the chat on any web page
Parameterize additional chain settings??

Check the project page or subscribe to the Savantly blog to stay up to date.

GitHub

The project source code can be located here -