Building a Complete Chatbot with LangChain
This topic brings together everything from the course — models, prompts, memory, document loading, embeddings, vector stores, RAG, and output parsers — into a single working chatbot application. By the end you will have a fully functional AI assistant that answers questions from your own documents, remembers the conversation, handles errors gracefully, and runs as a command-line application ready to be extended into a web service.
The Application We Are Building
The chatbot is a knowledge base assistant for a fictional company called NovaTech. It reads from company documents (FAQ, policy guide, product catalog), answers employee and customer questions using those documents, and remembers the conversation context across multiple turns. When the answer is not in the documents, it says so clearly instead of making something up.
NovaTech Assistant Architecture:
User Question
│
▼
┌─────────────────┐
│ Input Handler │ (validates, classifies question)
└────────┬────────┘
│
┌────────────┴────────────┐
│ │
▼ ▼
┌──────────────┐ ┌──────────────────┐
│ Rephrase │ │ Direct Answer │
│ with │ │ (greetings, │
│ history │ │ commands) │
└──────┬───────┘ └──────────────────┘
│
▼
┌──────────────┐
│ Retriever │ (searches company knowledge base)
└──────┬───────┘
│ relevant chunks
▼
┌──────────────┐
│ RAG Chain │ (answers using retrieved context)
└──────┬───────┘
│
▼
┌──────────────┐
│ Memory │ (saves turn to history)
└──────┬───────┘
│
▼
Answer to User
Project Structure
novatech_bot/ ├── .env ← API keys ├── .gitignore ├── requirements.txt ├── documents/ │ ├── faq.txt │ ├── policies.txt │ └── products.txt ├── knowledge_base/ ← FAISS vector store (generated) ├── config.py ← Settings and constants ├── knowledge_base_builder.py ← Index documents ├── chains.py ← All LangChain chains └── chatbot.py ← Main application entry point
Step 1: Configuration (config.py)
# config.py # Model settings CHAT_MODEL = "gpt-3.5-turbo" EMBEDDING_MODEL = "text-embedding-ada-002" TEMPERATURE = 0.2 MAX_TOKENS = 800 # Retrieval settings CHUNK_SIZE = 800 CHUNK_OVERLAP = 150 TOP_K_RESULTS = 4 # Memory settings MAX_HISTORY_MESSAGES = 20 # Keep last 20 messages (10 turns) # Paths DOCUMENTS_DIR = "./documents" VECTOR_STORE_PATH = "./knowledge_base" # Bot identity BOT_NAME = "Nova" SYSTEM_PROMPT = """You are Nova, the NovaTech AI assistant. Your job is to help employees and customers by answering questions based on NovaTech's official documents. Rules: - Answer ONLY using the provided context documents. - If the answer is not in the context, say: "I don't have that information in my documents. Please contact support@novatech.com." - Be friendly, professional, and concise. - Always cite which document your answer comes from when possible. - Do not make up information, prices, policies, or features."""
Step 2: Create Sample Documents (documents/)
# documents/faq.txt Frequently Asked Questions - NovaTech Q: What are your business hours? A: NovaTech support is available Monday to Friday, 9am to 6pm IST. Emergency support is available 24/7 for Enterprise customers. Q: How do I reset my password? A: Visit account.novatech.com and click "Forgot Password". Enter your registered email address. You will receive a reset link within 5 minutes. Q: What payment methods do you accept? A: We accept Visa, Mastercard, UPI, and bank transfers. All payments are processed securely through Razorpay.
# documents/policies.txt NovaTech Company Policies REFUND POLICY: Customers may request a full refund within 14 days of purchase for any reason. After 14 days, refunds are considered on a case-by-case basis. To initiate a refund, email billing@novatech.com with your order number. REMOTE WORK POLICY (Employee): Full-time employees may work remotely up to 3 days per week with manager approval. New employees must work from office for their first 60 days. Remote work equipment allowance: Rs 15,000 per year. LEAVE POLICY (Employee): Annual leave: 18 days per year. Sick leave: 12 days per year, no carry-forward. Maternity leave: 26 weeks fully paid. Paternity leave: 15 days fully paid.
# documents/products.txt NovaTech Product Catalog NOVA STARTER PLAN: Price: Rs 999/month Users: Up to 5 Storage: 20 GB Features: Basic analytics, email support, API access (100k calls/month) NOVA BUSINESS PLAN: Price: Rs 3,999/month Users: Up to 25 Storage: 200 GB Features: Advanced analytics, priority support, API access (1M calls/month), custom integrations NOVA ENTERPRISE PLAN: Price: Custom pricing Users: Unlimited Storage: Unlimited Features: All Business features, dedicated account manager, SLA guarantee, on-premise option Contact: sales@novatech.com
Step 3: Build the Knowledge Base (knowledge_base_builder.py)
# knowledge_base_builder.py
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from config import (
DOCUMENTS_DIR, VECTOR_STORE_PATH,
CHUNK_SIZE, CHUNK_OVERLAP, EMBEDDING_MODEL
)
load_dotenv()
def build_knowledge_base():
print("Building NovaTech knowledge base...")
# Load all text files from documents folder
loader = DirectoryLoader(
DOCUMENTS_DIR,
glob="**/*.txt",
loader_cls=TextLoader,
show_progress=True
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")
# Split into chunks
splitter = RecursiveCharacterTextSplitter(
chunk_size=CHUNK_SIZE,
chunk_overlap=CHUNK_OVERLAP
)
chunks = splitter.split_documents(documents)
print(f"Created {len(chunks)} chunks")
# Add source category to metadata
for chunk in chunks:
filename = os.path.basename(chunk.metadata.get("source", ""))
if "faq" in filename:
chunk.metadata["category"] = "FAQ"
elif "policies" in filename:
chunk.metadata["category"] = "Policy"
elif "products" in filename:
chunk.metadata["category"] = "Product"
# Build and save vector store
embeddings = OpenAIEmbeddings(model=EMBEDDING_MODEL)
vector_store = FAISS.from_documents(chunks, embeddings)
vector_store.save_local(VECTOR_STORE_PATH)
print(f"Knowledge base saved to {VECTOR_STORE_PATH}")
print("Done! Run chatbot.py to start the assistant.")
if __name__ == "__main__":
build_knowledge_base()
Step 4: Build the Chains (chains.py)
# chains.py
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough, RunnableParallel, RunnableLambda
from config import (
CHAT_MODEL, EMBEDDING_MODEL, TEMPERATURE, MAX_TOKENS,
TOP_K_RESULTS, VECTOR_STORE_PATH, SYSTEM_PROMPT
)
load_dotenv()
# Initialize components
model = ChatOpenAI(model=CHAT_MODEL, temperature=TEMPERATURE, max_tokens=MAX_TOKENS)
embeddings = OpenAIEmbeddings(model=EMBEDDING_MODEL)
parser = StrOutputParser()
def load_retriever():
store = FAISS.load_local(
VECTOR_STORE_PATH, embeddings,
allow_dangerous_deserialization=True
)
return store.as_retriever(
search_type="mmr",
search_kwargs={"k": TOP_K_RESULTS, "fetch_k": 15}
)
def format_docs(docs) -> str:
"""Format retrieved chunks into a readable context block with source labels."""
sections = []
for doc in docs:
category = doc.metadata.get("category", "Document")
source = doc.metadata.get("source", "unknown")
sections.append(f"[{category} — {source}]\n{doc.page_content}")
return "\n\n---\n\n".join(sections)
# Chain 1: Rephrase follow-up questions using conversation history
rephrase_prompt = ChatPromptTemplate.from_messages([
("system",
"Rephrase the user's question to be fully standalone using the conversation history. "
"Return only the rephrased question. Preserve the original intent exactly."),
MessagesPlaceholder(variable_name="history"),
("human", "{question}")
])
rephrase_chain = rephrase_prompt | model | parser
# Chain 2: Answer using retrieved context
answer_prompt = ChatPromptTemplate.from_messages([
("system", SYSTEM_PROMPT + "\n\nContext from NovaTech documents:\n{context}"),
MessagesPlaceholder(variable_name="history"),
("human", "{question}")
])
answer_chain = answer_prompt | model | parser
retriever = None # Loaded lazily to avoid errors before KB is built
def get_retriever():
global retriever
if retriever is None:
retriever = load_retriever()
return retriever
def build_rag_response(question: str, history: list) -> dict:
"""Full RAG pipeline: retrieve context and generate answer."""
r = get_retriever()
docs = r.invoke(question)
context = format_docs(docs)
answer = answer_chain.invoke({
"context": context,
"history": history,
"question": question
})
sources = list(set(
doc.metadata.get("category", "Unknown")
for doc in docs
))
return {"answer": answer, "sources": sources, "docs": docs}
Step 5: Main Chatbot Application (chatbot.py)
# chatbot.py
import os
from pathlib import Path
from dotenv import load_dotenv
from langchain_core.messages import HumanMessage, AIMessage
from chains import rephrase_chain, build_rag_response
from config import BOT_NAME, VECTOR_STORE_PATH, MAX_HISTORY_MESSAGES
load_dotenv()
# Conversation state
history = []
COMMANDS = {
"/clear": "Conversation history cleared.",
"/help": "Commands: /clear, /help, /history",
"/history": None # Handled dynamically
}
def trim_history(history: list) -> list:
"""Keep only the last MAX_HISTORY_MESSAGES messages."""
if len(history) > MAX_HISTORY_MESSAGES:
return history[-MAX_HISTORY_MESSAGES:]
return history
def handle_command(cmd: str) -> str | None:
if cmd == "/clear":
history.clear()
return COMMANDS["/clear"]
if cmd == "/help":
return COMMANDS["/help"]
if cmd == "/history":
if not history:
return "No conversation history yet."
lines = []
for msg in history[-10:]: # Show last 10 messages
speaker = "You" if msg.type == "human" else BOT_NAME
lines.append(f"{speaker}: {msg.content[:80]}...")
return "\n".join(lines)
return None
def chat(user_input: str) -> str:
"""Main chat function: routes input through rephrase → retrieve → answer."""
# Rephrase if there is history
if history:
try:
standalone_q = rephrase_chain.invoke({
"history": history,
"question": user_input
})
except Exception:
standalone_q = user_input # Fall back to original if rephrase fails
else:
standalone_q = user_input
# Retrieve and answer
try:
result = build_rag_response(standalone_q, history)
answer = result["answer"]
sources = result["sources"]
# Append source footnote if sources were found
if sources:
answer += f"\n\n[Sources: {', '.join(sources)}]"
except FileNotFoundError:
return ("Knowledge base not found. Please run knowledge_base_builder.py first.")
except Exception as e:
return f"I encountered an error. Please try again. (Error: {type(e).__name__})"
# Save to history
history.append(HumanMessage(content=user_input))
history.append(AIMessage(content=answer))
# Trim to keep history manageable
trimmed = trim_history(history)
history.clear()
history.extend(trimmed)
return answer
def main():
"""Run the chatbot in the terminal."""
print(f"\n{'='*60}")
print(f" {BOT_NAME} — NovaTech AI Assistant")
print(f"{'='*60}")
print(f" Type your question and press Enter.")
print(f" Commands: /help | /clear | /history | quit")
print(f"{'='*60}\n")
# Check knowledge base exists
if not Path(VECTOR_STORE_PATH).exists():
print(f"WARNING: Knowledge base not found at {VECTOR_STORE_PATH}")
print("Run: python knowledge_base_builder.py\n")
return
while True:
try:
user_input = input("You: ").strip()
except (KeyboardInterrupt, EOFError):
print(f"\n{BOT_NAME}: Goodbye!")
break
if not user_input:
continue
if user_input.lower() in ["quit", "exit", "bye"]:
print(f"{BOT_NAME}: Goodbye! Have a great day.")
break
if user_input.startswith("/"):
result = handle_command(user_input.lower())
if result is not None:
print(f"{BOT_NAME}: {result}\n")
continue
else:
print(f"{BOT_NAME}: Unknown command. Type /help for available commands.\n")
continue
response = chat(user_input)
print(f"\n{BOT_NAME}: {response}\n")
if __name__ == "__main__":
main()
Running the Application
Follow these steps to run the complete chatbot:
# Step 1: Install dependencies pip install langchain langchain-openai langchain-community faiss-cpu python-dotenv # Step 2: Set up .env with your API key echo "OPENAI_API_KEY=sk-your-key-here" > .env # Step 3: Create the documents folder and add your text files mkdir documents # (copy the sample faq.txt, policies.txt, products.txt files shown above) # Step 4: Build the knowledge base (do this once, or whenever documents change) python knowledge_base_builder.py # Step 5: Start the chatbot python chatbot.py
Sample Conversation
You: Hi, what are your business hours?
Nova: NovaTech support is available Monday to Friday, 9am to 6pm IST.
Enterprise customers have access to 24/7 emergency support.
[Sources: FAQ]
You: What about refunds?
Nova: You can request a full refund within 14 days of purchase for any reason.
After 14 days, refunds are reviewed case by case. To start a refund,
email billing@novatech.com with your order number.
[Sources: Policy]
You: How much does the business plan cost?
Nova: The Nova Business Plan is priced at Rs 3,999 per month. It supports up to
25 users, 200 GB of storage, advanced analytics, priority support, and
1 million API calls per month.
[Sources: Product]
You: What about leave for new parents?
Nova: NovaTech offers 26 weeks of fully paid maternity leave and 15 days of
fully paid paternity leave.
[Sources: Policy]
You: Can I work from home?
Nova: Full-time employees can work remotely up to 3 days per week with manager
approval. New employees must work from the office for their first 60 days.
The annual remote work equipment allowance is Rs 15,000.
[Sources: Policy]
Extending the Chatbot
Add More Document Types
Add PDF loaders to knowledge_base_builder.py to include product manuals, presentations, or reports. The rest of the application works without changes.
Add a Web Interface
Replace the terminal loop in chatbot.py with a FastAPI or Flask endpoint. The chat() function becomes the API handler — it accepts a string and returns a string, making the transition trivial.
Add a WhatsApp or Telegram Bot
Use the Twilio WhatsApp API or python-telegram-bot library. Both call your chat() function with the incoming message and send back the response. The LangChain logic never changes.
Add Streaming for Web UIs
Replace answer_chain.invoke() with answer_chain.stream() and yield chunks to the frontend via server-sent events. Users see words appearing in real time.
Summary
A complete production chatbot combines document loading, text splitting, embedding, vector storage, retrieval-augmented generation, conversation memory, input validation, command handling, and error resilience. The modular project structure separates configuration, chain logic, and application flow into distinct files. The knowledge base is built once and reused across many sessions. The chat function handles rephrasing, retrieval, answering, and memory management in a clean sequence.