LangChain Conversation History and Chat Models
You now understand how memory stores conversation history and how models process messages. This topic brings those ideas together and shows you how to build a complete, production-ready conversational loop. You will learn how Chat Models handle multi-turn dialogue, how to structure your conversation management code cleanly, and how to handle the common patterns that appear in real chatbot applications.
The Telephone Operator Analogy
Old telephone operators connected callers by physically plugging cables between switchboard ports. When you called someone, the operator noted who you were, who you wanted to reach, and connected the right cables. Chat Models work similarly — they accept a structured list of "cables" (messages) from different parties, process the full conversation context, and generate the next appropriate response.
Conversation as a Structured List: ┌────────────────────────────────────────────────────┐ │ [1] SystemMessage: "You are a cooking tutor." │ │ [2] HumanMessage: "How do I make pasta sauce?" │ │ [3] AIMessage: "Start with olive oil..." │ │ [4] HumanMessage: "What oil should I use?" │ │ ↑ │ │ Model receives this list and generates [5] │ └────────────────────────────────────────────────────┘
Building a Complete Chat Loop
A working chatbot needs four things: a prompt template with memory support, a model, a parser, and a loop that accepts user input and calls the chain. Here is a fully functional command-line chatbot:
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage, AIMessage
from langchain_core.output_parsers import StrOutputParser
load_dotenv()
# Build the chain
model = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.5)
parser = StrOutputParser()
prompt = ChatPromptTemplate.from_messages([
("system", "You are a knowledgeable and friendly assistant called Nova. "
"Answer concisely and accurately. Ask follow-up questions when helpful."),
MessagesPlaceholder(variable_name="history"),
("human", "{input}")
])
chain = prompt | model | parser
# Conversation state
history = []
def chat(user_input: str) -> str:
response = chain.invoke({"history": history, "input": user_input})
history.append(HumanMessage(content=user_input))
history.append(AIMessage(content=response))
return response
# Main conversation loop
print("Nova: Hello! I am Nova. Ask me anything. Type 'quit' to exit.\n")
while True:
user_text = input("You: ").strip()
if not user_text:
continue
if user_text.lower() in ["quit", "exit", "bye"]:
print("Nova: Goodbye! Have a great day!")
break
response = chat(user_text)
print(f"Nova: {response}\n")
Run this script and you have a working chatbot that remembers everything you say during the session. The history list grows with each turn. The MessagesPlaceholder injects it into the prompt automatically.
System Message Best Practices
The system message is the single most important part of your chatbot's prompt. It sets the tone, rules, and persona for the entire conversation. A weak system message produces an AI that acts generic and inconsistent. A strong system message produces a focused, reliable assistant.
Weak System Message
"You are a helpful assistant."
This tells the model almost nothing. The AI fills in the blanks with its training defaults, which may not match your application's needs at all.
Strong System Message
"You are Nova, a customer support assistant for CloudStore, a cloud storage service. Your job: - Answer questions about CloudStore plans, pricing, and features - Help users with account issues and billing questions - Guide users through technical setup steps clearly Your rules: - Only discuss topics related to CloudStore and general cloud storage concepts - If a question is outside your scope, say: 'That is outside my area. Let me connect you with our team.' - Always be polite and patient - Do not make promises about pricing or features you are not certain about Tone: Professional but friendly. Use clear, jargon-free language."
This system message gives the AI a name, a specific job, clear behavioral rules, and a defined scope. The result is a chatbot that behaves consistently and predictably.
Handling Edge Cases in Conversation
Empty or Whitespace Input
def chat_safe(user_input: str) -> str:
# Reject empty input before calling the API
if not user_input or not user_input.strip():
return "Please type a message."
response = chain.invoke({"history": history, "input": user_input.strip()})
history.append(HumanMessage(content=user_input.strip()))
history.append(AIMessage(content=response))
return response
Very Long User Input
MAX_INPUT_LENGTH = 2000 # characters
def chat_with_limits(user_input: str) -> str:
if len(user_input) > MAX_INPUT_LENGTH:
return f"Your message is too long ({len(user_input)} characters). Please keep it under {MAX_INPUT_LENGTH} characters."
return chat_safe(user_input)
API Errors
def chat_resilient(user_input: str) -> str:
try:
return chat_safe(user_input)
except Exception as e:
# Log the error internally but show a friendly message to the user
print(f"Internal error: {e}")
return "I am having trouble connecting right now. Please try again in a moment."
Adding Commands to Your Chatbot
Real chatbots often support special commands: clearing history, changing language, switching modes. Handle these before calling the AI chain.
COMMANDS = {
"/clear": "Conversation history cleared.",
"/help": "Commands: /clear (clear history), /help (show help), /status (show info)",
"/status": None # Handled dynamically
}
def handle_command(command: str) -> str | None:
if command == "/clear":
history.clear()
return COMMANDS["/clear"]
if command == "/help":
return COMMANDS["/help"]
if command == "/status":
return f"Nova active. History: {len(history)} messages. Model: gpt-3.5-turbo"
return None # Not a command
def chat_with_commands(user_input: str) -> str:
# Check for command first
if user_input.startswith("/"):
result = handle_command(user_input.lower())
if result is not None:
return result
return "Unknown command. Type /help for available commands."
return chat_resilient(user_input)
Streaming Responses for Better User Experience
When the model generates long responses, the user stares at a blank screen waiting. Streaming displays tokens as they are generated, creating a typing effect that feels more natural.
def chat_streaming(user_input: str):
"""Print response as it streams, return full response."""
full_response = ""
print("Nova: ", end="", flush=True)
for chunk in chain.stream({"history": history, "input": user_input}):
print(chunk, end="", flush=True)
full_response += chunk
print() # Newline after response finishes
# Save the complete response to history
history.append(HumanMessage(content=user_input))
history.append(AIMessage(content=full_response))
return full_response
The flush=True parameter forces Python to display each chunk immediately rather than buffering it. Without this, the entire response would still appear all at once at the end.
Context-Aware Follow-Up Questions
A common pattern in chatbots is asking clarifying questions based on the conversation context. You can build this by having the model check whether it has enough information before answering.
clarifying_prompt = ChatPromptTemplate.from_messages([
("system", """You are a helpful assistant. When a user asks a question:
1. If you have all the information needed, answer directly.
2. If you need clarification to give a good answer, ask ONE specific question.
Never ask multiple questions at once. Be direct and helpful."""),
MessagesPlaceholder(variable_name="history"),
("human", "{input}")
])
With this system message, the chatbot naturally asks for details when needed and answers directly when it has enough context. The conversation feels genuinely helpful rather than scripted.
Building a Context Summary Display
For long conversations, it helps to show the user a brief summary of what the AI knows about them. This builds trust and helps users verify that the AI has the right information.
def get_context_summary() -> str:
"""Ask the AI to summarize what it knows about the user from history."""
if not history:
return "No conversation history yet."
summary_prompt = ChatPromptTemplate.from_messages([
("system", "Summarize the key facts you know about the user from this conversation in 2-3 bullet points. Be concise."),
MessagesPlaceholder(variable_name="history"),
("human", "What do you know about me from our conversation?")
])
summary_chain = summary_prompt | model | parser
return summary_chain.invoke({"history": history})
# Show summary when user types /status
print(get_context_summary())
Detecting Conversation Topic Changes
Some applications need to know when the user switches topics so they can reset context or apply different handling logic. Build a simple topic detection step:
def detect_topic_change(current_input: str) -> bool:
"""Return True if the user seems to be changing topics."""
if len(history) < 2:
return False
detection_prompt = ChatPromptTemplate.from_messages([
("system", "Classify whether the new message is on a completely different topic from the conversation history. Reply with only: same_topic or new_topic"),
("human", "History summary: {history_summary}\n\nNew message: {new_message}")
])
# Get last few messages as summary
recent = history[-4:] if len(history) >= 4 else history
history_text = " | ".join([f"{m.type}: {m.content}" for m in recent])
detection_chain = detection_prompt | model | parser
result = detection_chain.invoke({
"history_summary": history_text,
"new_message": current_input
})
return "new_topic" in result.lower()
Conversation Flow Diagram
User types message
│
▼
Is it a command? ──Yes──▶ Handle command, return result
│No
▼
Is input valid? ──No───▶ Return validation error message
│Yes
▼
Build prompt:
[System] + [History] + [New Human Message]
│
▼
Send to Chat Model
│
▼
Receive AIMessage
│
▼
Extract text with StrOutputParser
│
▼
Save [HumanMessage, AIMessage] to history
│
▼
Display response to user
│
▼
Loop back to "User types message"
Testing Your Chatbot
Manual testing (typing messages yourself) reveals surface issues. Systematic testing catches deeper problems. Write tests that check specific behaviors:
def test_memory():
"""Verify the chatbot remembers facts across turns."""
test_history = []
def test_chat(msg):
resp = chain.invoke({"history": test_history, "input": msg})
test_history.append(HumanMessage(content=msg))
test_history.append(AIMessage(content=resp))
return resp
test_chat("My name is Vikram and I am a teacher.")
response = test_chat("What is my profession?")
assert "teacher" in response.lower(), f"Memory test failed. Response: {response}"
print("Memory test passed!")
def test_system_prompt_adherence():
"""Verify the chatbot stays in scope."""
test_history = []
response = chain.invoke({
"history": test_history,
"input": "Tell me the stock price of Apple"
})
# Should redirect if out of scope
print(f"Out-of-scope response: {response}")
test_memory()
test_system_prompt_adherence()
Conversation Export Feature
Users often want to save or share their conversations. Add a simple export function:
def export_conversation(history: list, filename: str = "conversation.txt"):
"""Export conversation to a readable text file."""
lines = ["=== Conversation Export ===\n"]
for msg in history:
speaker = "You" if msg.type == "human" else "Nova"
lines.append(f"{speaker}: {msg.content}\n")
with open(filename, "w") as f:
f.writelines(lines)
print(f"Conversation saved to {filename}")
Summary
A complete conversational application needs a clear system prompt, a history management mechanism, input validation, command handling, error handling, and streaming output. Chat Models process structured message lists where each message has a type (system, human, or ai). The conversation loop builds the message list incrementally, appending new turns after each exchange. Testing specific behaviors catches memory and scope issues early.