LangChain History Trimming

Long chats grow heavy. Each new turn adds tokens, and old turns often matter less than recent ones. History trimming keeps a conversation light by removing or shrinking older messages before each model call. A trimmed history cuts cost, speeds up replies, and prevents context window errors.

The Backpack Idea

Think of a hiking backpack with limited space. A smart hiker keeps water and a map, and leaves behind the heavy items that serve no purpose today. Trimming applies the same judgment to messages. The system rules stay, the latest turns stay, and the oldest chatter goes.

 BEFORE TRIMMING                     AFTER TRIMMING
 [System rules]                      [System rules]
 [Turn 1]  ← old                     [Turn 5]
 [Turn 2]  ← old                     [Turn 6]
 [Turn 3]  ← old                     [Turn 7]
 [Turn 4]  ← old                     [Turn 8]
 [Turn 5]
 [Turn 6]
 [Turn 7]
 [Turn 8]

Strategy 1: Keep the Last Messages

The helper trim_messages removes messages until the list fits your limit. Setting the strategy to "last" keeps the newest messages.

from langchain_core.messages import trim_messages

trimmed = trim_messages(
    messages,
    strategy="last",
    max_tokens=6,
    token_counter=len,
    include_system=True,
    start_on="human",
)

Passing len as the counter treats each message as one unit, so max_tokens=6 means six messages. Replace len with a model object to count real tokens.

trimmed = trim_messages(
    messages,
    strategy="last",
    max_tokens=1000,
    token_counter=model,
    include_system=True,
    start_on="human",
)

Why start_on Matters

A reply that begins with an AI message and no matching question confuses some models. Setting start_on="human" makes the trimmed list begin with a user message, which keeps the dialogue valid. The include_system setting keeps your rules even when older messages vanish.

Using Trimming Inside a Chain

A trimmer works as a runnable, so you can place it directly before the model.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

trimmer = trim_messages(
    strategy="last",
    max_tokens=1000,
    token_counter=model,
    include_system=True,
    start_on="human",
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful tutor."),
    MessagesPlaceholder("history"),
])

chain = prompt | trimmer | model

Strategy 2: Summarize the Past

Trimming discards information. Summarizing compresses it. A summary keeps names, goals, and decisions from early turns without the full word count.

 Old turns 1-10 → [Model writes a short summary]
                          |
                          v
 Prompt = [System] + [Summary] + [Turns 11-14] + [New question]

def summarize(old_messages):
    text = "\n".join(m.content for m in old_messages)
    reply = model.invoke(
        "Summarize the key facts and decisions in under 80 words:\n" + text
    )
    return reply.content

summary = summarize(messages[:-4])
recent = messages[-4:]

Build the next prompt from the summary plus the recent messages.

from langchain_core.messages import SystemMessage

compact = [SystemMessage(content="Conversation summary: " + summary)] + recent
reply = model.invoke(compact)

Choosing the Right Strategy

• Last messages suits quick chats where only recent context matters.
• Summary plus recent suits long support sessions or coaching apps where early facts matter.
• Both together suits large apps: summarize at a threshold, then trim the rest.

Deciding When to Summarize

Track the token count after each turn. Summarize when the history passes a threshold, such as 70 percent of your budget. Running a summary on every turn wastes calls.

Common Mistakes

• Trimming away the system message and losing the rules.
• Cutting a tool call away from its tool result, which breaks the message chain.
• Summarizing too often and paying for extra model calls.
• Trimming so aggressively that the assistant forgets the user's name or goal.

Real-World Scenario

A fitness coaching chat runs for weeks. Early messages hold important facts: the user's goal, an old knee injury, and available equipment. Messages from last Tuesday about breakfast matter much less. The developer summarizes everything older than ten messages into one short note and keeps the latest ten messages in full. The coach still remembers the knee injury, and the prompt stays small. A pure trim would have dropped the injury, so the summary protected a vital detail.

Practice Tasks

1. Create a list of twelve messages and trim it to the last four with trim_messages.
2. Turn include_system on and off and compare the results.
3. Write a summary function and test it on a ten-message conversation.
4. Decide a token threshold for summarizing and explain your choice in one sentence.

Key Points

• Trimming removes old messages to protect cost and context space.
• trim_messages keeps the last messages and the system rules.
• start_on="human" keeps the dialogue valid.
• Summaries preserve key facts from older turns.
• Trimming and summarizing combine well in long conversations.