Deep Learning Large Language Models

A large language model, or LLM, is a Transformer network trained on enormous amounts of text to predict what comes next. That single skill, repeated billions of times, produces systems that answer questions, summarize documents, translate languages, and write code. Chat assistants you meet online run on LLMs.

The Core Idea: Predict the Next Token

An LLM reads text and guesses the next small piece. Pieces are called tokens, and a token can be a word, part of a word, or a punctuation mark. The model assigns a probability to every possible next token and picks one.

Input:  "The sun rises in the"

Model probabilities:
  east    82%
  morning  9%
  sky      4%
  other    5%

Pick: "east"
Repeat with the new text to keep writing

Think of the autocomplete feature on a phone keyboard, scaled up massively and trained on a huge library.

Understanding Tokens

Sentence: "Unbelievable results today"
Tokens:   [Un] [believ] [able] [ results] [ today]
Token IDs: [3118] [24761] [481] [3504] [3432]

The model never sees letters directly. It sees numeric IDs, which an embedding layer converts into rich number lists that capture meaning.

How an LLM Is Built

Stage 1: Pretraining

The model reads vast collections of books, articles, websites, and code. It learns grammar, facts, writing styles, and reasoning patterns by predicting hidden tokens. This stage demands thousands of processors running for weeks.

Stage 2: Fine-Tuning

Developers train the pretrained model on curated examples of questions paired with high-quality answers. The model learns to follow instructions instead of merely continuing text.

Stage 3: Alignment with Human Feedback

People rank several model answers from best to worst. The system learns from those rankings to give responses that are helpful, honest, and safe.

Raw text -> Pretraining -> Fine-tuning -> Human feedback -> Assistant
 (knowledge)  (language skill)  (follows instructions)  (helpful behaviour)

Why Size Matters

Parameters are the learned weights of the model. Modern LLMs hold billions of them. More parameters, more data, and more computing power tend to improve ability. Larger models show surprising skills, such as solving multi-step problems, that smaller ones lack.

Context Window

The context window is the amount of text the model can consider at once. It includes your question and the answer so far. Text beyond the window falls out of view, and the model cannot recall it.

Context window of 8,000 tokens:
[ earlier chat ........ your latest question ][ answer being written ]
Oldest text drops out when the window fills

Common Uses

• Drafting and editing emails, reports, and study notes
• Summarizing long documents
• Translating between languages
• Explaining code and finding bugs
• Answering questions from a company help library

Known Limitations

Hallucination

An LLM sometimes states false information in a confident tone. It predicts plausible text, not verified truth. Check important facts against reliable sources.

Outdated Knowledge

The model knows only what its training data contained. Connecting it to search or document tools supplies fresh information.

Bias

Training text reflects human writing, including its flaws. Careful testing and filtering reduce harmful patterns but cannot remove every one.

Using LLMs Wisely

• Write clear, specific prompts with the context the model needs
• Verify facts, numbers, and citations
• Keep private data out of tools that lack proper protections
• Treat answers as drafts that a person reviews

Key Terms

• LLM - a very large Transformer trained on text
• Token - a small unit of text the model reads
• Pretraining - learning language from huge text collections
• Fine-tuning - training on curated examples for a goal
• Context window - the amount of text visible to the model at once
• Hallucination - a confident but false output