Databricks Foundation Models

A foundation model is a large AI model trained on vast amounts of text, code, or images. It can answer questions, summarize documents, write drafts, and extract facts without special training for each task. Databricks hosts popular foundation models and offers them through simple endpoints, so developers use them without owning any GPU hardware.

The Expert on Call Analogy

Imagine a well-read consultant who answers questions by phone. You describe your problem, and the consultant replies in plain language. The consultant does not know your private business details unless you share them. A foundation model behaves the same way: it holds broad knowledge and relies on your prompt for specific context.

CALLING A FOUNDATION MODEL
──────────────────────────────────────────────────────────
Your app ──► prompt (instructions + question) ──► Endpoint
                                                     │
Your app ◄── generated answer ◄── Model reads and writes

How Databricks Serves These Models

Pay-per-Token Endpoints

Pay-per-token endpoints bill by the amount of text processed. They suit experiments, prototypes, and light usage. Setup takes no effort because Databricks manages the capacity.

Provisioned Throughput Endpoints

Provisioned throughput reserves dedicated capacity for steady, high-volume traffic. Performance stays predictable, and teams plan costs in advance. Fine-tuned models also use this style.

External Model Endpoints

An external endpoint routes requests to a model hosted by another provider. Companies manage keys, rate limits, and logs in one governed place.

ENDPOINT CHOICES
──────────────────────────────────────────────────────────
Prototype or low volume ──► Pay-per-token
Steady production load  ──► Provisioned throughput
Model from outside      ──► External model endpoint

Key Ideas for Prompting

Prompt

The prompt is the full text you send. A clear prompt contains the task, the context, and the format you expect.

System Message

The system message sets the role and rules, such as "You are a polite support assistant."

Temperature

Temperature controls creativity. A low value gives steady, predictable answers. A high value gives varied wording.

Tokens

A token is a small piece of text, roughly three-quarters of a word. Models read and write tokens, and every model has a maximum token limit for each request.

Call a Model from Python

Databricks endpoints accept requests in a format compatible with the popular OpenAI client library. Replace the placeholders with your workspace address, access token, and endpoint name.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_TOKEN",
    base_url="https://YOUR-WORKSPACE-HOST/serving-endpoints"
)

reply = client.chat.completions.create(
    model="YOUR-ENDPOINT-NAME",
    messages=[
        {"role": "system", "content": "You write short, friendly product summaries."},
        {"role": "user", "content": "Summarize: waterproof hiking boots with ankle support."}
    ],
    temperature=0.3,
    max_tokens=100
)

print(reply.choices[0].message.content)

The response contains the generated text inside choices. Store tokens in a secret scope and keep them out of shared notebooks.

Use a Model Inside SQL

The ai_query function calls an endpoint for every row of a table.

SELECT ticket_id,
       ai_query(
         'YOUR-ENDPOINT-NAME',
         CONCAT('Classify this ticket as billing, delivery, or product: ', ticket_text)
       ) AS category
FROM main.support.tickets;

BATCH CLASSIFICATION
──────────────────────────────────────────────────────────
tickets table (10,000 rows)
      │  ai_query calls the endpoint for each row
      ▼
tickets with a new category column

Common Tasks

Task	Sample Prompt Idea
Summarize	Summarize this report in three sentences.
Classify	Label this message as positive, neutral, or negative.
Extract	List the dates and amounts mentioned in this invoice.
Rewrite	Rewrite this note in a polite tone.
Translate	Translate this paragraph into Spanish.

Limits to Remember

Foundation models sometimes produce confident answers that contain mistakes. Models also lack knowledge of events after their training date and of your private data. Retrieval techniques supply fresh and private context. Review important outputs before they reach customers, and avoid sending sensitive personal data in prompts.

Example: Product Description Writer

An online store holds 5,000 products with short technical notes. A notebook loops through the table and asks a model to write a friendly two-sentence description for each item. Editors review a sample and approve the style before the descriptions go live.

DESCRIPTION WRITER
──────────────────────────────────────────────────────────
Product notes table ──► model writes descriptions ──► draft table
                                                          │
                                           editor reviews a sample
                                                          ▼
                                                  approved and published

Key Points

• A foundation model is a large, general-purpose AI model.
• Databricks serves models through pay-per-token, provisioned throughput, and external endpoints.
• Prompts, system messages, temperature, and token limits shape each response.
• Python code and the ai_query SQL function both call endpoints.
• Models can make mistakes, so human review protects quality.
• Secret scopes keep access tokens safe.