Databricks Model Serving

A trained machine learning model delivers value only when other systems can use it. Model Serving turns a registered model into a live web service that accepts requests and returns predictions in milliseconds. A mobile app, a website, or a nightly job sends data to the endpoint and receives an answer.

The Restaurant Kitchen Analogy

Training a model resembles testing recipes in a kitchen. Serving a model resembles opening the restaurant. Customers place orders through a window, and the kitchen returns finished dishes without sharing the recipe. A serving endpoint plays the role of that window.

MODEL SERVING FLOW
──────────────────────────────────────────────────────────
Client app ──► HTTPS request ──► SERVING ENDPOINT ──► Model
(sends data)   (with a token)    (managed by Databricks)  │
     ▲                                                    │
     └─────────────────── prediction ◄────────────────────┘

What Model Serving Provides

• A REST API URL for each model.
• Automatic scaling that adds capacity when requests rise.
• Scale to zero, which pauses idle endpoints to reduce cost.
• Version management for safe updates.
• Request and response logging for monitoring.
• Access control through Unity Catalog permissions and tokens.

Types of Endpoints

Custom Model Endpoints

These endpoints serve models you train yourself, such as scikit-learn, XGBoost, or PyTorch models logged with MLflow.

Foundation Model Endpoints

Databricks hosts large language models and exposes them through the same style of API. Applications call those endpoints without managing any model infrastructure.

External Model Endpoints

An external endpoint acts as a governed gateway to a model hosted by another provider. Teams manage keys, limits, and logs in one location.

Prepare the Model

Serving starts with a model logged to MLflow and registered in Unity Catalog. The registration step gives the model a three-part name and a version number.

import mlflow
mlflow.set_registry_uri("databricks-uc")

mlflow.sklearn.log_model(
    sk_model=trained_model,
    artifact_path="model",
    registered_model_name="main.ml.churn_model"
)

Create an Endpoint

1. Open the Serving page from the sidebar.
2. Choose Create serving endpoint and type a name.
3. Select the registered model and the version to serve.
4. Pick the compute size and decide whether to allow scale to zero.
5. Click Create and wait until the status reads Ready.

Compute size controls how many requests the endpoint handles at once. Small sizes suit testing, and larger sizes suit busy applications.

Send a Request

Any program that speaks HTTP can call the endpoint. The example below uses Python with a personal access token. Replace the placeholder values with your own workspace address and token.

import requests

url = "https://YOUR-WORKSPACE-HOST/serving-endpoints/churn-endpoint/invocations"
headers = {"Authorization": "Bearer YOUR_TOKEN",
           "Content-Type": "application/json"}

payload = {"dataframe_records": [
    {"tenure_months": 8, "monthly_bill": 45.5, "support_calls": 3}
]}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

EXAMPLE RESPONSE
──────────────────────────────────────────────────────────
{"predictions": [0.82]}

Meaning: the model estimates an 82 percent chance that this
customer will cancel the subscription.

Keep tokens secret. Store them in a secret manager and never place them in code that you share.

Update a Model Safely

A serving endpoint can host two versions of the same model and split traffic between them. A team sends ten percent of requests to the new version and watches the results. The team raises the share when the new version performs well.

TRAFFIC SPLIT
──────────────────────────────────────────────────────────
              ┌──► Version 3 (current)  90% of requests
Requests ─────┤
              └──► Version 4 (new)      10% of requests

Monitor the Endpoint

Inference tables capture incoming requests and outgoing predictions. Teams review these tables to check speed, error rates, and drift. Drift means that live data starts to differ from the training data, which lowers accuracy over time.

Metric	What It Tells You
Latency	How fast the endpoint answers
Error rate	How many requests fail
Request volume	How busy the endpoint stays
Prediction spread	Whether outputs shift from normal ranges

Example: Subscription Churn Alert

A streaming service trains a model that predicts which subscribers may cancel. The billing system calls the endpoint each night with fresh customer data. Customers with a high score receive a friendly reminder about new content.

CHURN ALERT PIPELINE
──────────────────────────────────────────────────────────
Nightly job ──► send customer rows ──► Serving endpoint
                                             │
                                  score for each customer
                                             ▼
                          score above 0.7 ──► reminder list ──► email team

Key Points

• Model Serving publishes a registered model as a live REST endpoint.
• Endpoints scale automatically and can scale down to zero.
• Custom, foundation, and external model endpoints cover different needs.
• Applications call the endpoint with an HTTPS request and a secure token.
• Traffic splitting lets teams test new versions on a small share of requests.
• Inference tables support monitoring for speed, errors, and drift.