TensorFlow Functional API
The Functional API is TensorFlow Keras's most flexible way to build neural networks. While the Sequential API works perfectly for models where data flows in a straight line, the Functional API handles models with branches, merges, shared layers, and multiple inputs or outputs. Real-world architectures — ResNet, Inception, BERT, and U-Net — all require the Functional API because they have complex, non-linear connection patterns.
The Sequential API's Limitation
The Sequential API forces data to travel in one direction through a single chain of layers. Some powerful architectures need layers to:
- Receive input from multiple previous layers simultaneously
- Send output to multiple downstream layers
- Skip layers entirely (residual connections)
- Accept more than one input tensor (multi-modal models)
- Produce more than one output tensor (multi-task models)
The Functional API supports all of these patterns.
Core Concept: Layers as Functions
In the Functional API, every layer behaves like a mathematical function. You call the layer on a tensor and get a new tensor back. This lets you chain layers together explicitly, routing data exactly where you want it to go.
# Sequential approach (implicit routing):
model = tf.keras.Sequential([
Dense(64, activation='relu'),
Dense(32, activation='relu'),
Dense(10, activation='softmax')
])
# Functional approach (explicit routing):
inputs = tf.keras.Input(shape=(128,))
x = tf.keras.layers.Dense(64, activation='relu')(inputs)
x = tf.keras.layers.Dense(32, activation='relu')(x)
outputs = tf.keras.layers.Dense(10, activation='softmax')(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
The result is identical, but notice that in the Functional API you can see exactly how data flows from inputs through each layer to outputs.
Building a Multi-Input Model
Suppose you build a model that predicts a house price using two types of input data: numerical features (square footage, number of rooms) and image features (photo of the house). The Sequential API cannot combine two different inputs — the Functional API handles this easily.
Diagram — Multi-Input Architecture:
[Numerical Data] [House Photo]
(12 features) (128×128×3 pixels)
│ │
▼ ▼
[Dense(32, relu)] [Conv2D→Pool×2→Flatten]
│ │
└──────────┬────────────┘
▼
[Concatenate Layer]
(merges both streams)
│
▼
[Dense(64, relu)]
│
▼
[Dense(1, linear)]
(price prediction)
import tensorflow as tf # Input 1: numerical features num_input = tf.keras.Input(shape=(12,), name='numerical') x1 = tf.keras.layers.Dense(32, activation='relu')(num_input) # Input 2: image features img_input = tf.keras.Input(shape=(128, 128, 3), name='image') x2 = tf.keras.layers.Conv2D(32, 3, activation='relu')(img_input) x2 = tf.keras.layers.MaxPooling2D(2)(x2) x2 = tf.keras.layers.Conv2D(64, 3, activation='relu')(x2) x2 = tf.keras.layers.MaxPooling2D(2)(x2) x2 = tf.keras.layers.Flatten()(x2) # Merge both streams combined = tf.keras.layers.Concatenate()([x1, x2]) # Shared head x = tf.keras.layers.Dense(64, activation='relu')(combined) output = tf.keras.layers.Dense(1)(x) # Price prediction # Build model with two inputs model = tf.keras.Model(inputs=[num_input, img_input], outputs=output) model.summary()
Building a Multi-Output Model
A multi-output model solves more than one problem at once. For example, a model that looks at a person's face and simultaneously predicts age (a number) and emotion (a category) produces two outputs from one input.
Diagram — Multi-Output Architecture:
[Face Image]
│
▼
[Shared CNN Base]
│
├─────────────────┐
▼ ▼
[Dense(64)] [Dense(64)]
│ │
▼ ▼
[Dense(1)] [Dense(7, softmax)]
Age Prediction Emotion (7 classes)
import tensorflow as tf
inputs = tf.keras.Input(shape=(64, 64, 3))
# Shared base
x = tf.keras.layers.Conv2D(32, 3, activation='relu')(inputs)
x = tf.keras.layers.MaxPooling2D(2)(x)
x = tf.keras.layers.Conv2D(64, 3, activation='relu')(x)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
# Output branch 1: age prediction
age_branch = tf.keras.layers.Dense(64, activation='relu')(x)
age_output = tf.keras.layers.Dense(1, name='age')(age_branch)
# Output branch 2: emotion classification
emotion_branch = tf.keras.layers.Dense(64, activation='relu')(x)
emotion_output = tf.keras.layers.Dense(7, activation='softmax',
name='emotion')(emotion_branch)
model = tf.keras.Model(inputs=inputs,
outputs=[age_output, emotion_output])
model.compile(
optimizer='adam',
loss={
'age': 'mse',
'emotion': 'categorical_crossentropy'
},
metrics={
'age': 'mae',
'emotion': 'accuracy'
}
)
Residual Connections: The Skip Connection Pattern
ResNet (Residual Network) introduced skip connections — a path that bypasses one or more layers and adds the input directly to the output of a later layer. This solved the vanishing gradient problem in very deep networks and enabled models with 50, 100, or even 1,000 layers to train successfully.
Residual Block Diagram:
Input Tensor
│──────────────────┐
▼ │ (skip connection)
[Dense(64, relu)] │
│ │
▼ │
[Dense(64)] │
│ │
└────────[Add]─────┘
│
▼
[Activation]
import tensorflow as tf
def residual_block(x, units):
shortcut = x # Save the input
x = tf.keras.layers.Dense(units, activation='relu')(x)
x = tf.keras.layers.Dense(units)(x)
x = tf.keras.layers.Add()([x, shortcut]) # Add skip connection
x = tf.keras.layers.Activation('relu')(x)
return x
# Build a model with residual blocks
inputs = tf.keras.Input(shape=(128,))
x = tf.keras.layers.Dense(64, activation='relu')(inputs)
x = residual_block(x, 64)
x = residual_block(x, 64)
outputs = tf.keras.layers.Dense(10, activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
The Add layer merges the input (shortcut) with the output of two Dense layers. If those Dense layers fail to learn anything useful, the shortcut simply passes the original data through. This makes deeper models much more stable to train.
Shared Layers
The Functional API allows the same layer object to be reused across different paths. This forces two branches to use identical weights — useful for models that compare two inputs, like a model that determines whether two sentences have similar meaning.
import tensorflow as tf shared_layer = tf.keras.layers.Dense(64, activation='relu') input_a = tf.keras.Input(shape=(100,)) input_b = tf.keras.Input(shape=(100,)) # Same layer processes both inputs encoded_a = shared_layer(input_a) encoded_b = shared_layer(input_b) # Compute absolute difference diff = tf.keras.layers.Subtract()([encoded_a, encoded_b]) output = tf.keras.layers.Dense(1, activation='sigmoid')(diff) model = tf.keras.Model([input_a, input_b], output)
Both input_a and input_b pass through the exact same Dense layer with the exact same weights. Training updates those shared weights to produce useful encodings for both inputs simultaneously.
Inspecting the Model Graph
The Functional API builds an explicit computation graph. You can inspect and visualize this graph:
# Print a text summary
model.summary()
# Save a visual diagram of the model architecture
tf.keras.utils.plot_model(
model,
to_file='model_diagram.png',
show_shapes=True,
show_layer_names=True
)
The diagram shows every layer as a box and every connection as an arrow. This visual makes complex architectures much easier to understand and debug.
Functional API vs. Sequential API Summary
Feature Sequential Functional ────────────────────────────────────────────────── Straight-line models Yes Yes Multi-input models No Yes Multi-output models No Yes Residual connections No Yes Shared layers No Yes Model visualization Limited Full graph Code complexity Simple Moderate
The Functional API unlocks the full design space of neural network architectures. Any network structure you can draw as a directed graph, you can build with the Functional API. The next topic takes this further by showing you how to create completely custom layers — building blocks with your own learnable behavior that go beyond what Keras provides out of the box.