TensorFlow Lite for Mobile
TensorFlow Lite (TFLite) is a framework that runs TensorFlow models on devices with limited resources: smartphones, tablets, microcontrollers, Raspberry Pis, and embedded systems. A model trained on a powerful server can be converted to TFLite format and deployed to millions of mobile devices where it runs offline, with low latency, and without sending any data to the cloud. This makes machine learning private, fast, and available even without an internet connection.
Why Mobile Needs a Special Format
Server (Full TensorFlow) Mobile (TFLite) ────────────────────────────────────────────────────── CPU: 64 cores CPU: 4–8 cores RAM: 256 GB RAM: 4–12 GB Storage: Terabytes Storage: 64–256 GB Power: Unlimited Power: Battery Internet: Always on Internet: Optional Model size: 100 MB+ Model size: 1–10 MB ──────────────────────────────────────────────────────
A model that runs comfortably on a server would drain a smartphone's battery in minutes and consume too much memory to run alongside other apps. TFLite solves this through a combination of model format optimization and hardware-specific acceleration.
The TFLite Conversion Pipeline
[Full TensorFlow Model]
Trained on server
Format: SavedModel or Keras .h5
│
▼
[TFLite Converter]
Applies optimizations:
- Quantization (optional)
- Op fusion (combines adjacent operations)
- Dead node elimination
│
▼
[.tflite File]
Compact binary format
Typically 4–10× smaller than the original
│
▼
[Mobile App]
TFLite Interpreter loads and runs the model
Android: Java/Kotlin API
iOS: Swift/Objective-C API
Python: For testing and embedded Linux
Converting a Keras Model to TFLite
import tensorflow as tf
# Step 1: Train and save your model (assuming it is already trained)
model.save('my_model.keras')
# Step 2: Convert to TFLite
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Step 3: Save the .tflite file
with open('model.tflite', 'wb') as f:
f.write(tflite_model)
print(f"TFLite model size: {len(tflite_model) / 1024:.1f} KB")
Converting From a SavedModel
# If you saved in SavedModel format
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model_path/')
tflite_model = converter.convert()
Quantization: Making Models Even Smaller and Faster
Quantization reduces the precision of the numbers stored in the model. A standard TensorFlow model stores weights as 32-bit floating-point numbers (float32). Quantization compresses these to 8-bit integers (int8), cutting the model size to about one-quarter while making inference 2–4× faster on hardware that supports integer math.
Quantization Effect:
────────────────────────────────────────────────
float32 int8 quantized
Model size 12 MB 3 MB
Inference 40 ms 12 ms
Accuracy 94.2% 93.8% (small drop)
────────────────────────────────────────────────
Dynamic Range Quantization (Easiest)
Quantizes weights only. Activations stay as float32. Best for models where size reduction is the priority.
converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] tflite_model = converter.convert()
Full Integer Quantization (Most Hardware-Compatible)
Quantizes both weights and activations to int8. Requires a small representative dataset to calibrate the activation value ranges.
import numpy as np
def representative_dataset():
# Provide 100–500 samples from your training or validation set
for i in range(200):
sample = x_train[i:i+1].astype(np.float32)
yield [sample]
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
Float16 Quantization
Reduces model size by half with virtually no accuracy loss. Better choice than int8 when accuracy is critical.
converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] tflite_model = converter.convert()
Running a TFLite Model With Python (Testing)
import tensorflow as tf
import numpy as np
# Load the TFLite model
interpreter = tf.lite.Interpreter(model_path='model.tflite')
interpreter.allocate_tensors()
# Get input and output tensor details
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print("Input shape:", input_details[0]['shape'])
print("Output shape:", output_details[0]['shape'])
# Run inference on one sample
sample = np.random.random((1, 224, 224, 3)).astype(np.float32)
interpreter.set_tensor(input_details[0]['index'], sample)
interpreter.invoke()
output = interpreter.get_tensor(output_details[0]['index'])
print("Prediction:", output)
Deploying on Android
To use a TFLite model in an Android app, add the TFLite library to your project's Gradle file and place the .tflite file in the app's assets/ folder.
Gradle dependency (build.gradle): implementation 'org.tensorflow:tensorflow-lite:2.14.0' implementation 'org.tensorflow:tensorflow-lite-support:0.4.4'
// Kotlin example
val model = Interpreter(loadModelFile(context, "model.tflite"))
val input = Array(1) { FloatArray(inputSize) }
val output = Array(1) { FloatArray(numClasses) }
model.run(input, output)
Deploying on iOS
For iOS apps, add TFLite through CocoaPods and import the model the same way.
# Podfile pod 'TensorFlowLiteSwift', '~> 2.14'
// Swift example let interpreter = try Interpreter(modelPath: modelPath) try interpreter.allocateTensors() try interpreter.copy(inputData, toInputAt: 0) try interpreter.invoke() let output = try interpreter.output(at: 0)
Hardware Acceleration on Mobile
TFLite supports hardware delegates — specialized processors that run neural network operations much faster than the CPU:
- GPU Delegate — runs float32 operations on the mobile GPU. Typically 2–7× faster than CPU.
- NNAPI Delegate (Android) — uses Android's Neural Networks API, which may route operations to a dedicated NPU (Neural Processing Unit) on supported chips.
- Core ML Delegate (iOS) — runs models on Apple's Neural Engine, the dedicated AI chip inside A-series and M-series chips.
- Hexagon Delegate (Qualcomm) — runs quantized models on the Qualcomm DSP for very low power consumption.
// Kotlin: Enable GPU delegate val options = Interpreter.Options() options.addDelegate(GpuDelegate()) val model = Interpreter(modelFile, options)
TFLite Model Benchmark Tool
TFLite includes a benchmarking tool that measures inference latency on the target device. Run it from the command line on an Android device connected via ADB:
adb push model.tflite /data/local/tmp/ adb shell /data/local/tmp/benchmark_model \ --graph=/data/local/tmp/model.tflite \ --num_threads=4 \ --use_gpu=true
The output reports average latency in milliseconds, helping you decide whether additional optimization (more quantization, model pruning, or architecture changes) is needed before shipping.
Choosing the Right Optimization Strategy
Goal Recommended Approach ───────────────────────────────────────────────────────────── Reduce model size Dynamic range quantization Maximum speed on mobile GPU Float16 quantization + GPU delegate Maximum speed on mobile CPU Full int8 quantization + NNAPI delegate Preserve accuracy Float16 quantization Very tiny devices Full int8 + model pruning + smaller architecture ─────────────────────────────────────────────────────────────
TensorFlow Lite bridges the gap between powerful server-side machine learning and practical on-device applications. The same model that detects diseases, translates languages, or recognizes faces can run privately and instantly on the device in a user's hand. The next topic covers TensorFlow Serving, which takes the opposite approach: keeping models on a server and exposing them through a REST API for web and backend applications.