Google brings LiteRT.js to browsers for faster on-device AI inference

Google has introduced LiteRT.js, a JavaScript binding for its LiteRT on-device inference runtime, giving web developers a new way to run machine-learning and AI models directly inside mobile and desktop browsers. The July 9 announcement positions the project as an evolution path for teams that already use .tflite models and want lower-latency browser deployments without routing every request through a cloud service.

The key change is architectural. Instead of relying mainly on JavaScript-based kernels, Google says LiteRT.js exposes its native cross-platform runtime to web apps through WebAssembly. The first release includes an npm package, documentation, demos, and support for JavaScript or TypeScript applications that handle tasks such as object detection, text generation, audio processing, vector search, image upscaling, and depth estimation on the client side.

Why It Matters

Browser AI is becoming a practical deployment target because it can reduce server cost, keep sensitive inputs on the user device, and make interactive features feel faster. Google says LiteRT.js can use XNNPACK for CPU acceleration, WebGPU-backed ML Drift for GPU acceleration, and experimental WebNN support for neural processing units in Chrome and Edge. That hardware mix is important for developers trying to ship one AI feature across laptops, phones, and emerging AI PCs without maintaining separate native apps.

Google’s benchmark claims are broad but notable. In controlled tests on a 2024 Apple MacBook Pro with M4 silicon, the company says LiteRT.js outperformed other web runtimes by up to 3x across CPU and GPU inference for classical computer vision and audio models. It also says GPU or NPU execution through WebGPU or WebNN delivered 5x to 60x speedups compared with standard CPU execution for demanding real-time workloads. Those figures are Google’s own measurements, so production results will still depend on model size, browser support, device thermals, and graphics drivers.

The announcement also ties LiteRT.js into the wider model toolchain. Google highlights one-step PyTorch conversion through LiteRT Torch, tailored quantization through AI Edge Quantizer, pretrained models on Kaggle and the LiteRT Hugging Face community, and official LiteRT export support in the Ultralytics Python package for YOLO computer-vision models.

For teams building privacy-sensitive web tools, the release makes browser-native AI less experimental. The next test will be whether LiteRT.js can expand model coverage and WebNN acceleration quickly enough for developers to treat local inference as a default option rather than a specialized optimization.

Sources

• developers.googleblog.com

Cover photo by Pixabay on Pexels, used under the Pexels License.