MediaTek NPU and LiteRT: Powering the next generation of on-device AI

Summary

The article discusses the advancements in on-device AI powered by MediaTek's Neural Processing Units (NPUs) and the introduction of the LiteRT NeuroPilot Accelerator. It highlights the challenges developers face in deploying AI models on NPUs due to the diversity of SoC variants and the lack of tailored infrastructure. The LiteRT NeuroPilot Accelerator aims to simplify this process by providing a unified API, supporting both Ahead-of-Time (AOT) and on-device compilation workflows. Key features include rich generative AI capabilities, efficient cross-platform development, and seamless integration with existing ML pipelines, enabling high-performance applications across various devices.

Key Learnings

• 1The LiteRT NeuroPilot Accelerator streamlines the deployment of AI models on MediaTek NPUs, addressing the complexities of hardware fragmentation.
• 2Developers can choose between AOT and on-device compilation strategies, optimizing for either initialization speed or flexibility in model distribution.
• 3The integration of a new C++ API enhances the efficiency of building ML pipelines, particularly for real-time applications involving camera and video processing.
• 4The collaboration with MediaTek enables the use of state-of-the-art generative AI models like the Gemma family, significantly improving on-device capabilities.

Who Should Read This

Senior Embedded Systems Engineers implementing on-device AI solutions for diverse hardware platforms

Test Your Knowledge

Topics