TFLite(LiteRT) - Generative AI
Overview
The Generative AI section describes how to run large language model (LLM) and vision-language model (VLM) inference on Genio platforms running Yocto. MediaTek provides pre-compiled Deep Learning Archive (DLA) model packages and built-in command-line tools to enable immediate evaluation without requiring a host-side build environment.
For performance data and model downloads, refer to TFLite(LiteRT) - Generative AI.
Supported Platforms and DLA Grouping
Generative AI model packages are compiled for specific DLA hardware. Genio 360, 360P, 420, 520, and 720 share the same DLA binary, while MT8875, MT8883, and MT8893 each have their own dedicated DLA binary. Select the package that matches your platform group when downloading.
Platform |
DLA Package |
|---|---|
Genio 360 / 360P / 420 / 520 / 720 |
Shared |
MT8875 |
Dedicated |
MT8883 |
Dedicated |
MT8893 |
Dedicated |
Built-in Inference Tools
The following inference tools are pre-installed in the Yocto image under /usr/share/tmp/llm/:
Tool |
Description |
|---|---|
|
Runs LLM inference from a prompt file using a YAML configuration. See llm_cmdline_tool. |
|
Runs VLM inference combining an image and text prompt. See vlm_cmdline_tool. |