GenAI Forum on the Edge

Song Han
Visual Language Models for Edge AI 2.0

This talk presents edge AI innovations across the full stack: I’ll first present VILA (CVPR’24), a visual language model with multi-image reasoning and in-context learning capabilities. With strong zero shot learning capabilities, VILA 2.7B is deployable on Jetson Orin Nano. Followed by AWQ (MLSys’24), a 4-bit LLM quantization algorithm that boosts model efficiency, and TinyChat, an inference library that powering visual language model inference. VILA, AWQ and TinyChat enable advanced visual reasoning on the edge and brings new opportunities for edge AI applications.