Question
A company wants to use language models to create an application for inference on edge devices. The inference must have the lowest latency possible.
Which solution will meet these requirements?
- A. Deploy optimized small language models (SLMs) on edge devices. Correct
- B. Deploy optimized large language models (LLMs) on edge devices.
- C. Incorporate a centralized small language model (SLM) API for asynchronous communication with edge devices.
- D. Incorporate a centralized large language model (LLM) API for asynchronous communication with edge devices.
Explanation
According to AWS documentation, deploying optimized small language models (SLMs) on edge devices (Option A) will meet the requirement of lowest latency because SLMs require fewer computational resources and can process requests more quickly than large language models. Additionally, processing on edge devices reduces dependency on network communication, further minimizing latency.
