Question
An organization recently created a custom model that integrates with a large language model (LLM). The developer notices that the application programming interface (API) costs have increased. Which of the following is the best control to reduce cost?
- A. Implementing prompt templates
- B. Increasing central processing unit (CPU) and memory
- C. Reducing the model size
- D. Adjusting token limits Correct
Explanation
LLM APIs are billed per token, so setting limits on input and output tokens per request or per user directly caps what each call can cost. Prompt templates standardize prompts but do not cap consumption, adding CPU and memory raises infrastructure cost rather than API cost, and the size of the hosted model behind a third-party API is not something the organization controls.
