Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher
LLM inference costs are rising amid growing AI usage.
“your AI needs to be put on a diet and everyone needs to start auditing and budgeting for token usage.”
The workshop highlighted the increasing costs associated with LLM inference as AI usage grows. Participants were encouraged to manage token usage effectively, reflecting a trend in the industry towards optimizing AI resource consumption.