BURT CONSULTANTS
Topic

inference

3 pieces tagged “inference”.

Briefing

Why production LLM serving is now a senior-tier specialisation.

Senior MLOps engineers increasingly command a premium because LLM serving requires deep knowledge of inference architecture, cost optimisation and scalable infrastructure.

llmmlopsinferencecost
2 min read
Briefing

Disaggregated inference: lowering token cost for agentic AI.

DigitalOcean’s launch of NVIDIA Dynamo 1.0 claims 67 per cent higher GPU throughput and 79 per cent lower latency. The underlying idea — disaggregated inference — matters for agentic workloads.

nvidiadynamoinferenceagentic ai
3 min read
Briefing

Cache-aware inference routing: how Dynamo cuts LLM latency on AKS.

A Microsoft and Nvidia demonstration shows that KV-cache-aware routing can reduce Time-To-First-Token by around 20x on Azure Kubernetes Service. The result has implications for any team running LLM inference at scale.

akskubernetesnvidiainference
2 min read

Browse all tags →