Anthropic Interview Question

Showed a design for an inference batching system "from a junior developer", asked to provide feedback/improvements/etc.