# Inference and performance

> Exploring what makes model-powered software responsive and useful.

Canonical page: https://rajakarthik.com/work/inference-systems/

## Overview

Running a language model involves a series of practical tradeoffs. Responsiveness, resource use, and reliability all matter to the experience of using it.

I am interested in the fundamentals of inference performance and in how to reason about bottlenecks using reproducible, publicly discussable experiments.

## Topics I explore

- Latency and throughput as engineering concepts
- Measuring responsiveness under changing workloads
- Resource efficiency and the user experience
- Clear measurement, repeatability, and communication

Good engineering starts with a useful question and a measurement you can trust.