pavelfedorov.io
ML Systems Engineer | LLM & Speech Inference Optimization

hi, I'm Pavel Fedorov

I’m building and optimizing inference systems for LLMs and speech
Focused on real-time serving, GPU performance, and making inference
cheaper than public APIs, while building the infrastructure that keeps it reliable

Selected experience

Building since 2018
2024–nowCurrent focus

Production ML systems

LLM & speech inference

Building and optimizing production inference systems for LLMs and speech, from serving engines to deployment on customer-managed GPU infrastructure

  • LLM serving: custom vLLM and SGLang forks, multi-GPU deployments, request scheduling, cache reuse, and faster worker startup
  • Worked on deployments across 100+ H200/B200 GPUs, including distributed prefill/decode setups
  • Develop and profile custom GPU kernels
  • Real-time ASR and TTS: TensorRT compilation, quantization, batching improvements, and a custom TTS serving engine
  • On-premises deployment and capacity planning: adapting inference to customer hardware, latency requirements, and production workloads
2023–2024

brask ↗

MLOps Engineer · USA · Remote

Built scalable services for speech recognition, text-to-speech, and voice conversion. Worked on multilingual speech processing and speaker diarization, alongside translation and lip-sync pipelines

Created a GPU pipeline combining more than 10 models for training-data collection and processing. Synthesized over 150 voices and built a fast fine-tuning pipeline that ran in under one minute per voice

Developed a gRPC task balancer that improved GPU worker utilization by 50%, and doubled inference speed for models in translation and lip-sync pipelines. Led a team of four ML and DevOps engineers, taking models from notebooks into production services

2022–2023

Mars (food and pet care) ↗

Machine Learning Engineer · Moscow · On-site

Led a team of five ML engineers at Mars’ Russian subsidiary, building a real-time video segmentation service for factory operations. Worked across data collection, automatic labeling, model training, and production deployment

Built scalable GPU inference pipelines for data collection and training across more than 10 models. Worked closely with business teams to turn operational requirements into an MVP in four months

2021–2023

whalemap ↗

Machine Learning Engineer · London · Remote

Built ML pipelines for blockchain address tagging and real-time analytics. Reworked ETL processing to improve throughput by 50%, and automated data ingestion from more than 150 API endpoints

2020–2021

emovi ↗

Data Scientist / Backend Developer · Moscow · On-site

Transformer-based recommendation and search, deployed to backend services and mobile apps, including on-device ML inference

2018–2020

MegaFon (#1 Russian telecom company) ↗

Machine Learning Intern · Moscow · On-site

Researched transformer-based classification of customer requests and collected training data to support request handling automation

Inference optimization projects

01 /

LLM inference & GPU performance

Inference
  • Serving large models like DeepSeek V4, Kimi K3, and GLM-5.3 across multiple GPUs and nodes: tensor, pipeline, and expert parallelism, disaggregated prefill/decode, and KV-cache management. Tuning compute, memory, and communication for the target hardware and workload
  • Fixed and optimized custom vLLM and SGLang forks for production workloads
  • Used Nsight Systems to trace CPU/GPU execution, identify synchronization and memory-transfer bottlenecks, and guide changes to kernels, batching, and data flow
  • Optimized GPU execution with Triton kernels, fused operations, and hardware-aware memory access, targeting the parts of the model that limited serving performance
  • Optimized LLM execution across CPU and GPU backends, reducing prefill latency by 3× and doubling decoding throughput without degrading accuracy
  • Tuned KV-cache allocation, memory pinning, and GPU communication, increasing token throughput by 40%, reducing p99 latency by 30%, and raising GPU utilization from approximately 55% to 82%
  • Reduced model-worker startup time by 5×, to under one minute, using NVIDIA Dynamo Snapshot, compilation-cache reuse, warmup improvements, and remote caching of compiled artifacts
  • Working primarily with NVIDIA Blackwell (RTX PRO 6000 Blackwell, B200, B300) and Hopper (H100, H200) GPUs
  • Deployed large models on customer on-premises GPU clusters for development with sensitive data, including integration with Claude Code and Codex
vLLM · SGLang · NVIDIA TensorRT-LLM · NVIDIA Dynamo · Nsight Systems · Triton (GPU kernels) · FlashInfer
02 /

LLM inference costs

Cost optimization
  • Combined speculative decoding, KV-cache optimization, and dynamic batching to increase sustained throughput by 46% and reduce infrastructure costs by 35%
  • Implemented KEDA-based autoscaling driven by throughput, latency, and queue metrics, reducing GPU costs by more than 28%
  • Optimizing self-hosted LLM serving to generate more tokens per second from each GPU and reduce cost per token. In one case, benchmark-based estimates showed approximately 87% lower GPU inference costs than a hosted API under sustained load
03 /

Real-time speech systems

Speech
  • Optimized streaming ASR and TTS by rebuilding pipeline artifacts, compiling models with TensorRT, applying quantization, and improving batching and request scheduling for the target hardware and workload
  • Co-developed a custom TTS inference engine as an alternative to NVIDIA Triton Inference Server, with scheduling designed around first-audio latency, continuous audio delivery, and efficient batching across active sessions
  • ASR: optimized NVIDIA Nemotron and Conformer inference, reducing per-chunk latency from 400 ms to 50 ms in a real-time setup and scaling concurrent streams from 30 to 250 on a single NVIDIA L40S GPU
  • TTS: accelerated VITS and MOSS inference, reducing time to first byte (TTFB) from 800 ms to 100 ms and improving batching to scale concurrent streams from 30 to 150 on a single NVIDIA L40S GPU
  • Worked closely with the model-training team on quantization and inference optimization, maintaining ASR and TTS quality
ASR · TTS · NVIDIA Triton · TensorRT · gRPC
04 /

Speech inference costs

Cost optimization
  • Reduced speech recognition and synthesis costs through batching, request scheduling, and better GPU utilization. In selected sustained-load scenarios, estimated ASR and TTS inference costs were 6–12× lower than hosted API pricing. Savings depend on the provider and can be higher in some cases
05 /

Generative audio & video

Applied ML
  • Production pipelines for voice conversion, translation, and lip-sync. Connecting models into services and making better use of GPU workers
  • Improved the reliability of production audio and video generation services through monitoring and alerting
PyTorch · Diffusion · GPU pipelines

Blog

All articles

A little background

I’m an ML systems engineer working at the intersection of machine learning and infrastructure. My path started with recommendations and search, moved through computer vision and generative media, and led to what I spend most of my time on today: making LLM and speech inference fast, dependable, and useful in production. I enjoy understanding how GPU chips and models work under the hood, and using that knowledge to make inference faster. I like working across the whole system: finding the bottleneck, making the change, and measuring what actually improved

Education

Higher School of Economics

Moscow · 2017–2021

BSc in Computer Science (backed by Yandex)
Machine Learning and Applications

  • Top 3 in Computer Science across post-Soviet countries (“Stanford of Russia”, 2017)
  • Paid my full tuition from my third year onward

Teaching

Higher School of Economics

Moscow · 2022–2024

Taught deep learning seminars and developed course material on graph neural networks