# hi, I'm Pavel Fedorov Canonical source: https://pavelfedorov.io/ ML Systems Engineer | LLM & Speech Inference Optimization I’m building and optimizing inference systems for LLMs and speech Focused on **real-time serving**, **GPU performance**, and **making inference cheaper than public APIs**, while building the infrastructure that keeps it reliable [Email](mailto:dm@pavelfedorov.io) · [LinkedIn](https://www.linkedin.com/in/pavel-fed0r0v/) · [GitHub](https://github.com/psfed0r0v) · [Telegram](https://t.me/psfedorov) ## Work: Selected experience Building since 2018 2024–now · Current focus ### Production ML systems LLM & speech inference Building and optimizing production inference systems for LLMs and speech, from serving engines to deployment on customer-managed GPU infrastructure - LLM serving: custom vLLM and SGLang forks, multi-GPU deployments, request scheduling, cache reuse, and faster worker startup - Worked on deployments across 100+ H200/B200 GPUs, including distributed prefill/decode setups - Develop and profile custom GPU kernels - Real-time ASR and TTS: TensorRT compilation, quantization, batching improvements, and a custom TTS serving engine - On-premises deployment and capacity planning: adapting inference to customer hardware, latency requirements, and production workloads 2023–2024 ### brask [Visit brask](https://brask.ai) MLOps Engineer · USA · Remote Built scalable services for speech recognition, text-to-speech, and voice conversion. Worked on multilingual speech processing and speaker diarization, alongside translation and lip-sync pipelines Created a GPU pipeline combining more than 10 models for training-data collection and processing. Synthesized over 150 voices and built a fast fine-tuning pipeline that ran in under one minute per voice Developed a gRPC task balancer that improved GPU worker utilization by 50%, and doubled inference speed for models in translation and lip-sync pipelines. Led a team of four ML and DevOps engineers, taking models from notebooks into production services 2022–2023 ### Mars (food and pet care) [Visit Mars](https://www.mars.com) Machine Learning Engineer · Moscow · On-site Led a team of five ML engineers at Mars’ Russian subsidiary, building a real-time video segmentation service for factory operations. Worked across data collection, automatic labeling, model training, and production deployment Built scalable GPU inference pipelines for data collection and training across more than 10 models. Worked closely with business teams to turn operational requirements into an MVP in four months 2021–2023 ### whalemap [Visit whalemap](https://www.whalemap.io/) Machine Learning Engineer · London · Remote Built ML pipelines for blockchain address tagging and real-time analytics. Reworked ETL processing to improve throughput by 50%, and automated data ingestion from more than 150 API endpoints 2020–2021 ### emovi [Visit emovi on Product Hunt](https://www.producthunt.com/products/emovi#emovi) Data Scientist / Backend Developer · Moscow · On-site Transformer-based recommendation and search, deployed to backend services and mobile apps, including on-device ML inference 2018–2020 ### MegaFon (#1 Russian telecom company) [Visit MegaFon](https://moscow.megafon.ru/) Machine Learning Intern · Moscow · On-site Researched transformer-based classification of customer requests and collected training data to support request handling automation ## Projects: Inference optimization projects ### LLM inference & GPU performance Inference - Serving large models like DeepSeek V4, Kimi K3, and GLM-5.3 across multiple GPUs and nodes: tensor, pipeline, and expert parallelism, disaggregated prefill/decode, and KV-cache management. Tuning compute, memory, and communication for the target hardware and workload - Fixed and optimized custom vLLM and SGLang forks for production workloads - Used Nsight Systems to trace CPU/GPU execution, identify synchronization and memory-transfer bottlenecks, and guide changes to kernels, batching, and data flow - Optimized GPU execution with Triton kernels, fused operations, and hardware-aware memory access, targeting the parts of the model that limited serving performance - Optimized LLM execution across CPU and GPU backends, reducing prefill latency by **3×** and **doubling decoding throughput** without degrading accuracy - Tuned KV-cache allocation, memory pinning, and GPU communication, increasing token throughput by **40%**, reducing p99 latency by **30%**, and raising GPU utilization from approximately **55% to 82%** - Reduced model-worker startup time by **5×, to under one minute**, using NVIDIA Dynamo Snapshot, compilation-cache reuse, warmup improvements, and remote caching of compiled artifacts - Working primarily with NVIDIA Blackwell (RTX PRO 6000 Blackwell, B200, B300) and Hopper (H100, H200) GPUs - Deployed large models on customer on-premises GPU clusters for development with sensitive data, including integration with Claude Code and Codex vLLM · SGLang · NVIDIA TensorRT-LLM · NVIDIA Dynamo · Nsight Systems · Triton (GPU kernels) · FlashInfer ### LLM inference costs Cost optimization - Combined speculative decoding, KV-cache optimization, and dynamic batching to increase sustained throughput by **46%** and reduce infrastructure costs by **35%** - Implemented KEDA-based autoscaling driven by throughput, latency, and queue metrics, reducing GPU costs by **more than 28%** - Optimizing self-hosted LLM serving to generate more tokens per second from each GPU and reduce cost per token. In one case, benchmark-based estimates showed **approximately 87% lower GPU inference costs** than a hosted API under sustained load ### Real-time speech systems Speech - Optimized streaming ASR and TTS by rebuilding pipeline artifacts, compiling models with TensorRT, applying quantization, and improving batching and request scheduling for the target hardware and workload - Co-developed a custom TTS inference engine as an alternative to NVIDIA Triton Inference Server, with scheduling designed around first-audio latency, continuous audio delivery, and efficient batching across active sessions - ASR: optimized NVIDIA Nemotron and Conformer inference, reducing per-chunk latency **from 400 ms to 50 ms** in a real-time setup and scaling concurrent streams **from 30 to 250** on a single NVIDIA L40S GPU - TTS: accelerated VITS and MOSS inference, reducing time to first byte (TTFB) **from 800 ms to 100 ms** and improving batching to scale concurrent streams **from 30 to 150** on a single NVIDIA L40S GPU - Worked closely with the model-training team on quantization and inference optimization, maintaining ASR and TTS quality ASR · TTS · NVIDIA Triton · TensorRT · gRPC ### Speech inference costs Cost optimization - Reduced speech recognition and synthesis costs through batching, request scheduling, and better GPU utilization. In selected sustained-load scenarios, estimated ASR and TTS inference costs were **6–12× lower than hosted API pricing**. Savings depend on the provider and can be higher in some cases ### Generative audio & video Applied ML - Production pipelines for voice conversion, translation, and lip-sync. Connecting models into services and making better use of GPU workers - Improved the reliability of production audio and video generation services through monitoring and alerting PyTorch · Diffusion · GPU pipelines ## Blog LLM · Infrastructure ### [Nearly 6× faster with a compile cache. B200 workers in 45 seconds.](https://pavelfedorov.io/blog/vllm-compile-cache/) An S3 compile cache made H200 startup nearly 6× faster. Dynamo Snapshot brought B200 workers online in about 45 seconds. **~45 s**B200 startup · Dynamo Snapshot Speech · TTS ### [The TTS bottleneck before the first sound](https://pavelfedorov.io/blog/tts-first-audio/) Reworking VITS from a model invocation into a compiled, batched pipeline for interactive voice calls. **120 ms**p50 first audio at 150 streams LLM · Voice serving ### [Scaling Qwen voice serving under a one-second budget](https://pavelfedorov.io/blog/qwen-voice-capacity/) How I tuned graph coverage, speculative decoding, and precision for voice-agent traffic with a one-second LLM budget. **500 ms**E2E p95 · 40 concurrent user streams [All articles](https://pavelfedorov.io/blog/) ## About: A little background I’m an ML systems engineer working at the intersection of machine learning and infrastructure. My path started with recommendations and search, moved through computer vision and generative media, and led to what I spend most of my time on today: making LLM and speech inference fast, dependable, and useful in production. I enjoy understanding how GPU chips and models work under the hood, and using that knowledge to make inference faster. I like working across the whole system: finding the bottleneck, making the change, and measuring what actually improved ### Education Higher School of Economics Moscow · 2017–2021 BSc in Computer Science (backed by Yandex) Machine Learning and Applications - Top 3 in Computer Science across post-Soviet countries (“Stanford of Russia”, 2017) - Paid my full tuition from my third year onward ### Teaching Higher School of Economics Moscow · 2022–2024 Taught deep learning seminars and developed course material on graph neural networks ## Working on something interesting? Let’s talk models, systems, and what comes next [Say hello](mailto:dm@pavelfedorov.io)