Sr. Performance Engineer - LLM Inference Optimization (CarbonForge/Remote)

Senior Performance Engineer - LLM Inference Optimization

Company Name: CarbonForge
Montreal (remote-friendly)
Full-time
Founding team

About CarbonForge

AI adoption is accelerating, but the infrastructure required to run AI models remains expensive, energy-intensive, and increasingly difficult to scale. Organizations are spending millions on GPU capacity while struggling to maximize the performance of the hardware they already have.

CarbonForge solves these challenges through AI Inference Optimization.

Founded by AI researchers and entrepreneurs from Mila, CarbonForge optimizes the full stack from hardware and kernels to serving engines and APIs to maximize tokens-per-watt and tokens-per-dollar. Leveraging frameworks like vLLM, TensorRT-LLM, and SGLang, we deliver low-latency, high-throughput inference that meets the most demanding power and cost constraints.

Join our team to push the state of the art in AI inference performance and work end‑to‑end across the system.

The role

Are you driven by the toughest challenges in AI? We’re seeking a Senior Performance Engineer to join our founding team and redefine LLM inference. In this role, you’ll architect high-impact solutions across the entire stack-from optimizing kernels, shaping the serving engine, improving the scheduling/orchestration and advancing open-source inference engines. If you’re ready to push the boundaries of AI performance, we want to hear from you.

What you’ll do

• Design, build, and optimize high‑performance inference systems for LLMs and multimodal models, from single‑GPU experiments to multi‑GPU, multi‑node deployments.

• Develop and tune custom kernels and runtime paths (CUDA / TensorRT / Triton / compiler‑based optimizations) to push GPU utilization and memory efficiency toward theoretical limits.

• Improve and extend serving engine (e.g., vLLM, TensorRT‑LLM, SGLang, or custom runtimes) including batching, KV‑cache management, scheduling, and routing policies for heterogeneous workloads (sync, async, batch, streaming).

• Design and operate distributed inference systems: autoscaling policies, load balancing, capacity management, and failure handling under strict latency and reliability SLOs.

• Instrument, profile, and analyze the entire inference lifecycle (from request to GPU to network and storage) to identify bottlenecks and drive systematic performance improvements.

• Collaborate with hardware partners and internal infra teams to co‑design software/hardware strategies that improve cost and energy efficiency (e.g., hardware SKUs, topology‑aware placement, mixed precision, quantization).

• Explore and implement advanced inference techniques such as speculative decoding, paged KV‑cache, tensor/parallellism, LoRA/multi‑LoRA serving, mixture‑of‑experts, and offloading for memory‑constrained environments.

• Work closely with research and product teams to translate abstract model and product requirements into concrete constraints and system designs, balancing latency, throughput, quality, and cost.

• Contribute to internal tooling and, when appropriate, open‑source projects around inference engines, benchmarking suites, and observability.

What we’re looking for

• Deep experience building or operating large‑scale inference or training systems in production (LLMs preferred, but other large‑scale deep learning systems also relevant).

• Understand GPU architectures and performance characteristics (compute vs memory vs bandwidth, kernel launch overheads…).

• Comfortable with the modern inference ecosystem: vLLM, TensorRT / TensorRT‑LLM, Triton, SGLang, or similar.

• Strong understanding in systems and distributed‑systems fundamentals (data structures, concurrency, scheduling, backpressure, observability, fault tolerance).

• Enjoy debugging performance issues across layers: from kernel traces and CUDA profiles to queueing behavior in the serving layer and autoscaler dynamics.

• Comfortable in a research‑adjacent environment: reading papers, prototyping new algorithms, exploring emerging hardware, and turning prototypes into robust systems.

• Can own problems end‑to‑end, from initial exploration and measurement to deployed systems and long‑term operational health.
  
  

Nice‑to‑have experience

• Contributions to inference frameworks or libraries (e.g., vLLM, SGLang, TensorRT‑LLM, FasterTransformer, custom CUDA/Triton kernels).

• Experience with K8s‑based infrastructure (e.g., operators, custom controllers, autoscalers) and modern observability stacks.

• Work on cost or energy‑aware scheduling, capacity planning, or optimization for large fleets of accelerators.

• Experience with post‑training pipelines (RL, preference optimization, evals) and their interaction with inference infrastructure.

On credentials

No PhD required. We care about what you’ve shipped and what you can prove on a GPU. A pull request closes the loop faster than a CV.

Why join CarbonForge

• Mission that compounds. Power is the binding constraint for AI deployment this decade. You’ll build the layer that makes every joule count — and every token more useful.

• World-class science bench. Work alongside two Canada CIFAR AI Chairs and core Mila researchers: Pierre-Luc Bacon (UdeM, scientific co-founder) and Christophe Dubach (McGill, scientific advisor). Full support of Mila, Canada’s leading AI research institute, with access to its talent pipeline and research bench.

• Real hardware from day one. H100, H200, AMD MI-series, and design-partner engagements already in motion.

• Resources to match the ambition. Seed-funded by strong operators and institutional investors. We have the resources to hire the best, give them the compute they need, and run the long experiments that matter. Founding-engineer equity, direct ownership, no bureaucracy - your code reaches production in weeks, not quarters.

• Remote-friendly. Montreal preferred for regular in-person work with the founding team.

A note on fit

If not every bullet above describes you perfectly, apply anyway. We’re hiring for trajectory and proof of work, not a checklist. What matters is whether you can ship the outcomes we’ve described and push us to be better at the work.

CarbonForge is committed to building a team where the best engineers want to work, from any background. We welcome applicants from all origins, identities, and paths into this field. Equal opportunity employer.