94 model serving engineer jobs at 82 companies in United States
Model Serving Engineer
United States
$74k-$98k/yr RemoteFull Time
6+ YOEBachelor's or Master's in Computer Science or related field; 6+ years distributed systems or ML platform engineering; Python and Go/Rust/C++; vLLM or TensorRT-LLM; Kubernetes; observability; GPU architecture; U.S. work authorization without new H-1B sponsorship.
Python, Go, Rust, C++, vLLM, TensorRT-LLM, Kubernetes
Software Engineer, Model Serving System
Mountain View or Redmond
$120k-$235k/yr OnsiteFull Time
5+ YOEBachelor's in Computer Science or related field and 5+ years engineering experience with Python, C++, or similar; experience with LLM inference/serving systems (vLLM, SGLang, TensorRT-LLM); strong systems and debugging skills; distributed serving concepts.
Python, C++, vLLM, SGLang, TensorRT-LLM, Kubernetes, Grafana, Prometheus, Kusto
Inference Infrastructure Engineer, Serving
Palo Alto, California, United States
$275k-$475k/yr OnsiteFull Time
3+ YOE3+ years building low-latency, high-throughput inference serving systems; knowledge of quantization, batching, speculative decoding, KV cache; experience with vLLM/TensorRT-LLM/Triton/SGLang; multi-GPU model parallelism; C++/CUDA/Python; autoscaling and GPU cost optimization.
vLLM, TensorRT-LLM, Triton, SGLang, C++, CUDA, Python
Solutions Engineer (Model Studio)
San Jose or Austin
HybridFull Time
5+ YOE5-8 years in customer-facing technical role; solid knowledge of LLM inference and model serving; exceptional presentation and communication skills; Bachelor's in Computer Science or Engineering; willingness to travel.
Hugging Face, vLLM
Staff/ Principal Software Engineer (L6/L7): AI Platform Model Serving
United States
$890k-$1690k/yr RemoteFull Time
12+ YOEBS/MS/PhD in Computer Science or related field; 12+ years experience in AI/ML infrastructure and distributed systems; expertise in model serving, GPU optimization, Kubernetes, and public cloud (AWS, Azure, GCP).
PyTorch, TensorFlow, JAX, Hugging Face Transformers, vLLM, SGLang, TensorRT-LLM, TGI, DeepSpeed-Inference, NVIDIA Dynamo, CUDA, NCCL, NVIDIA Nsight, Kubernetes, AWS, Azure, GCP
Software Engineer, Model Lifecycle and Signals
New York or San Francisco +1 more
OnsiteFull Time
Exceptional engineering track record; experience building evaluation, measurement, or model serving systems; fluency with data; strong taste and curiosity about model behavior.
AIML - Distinguished Engineer, Foundation Model
Cupertino, California, United States
OnsiteFull Time
15+ YOEMS or PhD in Computer Science, Machine Learning, or related field; 15+ years building large-scale ML or distributed systems; deep expertise in foundation model inference engines; proficiency with JAX, PyTorch, and inference/serving stacks; experience leading engineers and partnering with ML researchers.
JAX, PyTorch, Kubernetes, Docker, AWS, GCP, Azure
Senior Software Engineer, Data & Model
Fremont, California, United States
$150k-$300k/yr OnsiteFull Time
5+ YOERequires 5+ years in software, data or ML infrastructure, or distributed systems; experience with data pipelines, distributed computing, model serving, cloud infrastructure, containers, orchestration, and ML workflows.
AI, Physical AI, cloud infrastructure, storage, containers, orchestration
Software Engineer, Model Runtime
San Francisco, California, United States
$266k-$445k/yr HybridFull Time
Strong systems programming in C++, Rust, or Python; experience with runtimes, distributed systems, compilers, kernels, or serving infrastructure; knowledge of LLM inference and hardware-software performance optimization.
C++, Rust, Python, vLLM, SGLang
Senior Software Engineer - Model Platform
United States or Canada
RemoteFull Time
5+ YOE5+ years software engineering experience with ML-focused solutions; experience with large-scale distributed systems on AWS, GCP, or Azure; real-time data pipelines; collaboration with cross-functional teams; machine learning workflows and feature serving at 50K+ QPS.
AWS, GCP, Azure
Staff Software Engineer- Foundation Model Inference
San Francisco or Mountain View
$190k-$265k/yr OnsiteFull Time
8+ YOE8+ years backend or infrastructure engineering experience; distributed systems, scalable APIs, real-time serving or ML/GPU orchestration experience; familiarity with service-oriented architecture, deployment pipelines, and observability.
OpenAI, Anthropic, Gemini, Qwen, GPT-OSS, Llama, SageMaker, Vertex AI, Azure ML, MLflow, PyTorch, Ray, vLLM, SGLang, Apache Spark, Delta Lake
MLOps Engineer
Europe or Israel or United States
RemoteFull Time
5+ YOE5+ years MLOps/DevOps experience, degree in CS/Engineering or equivalent, experience with model serving, Kubernetes, cloud providers, IaC, Python/Bash/Go, and observability tools.
TorchServe, TensorFlow, Triton, MLflow, WandB, PyTorch, TensorFlow Serving, KServe, Kubernetes, AWS, GCP, Azure, Terraform, Helm, GitOps, Python, Bash, Go, Prometheus, Grafana, Datadog, OpenTelemetry, FastAPI, Databricks, Snowflake
Senior Machine Learning Platform Engineer — Model Hosting & MLOps
Spring or Tlaquepaque
$147k-$231k/yr OnsiteFull Time
Hands-on experience hosting LLM inference on GPU infrastructure, building model-serving platforms, strong Python skills, infrastructure as code (Terraform), Kubernetes, and MLOps pipelines across AWS and Azure.
Python, Terraform, Kubernetes, AWS, Azure, SageMaker, EKS, Azure Machine Learning, AKS
Staff, MLOps Engineer
New York City or San Francisco or United States
$220k-$280k/yr RemoteFull Time
4+ YOEMinimum 4+ years MLOps or ML/platform engineering; expertise with low-latency model serving, Python and PyTorch; cloud (AWS/GCP/Azure), Docker, Kubernetes, MLflow; strong distributed systems and pipeline experience.
Python, PyTorch, AWS, GCP, Azure, Docker, Kubernetes, MLflow, Rust, vLLM, Triton
Distinguished Engineer, Scaled Out Inferencing
Santa Clara or California
$320k-$489k/yr HybridFull Time
16+ YOE7+ MgmtRequires 16+ years in technical roles, 7–10+ years of leadership, and BS/MS or equivalent experience in systems or software engineering. Expertise in AI infrastructure, distributed systems, GPU architecture, CUDA, kernels, and cloud-native model serving.
CUDA, Dynamo, TensorRT-LLM, vLLM, SGLang, Linux, Kubernetes, Ray
Senior AI Infrastructure Engineer
Austin or Reston or United States
HybridFull Time
5+ YOE5–8 years building distributed systems or cloud/platform services; production Kubernetes and ML infra experience; strong software engineering in Python and Go/Rust/C++; familiarity with GPU inference, model serving frameworks, and cloud platforms.
Kubernetes, Helm, Argo CD, Docker, Prometheus, Grafana, OpenTelemetry, Python, Go, Rust, C++, vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Ray Serve, AWS, Azure, Oracle Cloud Infrastructure, Google Cloud Platform
Forward Deployed Engineer - AI Inference
San Francisco, California, United States
HybridFull Time
3+ YOE3+ years in cloud infrastructure, DevOps, or reliability engineering; Bachelor's or Master's in CS/CE/EE or equivalent; proficiency with Kubernetes, Docker, Terraform, Helm; experience with GPU computing and AI model serving; strong distributed systems and networking knowledge.
Kubernetes, Docker, Terraform, Helm, AWS, GCP, OCI, Triton, vLLM, TensorRT, DeepSpeed-Inference, Prometheus, Grafana, Loki, ELK, OTEL
Applications Engineer
Charlotte or Greensboro or Raleigh
FieldFull Time
Requires strong work ethic, team mindset, coachability, problem-solving, decision-making, relationship building, and ability to serve as a role model. Must handle medium work, lift up to 50 lbs, travel daily to customer sites, and use standard office equipment.
CRM, laptop, smartphone
Machine Learning Engineer (Staff)
San Francisco or Menlo Park
$220k-$270k/yr HybridFull Time
8+ YOE8+ years building production ML systems and infrastructure; experience with training/serving pipelines, feature pipelines, monitoring, deployment, cloud, containers, CI/CD, and model governance.
CI/CD, APIs, containers, feature stores, MLOps, LLM
Anti-Fraud Engineer (NYC / MIA)
New York City or Miami
$185k-$220k/yr HybridFull Time
5+ YOE5+ years building production systems, including 3+ years in fraud, risk, or trust and safety; production rules or model-serving ownership; programming, SQL, fraud metrics, and AI tool expertise; bachelor's degree or equivalent.
TypeScript, SQL, AI, KYC, KYB, 3DS, SCA, BSA, OFAC, PSD2, MiCA, DORA, SOC 2
Explore Jobs