Talent AgentClassic Search

892 spark engineer jobs at 220 companies in San Francisco, CA

3w
Save
Mark Applied
Hide
Senior Project Engineer - Spark Deployments
San Francisco, California, United States
$145k-$200k/yr OnsiteFull Time
Crusoe
Crusoe: Vertically integrated AI infrastructure and energy.
5+ YOEBachelor's degree in mechanical engineering or related field, 5+ years of mechanical engineering experience, industrial or mission-critical experience, HVAC and chilled-water expertise, and CAD proficiency.
CAD, Revit, Navisworks, Bluebeam, BIM, ICC, ASME B31.x, ASHRAE, NFPA, HVAC
Job PostingView all
2w
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Sunnyvale or Seattle or New York City
$131k-$192k/yr HybridFull Time
DoorDash
DoorDashNYSE: DASH: Technology enabling local on-demand delivery and commerce.
2+ YOEB.S., M.S., or PhD in Computer Science or equivalent; 2+ years operating production distributed systems; experience with Apache Spark at scale on EMR, Databricks, or in-house; Kubernetes operations; familiarity with schedulers and observability; Python, Go, Scala, or Java; SQL fluency.
Apache Spark, Kubernetes, Amazon EMR, Databricks, YuniKorn, Volcano, Kueue, Prometheus, OpenTelemetry, Python, Go, Scala, Java, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Machine Learning Engineer, Accelerated Apache Spark
Santa Clara, California, United States
$224k-$357k/yr OnsiteFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Computing platform for AI and accelerated graphics.
12+ YOE3+ MgmtBS, MS, or PhD in machine learning, data science, computer science, or related field; 12+ years ML/DL experience, 3+ years technical leadership, and 2+ years Apache Spark experience.
Apache Spark, Python, NumPy, pandas, scikit-learn, SciPy, PyTorch, TensorFlow, XGBoost, NVIDIA GPUs, CUDA, Scala, Java, C++
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer - Data Platform (Spark/Kafka/Flink/Scala/Java) - Onsite Hybrid
Cupertino, California, United States
$96k-$145k/yr HybridFull Time
NTT DATA
NTT DATATokyo Stock Exchange: 9613: Global digital business and IT services provider.
8+ YOERequires 8+ years of software or data engineering experience, strong Spark, Kafka, Flink, Java or Scala, SQL, Kubernetes, Azure, database, CI/CD, and distributed systems expertise, plus a bachelor's degree or equivalent experience.
Apache Spark, Apache Flink, Apache Kafka, Airflow, Kubernetes, Microsoft Azure, Java, Scala, SQL, Cassandra, YugabyteDB, PostgreSQL, GitLab, GitHub
Job PostingView all
2mo
Save
Mark Applied
Hide
Software Engineer - Infrastructure (Senior+)
San Francisco or New York
$100k-$300k/yr OnsiteFull Time
Cogent
Cogent: AI cybersecurity providing autonomous agents for enterprise vulnerability management.
5+ YOE5+ years infrastructure engineering experience, hands-on AWS, Clickhouse, Postgres, Spark; startup experience; security and on-prem deployment experience; able to drive large cross-stakeholder projects.
AWS, Clickhouse, Clickhouse Cloud, Postgres, Spark, GCP, Azure, Envoy, Istio, Hashicorp Vault, MicroVM (Runloop), SPIFFE, SPIRE
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Platform Engineer
Austin or San Francisco or New York City or Seattle or Los Angeles or Chicago or Gurugram or Bengaluru
$170k-$230k/yr RemoteFull Time
SentiLink
SentiLink: Your fraud and risk partner
5+ YOE5+ years engineering experience; strong Python or Golang skills; building ETL/ELT pipelines at scale with Spark/Hadoop/Kafka; cloud (AWS/Azure/GCP) and database expertise; containerization and IaC experience.
Python, Golang, PostgreSQL (RDS), OpenSearch, Redshift, EMR, Spark, Docker, Lambda, AWS, EKS, SQS, SNS, S3, Kubernetes, Hadoop, Kafka, Microsoft Azure, GCP
Job PostingView all
3mo
Save
Mark Applied
Hide
Principal Scientist - Data Pipeline Engineer
San Jose or Seattle or San Francisco
$206k-$388k/yr OnsiteFull Time
Adobe
AdobeNASDAQ: ADBE: Empowering everyone to create through innovative digital experiences.
10+ YOE10+ years in data engineering, ML infrastructure, or distributed systems; expertise in Ray or Spark, Python, a systems language, databases, GPU inference, data curation, and large-scale infrastructure. Degree required.
Ray, Python, Spark, C++, Rust, Go, Java, VLMs, LLMs, Adobe Acrobat Studio, Adobe Express, Adobe Firefly, Creative Cloud, Adobe Experience Platform, Adobe Experience Manager, GenStudio
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Data Engineer
Denver or San Francisco
$190k-$264k/yr HybridFull Time
Checkr
Checkr: AI verification platform providing background, identity, employment, mortgage, tenant, and trust checks for businesses and individuals.
10+ YOE10+ years building scalable data platforms; expert PySpark, Python, SQL; experience with Kafka, Spark, Iceberg, data lakes, AWS; strong data modeling and security awareness.
PySpark, Python, SQL, MongoDB, Kafka, Spark, Iceberg, EKS, EMR, Serverless, Glue, Athena, S3, Databricks, Snowflake
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Engineer
Los Angeles or San Francisco
$170k-$200k/yr HybridFull Time
Tatari
Tatari: Private ad-tech helping brands and agencies buy, measure, and optimize ads across linear and streaming TV.
5+ YOE5+ years building and operating production ETL pipelines, strong Python and SQL skills, Spark/PySpark, Databricks/Delta Lake, Airflow, data modeling and data quality expertise.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
Job PostingView all
3mo
Save
Mark Applied
Hide
Site Reliability Engineer, Compute Platform
San Jose, California, United States
$156k-$388k/yr OnsiteFull Time
TikTok
TikTok: Short-form mobile video and social media platform.
Bachelor's in CS/Engineering, strong Linux, networking, databases, Kubernetes, SRE/DevOps toolset knowledge, experience with ClickHouse/Spark/Presto/Doris/Hadoop, coding in Python/Shell/Java/Go, strong problem-solving and communication.
ClickHouse, Spark, Presto, Doris, Hadoop, Kubernetes, Python, Shell, Java, Go
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Software Engineer — Distributed Compute / Spark Systems
Mountain View or San Francisco
$160k-$240k/yr OnsiteFull Time
Granica
Granica: AI research and products building data infrastructure and models for enterprises.
5+ YOEDeep distributed systems experience; production experience with distributed compute or query systems; performance optimization expertise; systems programming skills in Scala, Java, Go, Rust, or C++.
Apache Spark, Spark SQL, Trino, Presto, Flink, Databricks, Snowflake, Apache Iceberg, Delta Lake, Apache Hudi, Parquet, ORC, Amazon S3, GCS, ADLS, Scala, Java, Go, Rust, C++, EMR, Glue, Hive, Catalyst, Adaptive Query Execution, Velox, DuckDB, DataFusion
Job PostingView all
3mo
Save
Mark Applied
Hide
Site Reliability Engineer, Compute Platform
San Jose, California, United States
OnsiteFull Time
ByteDance
ByteDance: Global technology specializing in AI-powered content platforms.
Experience with Linux, networking, databases, Kubernetes, ClickHouse/Hadoop/Doris/Spark/Presto, scripting or programming (Python, Shell, Java, Go), incident management, and capacity planning.
ClickHouse, Spark, Presto, Doris, Hadoop, Kubernetes, Linux, Python, Shell, Java, Go
Job PostingView all
2mo
Save
Mark Applied
Hide
Forward Deployed Engineer
Berlin or Munich or San Francisco
FieldFull Time
Databricks
Databricks: Data and AI software providing a unified platform.
6+ YOERequires 6+ years in data or software engineering, Python, Scala, JavaScript/TypeScript, cloud expertise, Apache Spark, CI/CD, MLOps, production architecture, project delivery, enterprise client experience, and Databricks certification.
Python, Scala, JavaScript, TypeScript, AWS, Azure, GCP, Apache Spark, CI/CD, MLOps, ML/AI models, AI APIs, Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, Unity Catalog
Job PostingView all
2w
Save
Mark Applied
Hide
Data Engineer
San Francisco, California, United States +1 more
$143k-$187k/yr HybridFull Time
Atlassian
AtlassianNASDAQ: TEAM: Unleashing the potential of every team.
4+ YOEBS/BA in Computer Science, Engineering, Information Management or related field; 4+ years data engineering experience; strong Python or Java; SQL; data modeling; Spark/SparkSQL with Airflow; Databricks and Tableau.
Python, Java, SQL, Spark, SparkSQL, Airflow, Databricks, Tableau
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Data Engineer
San Francisco, California, United States
$200k-$260k/yr HybridFull Time
Metriport
Metriport: Open-source healthcare data infrastructure providing APIs that help digital health organizations access and exchange patient data.
8+ YOE8+ years building and operating large-scale data platforms; experience with ingestion, storage, processing, warehousing, streaming, and leadership; strong software engineering and mentorship skills.
Spark, Parquet, Iceberg, Delta, S3, Snowflake, BigQuery, Redshift, dbt, Airflow, Dagster, Kafka, Kinesis, TypeScript, Python, Node.js, AWS, ECS, Lambda, SQS, SNS, Batch, CDK, PostgreSQL, Aurora, DynamoDB, FHIR, Athena, SageMaker, FHIR, HIE, IHE, EHR/EMR, NPI, TEFCA, ADT, HL7, HEDIS, RAF, SNOMED, LOINC, ICD-10
Job PostingView all
1w
Save
Mark Applied
Hide
Senior Data Engineer (Data Platform)
El Segundo or Austin or Washington or San Francisco
$80k-$210k/yr OnsiteFull Time
Gallatin AI
Gallatin AI: Defense technology providing AI-powered logistics decision-support software to military and allied defense organizations.
8+ YOE8+ years data engineering experience; production experience with open table formats (Iceberg/Delta/Hudi), distributed query engines (Trino/Presto/Spark SQL), streaming/batch pipelines (Kafka/NiFi/Debezium/Spark), Python, SQL, Kubernetes, Docker, AWS data services; ability to obtain security clearance.
Apache Iceberg, Delta, Hudi, Trino, Presto, Spark SQL, Kafka, NiFi, Debezium, Spark, Python, SQL, Airflow, Dagster, Argo Workflows, TypeScript, Vue, React, Kubernetes, Docker, AWS S3, AWS Glue, AWS Aurora, Azure, Nessie, Polaris, OpenLineage, DataHub, lakeFS, Palantir Foundry
Job PostingView all

Explore Jobs

Expand Your Job Search