Headquartered in Pleasanton, California, Veeva is a leading provider of cloud-based software and services for the life sciences industry. As an employer, Veeva

Senior Data Engineer

Data EngineerData EngineerOther Remote Senior Company Site

Location

California

Posted

2 days ago

Salary

$115K - $175K / year

Seniority

Senior

Bachelor Degree5 yrs expEnglishAWS Cloud ETL Kafka Kubernetes Spark

Job Description

• Design and implement a scalable Lakehouse environment. Own the lifecycle of efficient storage, including schema evolution, partition transformation, and snapshot management. • Build and manage real-time and batch data ingestion pipelines using Kafka and Spark • Deploy, manage, and scale data workloads (such as Spark executors) using Kubernetes (EKS). • Develop complex ETL workflows. Optimize jobs for large-scale data processing focusing on memory management and shuffle optimization. • Manage and monitor end-to-end data lifecycles using orchestration tools • Write and tune high-performance queries against the Lakehouse. Implement optimization strategies, compaction, and data skipping to reduce latency and cloud costs

Job Requirements

5+ years in Data Engineering, with at least 2 years focused on Lakehouse or modern Data Lake environments
Hands-on experience with AWS ecosystems, async/queue processing in a production environment and working with a Lakehouse
Expert in workflow management, data transformation, storage and exchange at scale.
Proven experience with events driven job processing. Mastery in data ingestion, processing large-scale data and performance tuning.
Experience with Kubernetes and infrastructure as code

Benefits

Medical, dental, vision, and basic life insurance
PTO and company-paid holidays
Retirement programs
1% charitable giving program

Related Categories

Data Engineer

Related Job Pages

Data Engineer Jobs in California More US Remote Jobs

More Data Engineer Jobs

Microsoft Fabric Specialist / Data Warehouse Engineer

Apex Companies

Apex Companies is one of the fastest growing engineering and environmental consulting firms in the US, recognized by the Zweig Group as one of the fastest growing firms in the AEC industry. We take pride in providing our clients with exceptional service and have earned numerous awards for project excellence. We continue to grow, and we want you to grow with us.

Data Engineer2 days ago

Other RemoteTeam 1,001-5,000

The role focuses on designing, implementing, and optimizing solutions within Microsoft Fabric for data integration, analytics, and governance across the enterprise. Responsibilities include developing lakehouses, dataflows, and pipelines while collaborating with stakeholders to build robust data models.

Microsoft Fabric Data Factory Synapse Power BI SQL DAX Azure Data Services data lakehouse architecture OneLake ETL

View details: Microsoft Fabric Specialist / Data Warehouse Engineer

United States

$80K - $120K / year

Apply

Senior Technical Architect

o9 Solutions, Inc.

Data Engineer2 days ago

Other RemoteTeam 1,001-5,000

The Senior Technical Architect will work with Extract, Transform, and Load (ETL) processes, including Informatica, to resolve batch automation disruptions and ensure customer Service Level Agreements (SLAs) are met with accurate results. Responsibilities also include validating, synthesizing, and transforming customer data for planning, visibility, and analytics using various tools like SQL, Hadoop, and PySpark.