The SpyGlass Group, LLC

Personalized Technology Expense Management (TEM) Audits, Surprising Savings

Lead Data Engineer – Databricks, Microsoft Fabric

Data EngineerData EngineerContractRemoteTeam 51-200Since 20 years agoH1B No SponsorCompany SiteLinkedIn

Location

Rhode Island

Posted

4 days ago

Salary

Not specified

Bachelor Degree7 yrs expEnglishApacheAzureCloudETLPy SparkPythonSpark

Job Description

• Design and implement modern data platforms using Databricks and Microsoft Fabric • Build and maintain scalable data pipelines and ingestion frameworks • Develop and optimize Spark and PySpark workloads within Databricks • Implement lakehouse and data warehouse architectures • Design and manage data models supporting analytics and reporting • Collaborate with data scientists and analytics teams to support AI and advanced analytics initiatives • Implement data quality, governance, and security best practices • Provide technical leadership and mentorship to data engineering team members • Optimize performance, reliability, and scalability of data platforms

Job Requirements

  • 7+ years of experience in data engineering or data platform development
  • Strong hands-on experience with Databricks and Apache Spark
  • Experience implementing Microsoft Fabric data solutions
  • Proficiency in Python and PySpark
  • Experience working with Azure data services and cloud-based data platforms
  • Experience designing modern lakehouse or data warehouse architectures
  • Strong understanding of data pipeline design, ETL/ELT processes, and data modeling
  • Experience supporting AI or machine learning workloads
  • Experience working in enterprise data environments
  • Familiarity with CI/CD and data platform automation

Related Categories

Related Job Pages

More Data Engineer Jobs

Data Engineer4 days ago
Full TimeRemoteTeam 201-500Since 1991H1B No Sponsor

Senior Data Engineer designing ETL pipelines for U.S. Federal Government support

ETLInformaticaOraclePythonSQL
United States
$115K - $125K / year

Senior Data Engineer

Miratech

Helping Visionaries Change the World

Data Engineer4 days ago
Full TimeRemoteTeam 501-1,000Since 1989H1B No Sponsor

The Senior Data Engineer will perform deep analytical investigations of IVR call data and routing behavior, writing advanced SQL queries to analyze call flows and interaction metrics. Responsibilities also include supporting troubleshooting of IVR issues through backend data analysis and producing insights using Tableau dashboards.

SQLAmazon Aurora MySQLAmazon RedshiftETLTableau
United States

Software Engineer, Data Infrastructure & Acquisition - Champaign-Urbana, USA

Speechify

Get your reading done faster, easier, and on the go. Listen to any book, document, or website with Speechify.

Data Engineer4 days ago
Full TimeRemoteTeam 51-200H1B Sponsor

This role is responsible for all aspects of data collection to support model training operations, focusing on building high-quality, petabyte-scale datasets cost-effectively. Key duties include finding new audio data sources, operating and extending cloud ingestion infrastructure (GCP/Terraform), and collaborating with scientists on data roadmaps.

PythonBashLinuxDockerTerraformGCPGoogle Cloud PlatformData InfrastructureWeb CrawlingData Processing
United States
$140K - $200K / year

Software Engineer, Data Infrastructure & Acquisition - Philadelphia, USA

Speechify

Get your reading done faster, easier, and on the go. Listen to any book, document, or website with Speechify.

Data Engineer4 days ago
Full TimeRemoteTeam 51-200H1B Sponsor

This role is responsible for all aspects of data collection to support model training operations, requiring the engineer to find new audio data sources and integrate them into the ingestion pipeline. The engineer will also operate and extend the cloud infrastructure for this pipeline, collaborating with scientists to improve data quality, scale, and cost-efficiency.

PythonbashLinuxDockerTerraformGoogle Cloud PlatformData IngestionWeb CrawlingLarge-Scale Data Processing
United States
$140K - $200K / year