Lead Databricks Administrator

AdministrationAdministrationFull TimeRemoteTeam 10,001+Since 1890H1B No SponsorCompany Site LinkedIn

Location

Virginia

Posted

128 days ago

Salary

Not specified

Bachelor Degree8 yrs expEnglishAWSAzureCloudETLMS SQL ServerPythonSparkSQLTerraformUnityVault

Job Description

• Lead administration, configuration, and support of Databricks, Databricks Lakehouse, Microsoft SQL Server, and Cosmos DB environments. • Ensure data integrity, performance, availability, and security across multiple database technologies. • Perform advanced SQL database administration, including backup/recovery, performance tuning, replication, clustering, and maintenance automation. • Configure and tune database parameters; define data repository requirements, schemas, and governance standards. • Provide actionable platform performance insights, usage analytics, and capacity planning reports to leadership. • Develop and maintain runbooks, SOPs, and troubleshooting guides for database and platform operations. • Design and implement automation, monitoring, and infrastructure-as-code solutions (Databricks CLI, REST APIs, Terraform). • Act as a senior technical escalation point, collaborating with engineering teams to resolve complex or high-impact issues. • Identify and integrate platform enhancements that improve reliability, observability, and scalability. • Mentor team members to build competency in Databricks, Databricks Lakehouse, Microsoft SQL Server, Cosmos DB, and AI-focused technologies. • Participate in both weekday and weekend on-call rotations to ensure 24/7 operational coverage.

Job Requirements

Bachelor's degree in computer science, Information Systems, or a related field (Required)
Databricks Administration (Required) 5+ years of hands-on Databricks experience in an Azure cloud environment.
Expertise with Unity Catalog, external locations, access control, and data governance.
Experience managing secrets and credentials using enterprise tools (Azure Key Vault, AWS Secrets Manager).
Proficiency with automation and infrastructure-as-code tools (Terraform, CLI, REST APIs).
Strong experience with Spark optimization, Delta Lake management, and schema design for analytics.
Familiarity with Databricks workflows, job orchestration, monitoring, and recovery strategies.
SQL Database Administration (Required) 7+ years of experience supporting and administering Microsoft SQL Server databases.
Strong skills in query tuning, indexing, backup/restore strategies, replication, and Always On availability groups.
Knowledge of database security, encryption, and compliance best practices.
Experience supporting ETL/ELT processes, metadata management, and data quality assurance.
Additional Technical Expertise (Highly Desired) Databricks Lakehouse: configuration, tuning, monitoring, replication, and backup/recovery management.
Cosmos DB: deployment, scaling, sharding, and performance optimization in enterprise environments.
AI/ML Integration: familiarity with AI tools and frameworks (Databricks ML flow, Azure AI services) and how they integrate into enterprise data pipelines.
Scripting and Automation: experience with Python, PowerShell, or Bash for platform automation.
Cross-platform data integration: experience connecting and managing multi-database architectures and hybrid environments.

Benefits

Medical, Dental, Vision plans
Adoption, Fertility and Surrogacy Reimbursement up to $10,000
Paid Time Off and Sick Leave
Paid Parental & Family Caregiver Leave
Emergency Backup Care
Long-Term, Short-Term Disability, and Critical Illness plans
Life Insurance
401k/403B with Employer Match
Tuition Assistance – $5,250/year and discounted educational opportunities through Guild Education
Student Debt Pay Down – $10,000
Reimbursement for certifications and free access to complete CEUs and professional development
Pet Insurance
Legal Resources Plan
Colleagues have the opportunity to earn an annual discretionary bonus if established system and employee eligibility criteria is met.