DataPelago
Accelerate Any Engine on Any Data on Any Hardware
Data Processing Software Developer
Full-stack EngineerSoftware EngineerFull TimeRemoteTeam 51-200Since 2022H1B SponsorCompany SiteLinkedIn
Location
United States
Posted
142 days ago
Salary
Not specified
Bachelor Degree5 yrs expExperience acceptedEnglishLinuxRustSQL
Job Description
• Implement software to perform data processing operations efficiently on large data sets as part of an overall data processing engine.
• Work closely with technical leads and other individual developers to develop this software and ensure data is processed accurately, efficiently, and reliably as part of the overall analytics engine.
• Investigate and address issues in the software and develop necessary enhancements in an ongoing manner.
• Working with cross-functional teams to deliver new algorithms and techniques in the core Datapelago GPU-powered, parallel and distributed execution engine
• Developing enterprise-grade, highly reliable code and owning the end-to-end delivery of solutions ensuring smooth production rollouts
• Designing, implementing, testing, and maintaining enhancements in different components of our engine.
• Developing highly modular components and ensuring clean integration with the entire stack
• Define algorithms, mechanisms, procedures, and policies for the required functionality. Document them, and secure review and approval from the technical lead(s)
• Continuously analyzing and identifying areas for differentiation and improvements in the execution engine
• Identifying bottlenecks in scaling and performance and implementing solutions
• Investigate and resolve all bugs before and after the release of the software
• Support troubleshooting and resolution of issues, incidents, and problems, working in collaboration with others as needed.
• Document findings, resolutions, and identify required enhancements to avoid recurrence
• Collaborating with other teams and team members and assisting with code reviews, design reviews, and development process improvements
Job Requirements
- S. EE/CS or equivalent with 5+ years of experience or MS with 3+ years’ experience
- 5+ years of experience developing database execution engine components including core database operations - joins, aggregations, sorts, analytic functions like window functions, etc., in an enterprise-class database serving large-scale data processing workloads
- Experience developing modern analytic databases and familiarity with vectorized execution, parallel and distributed database algorithms, memory management, etc.
- Good understanding of SQL and familiarity with Lakehouse architectures and technologies
- Solid experience in developing performant code that minimizes overheads and latency and maximizes throughput
- Experienced with the best practices in developing high quality, reliable software
- Experience developing, evaluating, and troubleshooting high-performance software
- Strong programming ability in C, C++. Rust experiences a strong plus
- Strong development experience on Linux platforms
Related Guides
Related Job Pages
More Full-stack Engineer Jobs
Cloud Software Engineer – Observability Platform
ClickHouseClickHouse is an open-source, column-oriented OLAP database management system.
Full-stack Engineer142 days ago
Full TimeRemoteTeam 51-200Since 2016H1B Sponsor
Cloud Software Engineer designing observability systems for ClickHouse.
AWSAzureCloudDistributed SystemsGoogle Cloud PlatformGrafanaKubernetesPrometheusTerraformGo
Senior Software Engineer
Magpie LiteracyMagpie Literacy is a research and design organization focused on unlocking the great reader in every child.
Full-stack Engineer143 days ago
Full TimeRemoteTeam 11-50H1B No Sponsor
Full Stack Senior Software Engineer developing systems for educational nonprofit
Full-stack Engineer143 days ago
Full TimeRemoteTeam 501-1,000H1B Sponsor
Senior Software Engineer extracting insights from electrical networks for fire prevention
ASP.NETAWSAzureCloudIoTPythonSQLTCP/IP.NET
United States
Software Developer, Avid/Adobe Premiere Plugins, Panel, REST API
DigitalFishDigitalFish Delivers the Art and Science of Digital Storytelling
Full-stack Engineer143 days ago
ContractRemoteTeam 11-50H1B Sponsor
Software Developer creating Avid/Adobe Premiere plugins and APIs
JavaScriptMacOS