Pryon logo
Pryon

AI-powered enterprise knowledge management

Senior Software Engineer, Ingestion Team

Full-stack EngineerSoftware EngineerFull TimeRemoteSeniorTeam 51-200H1B No SponsorCompany SiteLinkedIn

Location

New York

Posted

2 days ago

Salary

$175K - $200K / year

Seniority

Senior

Bachelor Degree5 yrs expEnglishAirflowAWSAzureCloudDistributed SystemsDockerETLGoogle Cloud PlatformKafkaKubernetesPythonRabbitMQTerraformGo

Job Description

• Design and build pipeline stages for our modern ingestion architecture - from document intake through embedding generation and index writing • Contribute to the design of next-generation pipeline architecture as the system evolves • Improve system stability and scale: identify bottlenecks, reduce failure rates, and build observability into every stage • Work with workflow orchestration tools to manage complex, multi-step document processing with retry logic, error handling, and state management • Handle the realities of document diversity: PDFs, HTML, Office formats, images, structured and semi-structured data - all flowing through the same pipeline • Collaborate with the Connectors team (upstream) and Retrieval team (downstream) to ensure data flows cleanly across system boundaries • Participate in the ongoing migration from legacy systems, balancing new development with operational stability

Job Requirements

  • 5+ years of software engineering experience, with meaningful time on data processing pipelines, ETL systems, or similar infrastructure
  • Strong proficiency in Python and/or Go
  • Experience with workflow orchestration tools — Temporal, Airflow, Prefect, Step Functions, or similar
  • Understanding of distributed systems patterns: queues, workers, backpressure, idempotency, retry strategies
  • Hands-on experience with Kubernetes, Docker, Terraform, and Helm
  • Familiarity with message brokers and event streaming (Kafka, RabbitMQ, SQS, or similar)
  • Comfort working across cloud providers (AWS, Azure, GCP)

Benefits

  • Remote first organization
  • 100% Company paid Health/Dental/Vision benefits for you and your dependents
  • Life Insurance, Short-term and Long-term Disability
  • 401k
  • Unlimited PTO

Related Job Pages

More Full-stack Engineer Jobs

Full-stack Engineer2 days ago
Full TimeRemoteTeam 10,001+Since 1931H1B Sponsor

Full Stack Developer for enterprise applications at Allstate

AWSAzureCloudDockerJavaJavaScriptKafkaKubernetesMicroservicesMongoDBNoSQLPostgreSQLPythonReactSpringSpring BootSpringBootSQLTypeScript
Illinois
$70.1K - $121.5K / year
DigitalOcean logo

Senior Full-Stack Engineer II - AI/ML

DigitalOcean

DigitalOcean is the Inference Cloud — a full-stack, production-ready cloud platform built to run AI applications with predictable performance, sustainable economics, and radically simpler operations at scale. We are built for teams turning AI into real products — not just training models. Our advantage is not fewer features, but fewer failure modes when operating AI at scale — combining minimal operational overhead, predictable cost efficiency, and a full-stack cloud that works as a system. Hyperscalers are broad by design. Neoclouds are infrastructure-first. DigitalOcean is inference-first — with a real cloud underneath. It combines inference-optimized compute, managed inference software, and integrated cloud capabilities that reduce operational burden for teams running real workloads. Inference is the foundation—not the boundary. Everything else builds on top of it.

Full-stack Engineer2 days ago
Full TimeRemoteTeam 1,400Since 2012

The engineer will own a technical area by providing technical leadership and independently drive large cross-functional projects to completion, primarily using Python and Go to build and scale AI application services like ADK and agentic tools. Responsibilities also include maintaining and developing APIs, MCPs, and SDKs, collaborating on design, and shipping product features from planning to maintenance with high autonomy.

PythonGoJavaScriptTypeScriptNode.jsReactREST APIGraphQLDockerKubernetesAWSPostgreSQLMongoDBGitCI/CDMicroservicesMachine LearningLLMsAI PlatformsCloud Architecture
United States
$184K - $231K / year
CelerData logo

Senior/Lead Software Engineer, Database System

CelerData

The only SQL engine that is fast enough to run the most demanding workloads directly on your data lakehouse.

Full-stack Engineer2 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor

Senior Software Engineer developing high-performance SQL engine for composable lakehouses

ApacheDistributed SystemsJavaSparkSQL
United States
Schoolhouse.world logo

Senior Software Engineer

Schoolhouse.world

Connecting the world through learning

Full-stack Engineer2 days ago
Full TimeRemoteTeam 1-10Since 2020H1B No Sponsor

Senior Software Engineer building impactful features for online learning platform

JavaScriptNext.jsNode.jsPostgreSQLReactTypeScript
United States
$181K - $207K / year