Job Description
As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for learning, growing and making a difference. At Deltek, we take immense pride in creating a balanced, values-driven environment, where every employee feels included and empowered to do their best work. Our employees put our core values into action daily, creating a one-of-a-kind culture that has been recognized globally. Thanks to our incredible team, Deltek has been named one of America's Best Midsize Employers by Forbes, a Best Place to Work by Glassdoor, a Top Workplace by The Washington Post and a Best Place to Work in Asia by World HRD Congress. www.deltek.com

Deltek is seeking a skilled specialized Automation Engineer with expertise in testing and test automation of AI-powered workflows and intelligent features across ERP product lines. This role blends traditional ERP quality engineering with next-gen AI validation, ensuring that embedded AI capabilities are accurate, safe, compliant, explainable, and scalable.
The ideal candidate needs strong ERP domain knowledge, hands-on automation expertise, and understands how to evaluate LLM-driven responses, predictive analytics, recommendations, and AI decision support outputs. This role requires a detail-oriented individual who can work collaboratively with cross-functional teams to ensure the delivery of high-quality ERP solutions.
Responsibilities

• Develop end-to-end test strategy for AI features across ERP modules such as Finance, Supply Chain, Projects, HR, and Payroll
• Execute functional, regression, scenario-based, and exploratory testing for AI-enhanced workflows
• Validate AI/ML outputs including:
  • Prediction accuracy & scoring
  • Recommendation relevance
  • Natural-language responses
  • Behavior consistency across edge cases
• Create and maintain automation suites for UI/API and AI validations
• Design AI safety test cases → hallucination detection, bias testing, ethical and compliance alignment
• Collaborate with Product Managers, AI/ML engineers, SMEs, and Developers to define release quality gates
• Ensure test coverage of data pipelines: input validation, data sampling, transformations, and output visualizations
• Contribute to performance, scalability & data-drift testing of AI features in ERP contexts
• Maintain test documentation — test plans, test cases, defect logs, and sprint reports
• Drive automation adoption in CI/CD pipelines for AI-enabled releases
• Provide mentorship to senior engineers and teams, guiding best practices and fostering professional development.

Qualifications & Experience:

• BS/MS degree in Computer Science or a related field.
• 6+ years of proven experience in automation testing, with at least 2+ years working with AI or ML-based systems
• Proven track record of designing and implementing AI Based automation solutions.
• Expertise in multiple automation tools and frameworks, with a deep understanding of the latest technologies.
• Experience in developing and refining overall strategies for API testing.
• Proficiency in using version control systems like Git for managing code.
• Experience with cloud-based ERP solutions.
• Experience with performance and security testing.
• Strong debugging and analytical skills

Good-to-Have Skills

• Understanding of LLM fine-tuning, vector databases, RAG pipelines, embeddings
• Familiarity with data privacy & compliance frameworks
• Experience with security testing for AI systems
• Exposure to AIOps or self-healing automation frameworks

Technical Qualifications:

• In-depth understanding of modern QA methodologies, tools, and processes with a focus on AI-driven features and intelligent workflows.
• Strong understanding of API architectures including REST, SOAP, GraphQL, and WebSocket, along with experience validating AI-generated responses via APIs (e.g., LLM response endpoints, inference APIs).
• Hands-on experience with API testing tools such as Postman, SoapUI, JMeter, or REST Assured, including automated evaluation of AI output scoring / JSON inference payloads.
• Proficiency in one or more programming languages — Java, Python, or JavaScript — with the ability to build AI output validation utilities, automated prompt-testing harnesses, or data-driven frameworks.
• Knowledge of CI/CD pipelines (Jenkins, Azure DevOps, GitHub Actions) and automation integration for AI feature regression and model-response validation.
• Familiarity with LLM / ML concepts from a QA perspective — prompt testing, hallucination detection, prediction scoring, explainability validation, bias checks, guardrail/output sanitization.
• Strong SQL and database testing skills, including validation of training datasets, feature store data, and end-to-end AI data pipelines.
• Good understanding of Agile / Scrum development practices with experience participating in AI model release cycles (shadow mode, AB comparison, fine-tuning validation, or phased rollout).

Soft Skills:

• Excellent communication and collaboration skills.
• Ability to manage multiple tasks/projects concurrently.
• Strong interpersonal skills for working in a team-oriented environment.
• Excellent problem-solving skills and attention to detail.
• Ability to work independently and as part of a team in a fast-paced environment.
• Proactive and self-motivated with a strong desire for continuous learning and improvement.

Applicant Privacy Notice
Deltek is committed to the protection and promotion of your privacy. In connection with your application for employment with us at Deltek, it is necessary for us to collect, store and use information about you (“Personal Data”) to administer and evaluate your application. We are the “controller” of the Personal Data you provide us and will process any such Personal Data in accordance with applicable law and the statements contained in this Employment Candidate Privacy Notice. Additionally, we have not sold and do not sell Personal Data you provide to us through the job application process.

Business Summary
The Deltek Engineering and Technology team builds best-in-class solutions to delight customers and meet their business needs. We are laser-focused on software design, development, innovation and quality. Our team of experts has the talent, skills and values to deliver products and services that are easy to use, reliable, sustainable and competitive. If you're looking for a safe environment where ideas are welcome, growth is supported and questions are encouraged – consider joining us as we explore the limitless opportunities of the software industry.