QA Engineer ID66279

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. Why join us If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! :) About the role We are looking for a Senior QA Engineer to perform manual testing of web applications and validate AI-generated outputs as part of a product team building LLM-integrated features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt–response workflows. What you will do • Perform manual testing of web applications to identify bugs, glitches, and inconsistencies; • Develop, maintain, and execute detailed manual test cases based on product requirements and specifications; • Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers; • Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs; • Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance; • Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate; • Conduct manual API testing to validate request and response accuracy, error handling, and integration; • Use TestRail to create, manage, and track test plans, test cases, and test execution results; • Use Jenkins CI/CD pipelines; • Report and document defects clearly using JIRA; • Participate in daily stand-ups, sprint planning, and other Agile team meetings; • Review user stories and acceptance criteria to understand the scope of testing; • Estimate own tasks and report risks to the team lead; • Collaborate with developers and other QA team members to reproduce issues and verify fixes; • Maintain and update test documentation as needed. Must haves • 4+ years of experience with manual testing; • Basic understanding of methodologies and SDLC and STLC models; • Knowledge of testing and quality assurance processes; • Strong experience with test design activities to cover a set of product requirements; • Knowledge of test design techniques and testing methodologies; • Defect analysis skills and the ability to write detailed and useful defect reports; • Experience with Postman for API testing and browser developer tools; • Experience with UI testing; • Experience with TestRail as a test management system; • Basic understanding of CI/CD processes; • Intermediate English level. Nice to haves • Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles; • Familiarity with AI and ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, or LangChain; • Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization; • Experience with SSO testing; • Experience in performance or load testing. The benefits of joining us • Professional growth Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps • Competitive compensation We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities • A selection of exciting projects Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands • Flextime Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive. Meet Our Recruitment Process Asynchronous stage – An automated, self-paced track that helps us move faster and give you quicker feedback:• Short online form to confirm basic requirements • 30–60 minute skills assessment via Codility – a platform founded in Poland that helps us provide quicker feedback and streamline this stage of the process. • 5-minute introduction video Synchronous stage – Live interviews • Technical interview with our engineering team (scheduled at your convenience) • Final interview with your future teammates If it’s a match — you’ll get an offer!