Design develop and execute automated tests using Python to ensure the functionality and performance of the product
Conduct manual testing where necessary to complement automated tests
Write simple SQL queries for data extraction and validation
Test and validate the product developed by the developers ensuring it meets the specified requirements
Understands the basics of GenAI related tools
Basic understanding of LLM evaluation tools Ragas DepEval SAFE
Define and implement testing strategies
Exhibit excellent communication and documentation skills to clearly report issues document test cases and articulate the results to technical and non technical stakeholders
Apply analytical skills to troubleshoot and resolve complex problems ensuring the highest quality standards are met
Proficient in automating API responses including handling both authorized and unauthorized scenarios
Strong expertise in using Postman and Jenkins
Skills and Qualifications
Proficiency in automated testing with Python
Familiarity with manual testing processes
Good knowledge of SQL for data manipulation and testing purposes
Understanding in prompt engineering for LLMs including question generation and evaluation Good to have
Experience with evaluating chatbot use cases and ensuring their effectiveness and relevance Good to have
Exceptional communication and documentation abilities
Strong analytical and problem solving skills
- Proven experience in testing AI ML based solutions particularly generative AI GenAI
- Proficiency in automated testing with Python
- Familiarity with manual testing processes
- Strong knowledge of SQL for data manipulation and testing purposes
- Demonstrated experience with GenAI evaluation frameworks and testing of ML DL solutions
- Expertise in prompt engineering for LLMs including question generation and evaluation metrics
- Comprehensive knowledge of RAGAS and RAG frameworks
- Experience with evaluating chatbot use cases and ensuring their effectiveness and relevance
- Exceptional communication and documentation abilities
- Strong analytical and problem solving skills
Key job details
About Virtusa
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa is an Equal Opportunity Employer. All applicants will receive fair and impartial treatment without regard to race, color, religion, sex, national origin, ancestry, age, legally protected physical or mental disability, protected veteran status, status in the U.S. uniformed services, sexual orientation, gender identity or expression, marital status, genetic information or on any other basis which is protected under applicable federal, state or local law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government-issued ID during each interview. All candidates must be authorized to work in the USA.
Learn more
Awards and recognition
Syncing your Virtusa Information
Syncing Complete
Exporting your personal Virtusa brand items
Export Complete
Check your downloads folder for files and implementation instructions.
Assets are now available in your profile for future editing and use.
Loading your existing headshots
Please select any one of the images below
Downloading Secure Assets
Signing out