Virtusa Recognized as Global Leader in Gen AI Services by ISG Provider Lens® - Read report
Job description
Key Responsibilities
Design develop and maintain scalable data pipelines using Apache Spark on Databricks
Write efficient and production ready PySpark or Scala code for data transformation and ETL processes
Integrate data from various structured and unstructured sources into a unified platform
Implement Delta Lake and manage data versioning updates and schema evolution
Optimize data processing workflows for performance scalability and cost efficiency
Collaborate with data scientists analysts and business stakeholders to deliver high quality datasets
Implement data quality checks validation routines and logging mechanisms
Monitor and debug production jobs using Databricks jobs notebooks and clusters
Ensure security privacy and compliance standards are met throughout the data lifecycle
Provide guidance and mentorship to junior team members
Required Skills and Qualifications
5 to 7 years of experience in Big Data development
Hands on experience with Databricks including Workflows Notebooks Delta Live Tables Unity Catalog
Strong programming skills in PySpark and or Scala
Solid understanding of Delta Lake architecture
Proficient in SQL for data analysis and transformation
Experience with cloud platforms such as Azure Azure Data Lake Data Factory Synapse or AWS S3 Glue Redshift
Familiarity with CICD for Databricks deployments using GitHub Actions Azure DevOps
Knowledge of data governance cataloguing and security best practices
Experience working in an Agile Scrum environment
Preferred Skills
Experience with Databricks Unity Catalog and Delta Live Tables
Exposure to machine learning workflows in Databricks
Experience with Apache Airflow Kafka or other orchestration messaging tools
Certifications Databricks Certified Data Engineer Associate Professional Azure AWS certification
Key job details
About Virtusa
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa is an Equal Opportunity Employer. All applicants will receive fair and impartial treatment without regard to race, color, religion, sex, national origin, ancestry, age, legally protected physical or mental disability, protected veteran status, status in the U.S. uniformed services, sexual orientation, gender identity or expression, marital status, genetic information or on any other basis which is protected under applicable federal, state or local law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government-issued ID during each interview. All candidates must be authorized to work in the USA.
Learn more
Awards and recognition
Syncing your Virtusa Information
Syncing Complete
Exporting your personal Virtusa brand items
Export Complete
Check your downloads folder for files and implementation instructions.
Assets are now available in your profile for future editing and use.
Loading your existing headshots
Please select any one of the images below
Downloading Secure Assets
Signing out