Job description
We are seeking a highly skilled Senior Databricks Engineer to join our data team. The ideal candidate will have strong expertise in Python programming, ETL processes, and in-depth knowledge of Medallion Architecture in modern Data Lakehouse environments. You will play a key role in designing, developing, and optimizing scalable data pipelines and analytics solutions on the Databricks platform.
________________________________________
Key Responsibilities:
Design, develop, and maintain robust ETL pipelines using Databricks and Python.
Implement and optimize the Medallion Architecture (Bronze, Silver, Gold layers) within our Data Lakehouse ecosystem.
Collaborate with data engineers, data scientists, and business stakeholders to translate business requirements into scalable data solutions.
Perform data ingestion, transformation, cleansing, and enrichment from various structured and unstructured data sources.
Optimize Spark jobs for performance and cost-efficiency on Databricks.
Implement best practices for data governance, security, and quality within the data pipelines.
Mentor junior team members and contribute to improving team processes and standards.
Troubleshoot and resolve data pipeline and platform-related issues promptly.
________________________________________
Required Skills & Qualifications:
10+ years of experience in data engineering or a related field.
Strong proficiency in Python programming and libraries related to data processing (PySpark preferred).
Hands-on experience with Databricks platform and Apache Spark.
Deep understanding of ETL concepts and implementation in large-scale data environments.
Expertise in Medallion Architecture and Data Lakehouse design patterns.
Experience with data storage technologies like Delta Lake, Parquet, and cloud data platforms (AWS, Azure, or GCP).
Familiarity with SQL and performance tuning of Spark SQL queries.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration skills.
________________________________________
Preferred Qualifications:
Experience with containerization (Docker/Kubernetes) and orchestration tools (Airflow, Azure Data Factory).
Knowledge of CI/CD pipelines for data workflows.
Exposure to machine learning pipelines and MLOps.
.
Key job details
About Virtusa
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa is an Equal Opportunity Employer. All applicants will receive fair and impartial treatment without regard to race, color, religion, sex, national origin, ancestry, age, legally protected physical or mental disability, protected veteran status, status in the U.S. uniformed services, sexual orientation, gender identity or expression, marital status, genetic information or on any other basis which is protected under applicable federal, state or local law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government-issued ID during each interview. All candidates must be authorized to work in the USA.
Learn more
Awards and recognition
Syncing your Virtusa Information
Syncing Complete
Exporting your personal Virtusa brand items
Export Complete
Check your downloads folder for files and implementation instructions.
Assets are now available in your profile for future editing and use.
Loading your existing headshots
Please select any one of the images below
Downloading Secure Assets
Signing out