english

data-integration

Apply for  job with Virtusa in Chennai, India. Virtusa helps business move forward, faster by combining deep industry expertise and frictionless technology delivery.

<html><body>Job Description: Data Engineer (Mid-Level, AWS &amp; Databricks) Role Overview As a Mid-Level Data Engineer, you will develop and maintain the data pipelines that form the backbone of our AWS Data Lakehouse. Your mission is to translate business requirements into efficient PySpark code and ensure our SQL Server data is accurately migrated and synchronized with Databricks. Key Responsibilities ETL Development: Develop and debug data pipelines in Databricks using PySpark and SparkSQL. Data Ingestion: Implement data movement from SQL Server to Amazon S3 using efficient batch and incremental loading techniques. Database Operations: Query and manage SQL Server environments to validate data consistency during the migration phase. Modernization: Refactor existing SQL-based transformations into modular Python scripts or Scala functions. Required Skills &amp; Qualifications 4-6 years of experience in Data Engineering. Hands-on Experience: Building production pipelines in Databricks on AWS. Core Competencies: Strong Python, PySpark, and complex SQL (T-SQL preferred). AWS Basics: Familiarity with S3 bucket management and basic IAM roles. Adaptability: Ability to work across Python and SQL comfortably, with an interest in learning/using Scala for performance tuning. Nice to Have Prior exposure to SSIS (this is a plus, not a requirement). Knowledge of AWS Glue or AWS Lambda for lightweight orchestration.</body></html>

Job Description: Data Engineer (Mid-Level, AWS &amp; Databricks) Role Overview As a Mid-Level Data Engineer, you will develop and maintain the data pipelines that form the backbone of our AWS Data Lakehouse. Your mission is to translate business requirements into efficient PySpark code and ensure our SQL Server data is accurately migrated and synchronized with Databricks. Key Responsibilities ETL Development: Develop and debug data pipelines in Databricks using PySpark and SparkSQL. Data Ingestion: Implement data movement from SQL Server to Amazon S3 using efficient batch and incremental loading techniques. Database Operations: Query and manage SQL Server environments to validate data consistency during the migration phase. Modernization: Refactor existing SQL-based transformations into modular Python scripts or Scala functions. Required Skills &amp; Qualifications 4-6 years of experience in Data Engineering. Hands-on Experience: Building production pipelines in Databricks on AWS. Core Competencies: Strong Python, PySpark, and complex SQL (T-SQL preferred). AWS Basics: Familiarity with S3 bucket management and basic IAM roles. Adaptability: Ability to work across Python and SQL comfortably, with an interest in learning/using Scala for performance tuning. Nice to Have Prior exposure to SSIS (this is a plus, not a requirement). Knowledge of AWS Glue or AWS Lambda for lightweight orchestration.

Data Engineer(ETL, SQL, SSIS, Python), AWS

Key Job Details

Primary Location

Job Type

Required Experience

Skill

Travel

Date Posted

Certified Top Employer

UK's Best Workplaces for Wellbeing 2024

Virtusa earns 2024 UK’s Best Workplaces in Tech

Virtusa Earns 2022 Great Place To Work Certification gggggggg

Candidate Experience (CandE) Awards

Services

Industries

Insights

Careers

About