english

Apply for  job with Virtusa in Chennai, India. Virtusa helps business move forward, faster by combining deep industry expertise and frictionless technology delivery.

AWS Data Engineer- lead Years of Experience: 5-7 YearsJob Summary: We are seeking a skilled Data Pipeline Engineer to design, develop, and maintain scalable data pipelines using AWS technologies. The ideal candidate will have hands-on experience with AWS Glue, Glue Catalog, Amazon S3, Athena, and AWS Lambda, and will be proficient in building and optimizing ETL/ELT workflows using PySpark and Python. This role requires an independent contributor who can work under tight timelines and deliver robust solutions.Responsibilities:<ul><li>Design, develop, and maintain scalable data pipelines using AWS Glue, Glue Catalog, Amazon S3, Athena, and AWS Lambda.</li><li>Build and optimize ETL/ELT workflows using PySpark and Python for large-scale data processing.</li><li>Develop and manage real-time data streaming pipelines using Apache Kafka, ensuring low latency and high reliability.</li><li>Create, maintain, and optimize SQL queries for data extraction, transformation, and analysis.</li><li>Implement data ingestion frameworks to handle both batch and streaming data from multiple sources.</li><li>Troubleshoot, debug, and optimize data workflows to meet performance and scalability requirements.</li><li>Ensure data quality, consistency, and integrity across different data platforms and pipelines.</li><li>Collaborate with cross-functional teams (data engineers, analysts, and stakeholders) to understand data requirements and deliver robust solutions.</li><li>Monitor and maintain production pipelines, ensuring high availability and quick issue resolution.</li></ul>Mandatory Skills:<ul><li>Hands-on experience with AWS Glue, Glue Catalog tables, Athena, S3, and Lambda.</li><li>Proficiency in PySpark and Python.</li><li>Experience with Apache Kafka for building real-time data pipelines.</li><li>Strong SQL skills for data extraction and transformation.</li><li>Ability to work independently and deliver solutions under tight timelines.</li></ul>Preferred Skills:<ul><li>Experience with Glue Streaming.</li><li>Familiarity with AWS CDK.</li></ul>Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field.

AWS Data Engineer- Lead

Key Job Details

Primary Location

Job Type

Required Experience

Skill

Travel

Date Posted

Certified Top Employer

UK's Best Workplaces for Wellbeing 2024

Virtusa earns 2024 UK’s Best Workplaces in Tech

Virtusa Earns 2022 Great Place To Work Certification gggggggg

Candidate Experience (CandE) Awards

Services

Industries

Insights

Careers

About