Our client is one of the top 3 life insurance companies and the largest mutual life insurance company in the United States. The client had a Hortonworks Data Hub platform that was obsolete and was at the end of its life. This data platform had to be migrated to the latest technology and be future-proof while addressing existing challenges, helping the business become faster and more efficient at decision making.
Our client had a Hortonworks Data Hub platform, which reached end-of-life support and needed to be migrated to a platform that supports future requirements and addresses current challenges with minimum development efforts. The existing platform had many challenges like:
- Performance issues, especially when a DB query and a batch load run simultaneously
- Scalability issues in meeting BI workloads on-demand reporting needs
- Data recovery processes
- Data validation processes
Our top priority was to address all these challenges while designing a new data platform solution and build secure, well-governed, reusable enterprise data assets to drive business insights and enable the business to make faster and more efficient decision making.
Virtusa collaborated with the client SMEs to develop a modern data platform strategy leveraging various AWS cloud capabilities, predominantly Redshift.
- Virtusa as a strategic technology partner, collaborated with the client to develop a plan to modernize the current data platform on top of AWS Cloud Services.
- The technology stack includes data ingestion capabilities, raw data storage, and data structuring. It will further leverage an array of proven, industry-leading tools available in AWS.
- Modern architecture that allows processes to run simultaneously without conflicts
- Compute resources that can be scaled automatically in a controlled environment to meet spikes in demand efficiently
- Backup strategy for point in time Object recovery
- Data replication leveraging cloud capabilities
- Automation to support code promotion across environments
- The process to minimize the use of clear text PII data and tenant zones by conforming PII data to Data Governance standards before tokenization
- 40% efficiency gain using a new metadata-driven PySpark framework to apply data governance standards