Business Objective
Our client is a leading global provider of mobility solutions. The client wanted to improve overall time to market and accelerate the dashboarding process.
The client also wanted to set up a standard mechanism for data ingestion and improve SQL data processing.
Challenges
- Slow time-to-market the reports and dashboards to business users
- Lack of standard data ingestion mechanism
- Normalized data models have been a bottleneck
- SQL heavy data processing slowing down the Development lifecycle
- Multiple file ingestion mechanisms
Solution MethodologyÂ
- Worked closely with business stakeholders and IT stakeholders to drive strategic initiatives and tactical use cases
- Leveraged metadata-driven Data Lake Ingestion accelerator to standardize and accelerate data ingestion pipelines
- Cloud Data platform using AWS Redshift that feeds in data from different data sources first to Amazon S3 and then to target external tables in Redshift
- Leveraged DBT to accelerate ETL development using Jinja templates and Macros
- Consolidated file Ingestion patterns
Business Impact
- Reduced time-to-market from 4 days to 3-4 hours by redesigning the existing data model and eliminating redundancy
- Improved overall business performance by providing actionable insights
- Reduced the maintenance cost by implementing a standard data ingestion mechanism
