Conceptualizing and Implementing an Automated Model Management Solution for a US-based Insurance Provider

Business Objective

Our client, a leading insurance provider in the US, had over 200 predictive models deployed in production that were monitored and retrained in silos.

The client’s data science team partnered with Tiger to accomplish the following objectives:

Develop a framework that underlies a consistent and ongoing model management solution
Build an automated solution to significantly reduce the time taken to monitor and re-train models

Exceptionally fragmented and laborious model management process, which required reaching out to multiple stakeholders to align objectives and solution approach
Lack of any model inventory that could help the team gain an initial understanding of models

Model Inventory: Captured business purpose, data source information, model process flows, quality check measures, SOPs (standard operating procedures), and made them available through a web-based tool
Data and Model Monitoring System: Captured data health reports, model execution status, model run times, model execution failure statistics, and notification/alert system
Model Retrain System: Included a web-based application in Python Flask that orchestrated the overall automation – data extraction, feature engineering, EDA, model retrain, result validation
Visualization Suite: Provided dashboards with an ability to drill down on model failure details in Kibana, Elastic Stack – such as failure instances, stage of failure, causes

Migrated hundreds of models to the new, custom-built platform for ongoing monitoring
Reduced effort by more than 90% allowing data scientists to focus on core development
Saved license costs by using open-source technology

Privacy Notice Cookie Policy