Automated Annual Model Monitoring to Drive Efficiency at Scale

Business Objective

Our client is a leading credit reporting agency that offers credit bureau and decision analytics services to businesses, along with providing credit reports to individual customers. The client conducts periodic model monitoring exercises for various generic and custom scorecard models for multiple institutions.

They want an Automated Annual Model Monitoring tool. The objective was to streamline, automate, and sequence the entire exercise of model monitoring so that it can run independently as a single process with minimal intervention and generate error-free results. The result is a distribution-ready set of summary reports and associated commentary on model performance and validation metrics.

  • The exercise involved validating over one hundred models for consumers and small businesses for different product portfolios, coming from disparate data sources
  • The existing process had multiple steps that required manual intervention at various steps, leaving room for errors
  • Several hundred different programs used hardcoded references and paths. Hundreds of intermediary files were generated and had to be checked manually before creating validation metrics

Solution Methodology
  • Restructured the code and streamlined the data collection process with in-built quality checks at each stage of data in motion
  • Different analytical platforms such as SAS and Python were streamlined with Microsoft Office integration tools so that there was no manual editing, hardcoding or intervention required
  • Generated a dashboard so that model performance can be easily shared with stakeholders
  • Automated creation of validation presentations with commentary and analyses updated at regular cycles
Business Impact
  • Increase of efficiency by over 92% (from 12 weeks to 1 week) for the model monitoring exercise

  • Reduction of defect opportunity by minimizing manual steps

Copyright © 2023 Tiger Analytics | All Rights Reserved