How AI-Led Intent and Resolution Modelling Enabled a UK Bank to Improve Customer Service Performance

Key Highlights: What This Case Study Covers

• NLP-based program for extracting intelligence from chatbot and telephonic conversations
• Automated intent categorisation with topic modelling and semantic clustering
• Call-resolution classification through semi-supervised learning and active-learning loops
• Cleaned, structured, and processed transcripts using robust pre-processing and embedding techniques
• Single workflow for bringing together labelling, modelling, validation, and SME feedback

Client Overview

One of the UK’s leading retail and commercial banks, serving millions across personal and business banking. The bank manages thousands of daily chatbot and phone interactions, generating large volumes of unstructured data that was previously underutilised due to scale, quality issues, and reliance on vendor-led analysis.

The Ask

Build an in-house NLP capability to extract actionable insights from customer interactions thus improving service quality, chatbot responses, first-call resolution, and agent performance benchmarking.

Challenges

• Noisy and inconsistent telephonic transcripts limited reliable intent and resolution detection.
• Call-resolution tagging rules were unclear and required repeated alignment with business teams.
• Customer interactions were split across chatbot and phone channels, reducing consistency of insights.
• Reliance on third-party vendors left no internal capability or reusable models.
• High transcript volumes required scalable, programmatic labelling approaches.

Our Solution: NLP-Driven Conversational Intelligence

Data Ingestion

Chat and telephonic transcripts were ingested from the client’s S3 environment. Invalid and low-quality records were filtered, and programmatic labelling was performed using semantic matching, custom scenarios, and business-rule logic.

Text Pre-Processing

Transcripts were standardised using part-of-speech tagging, lemmatisation, named entity recognition for entity removal, and stop-word elimination to reduce noise.

Feature Engineering

Text was converted into word and sentence embeddings. High-dimensional vectors were reduced using UMAP, and bi-grams were extracted to create structured model inputs.

Modeling

Intent modelling evaluated LDA and BERTopic, with BERTopic selected for interpretable clustering and active-learning feedback. Call resolution used semi-supervised fastText models, refined iteratively through SME-guided active learning.

Validation

Outputs were validated using probability scores and SME review, with intent aligned to chatbot categories and resolution assessed using precision, recall, and F1-score.

Output Generation

The solution produced intent outputs covering primary and secondary reasons for calling with supporting sentences and keywords, along with call-resolution outcomes with confidence scores and contextual evidence.

Impact Delivered

• Scalable Intelligence Foundation: Built an NLP foundation using 200k call and 60k chat transcripts.
• Clear Reasons for Calling: Enabled “Reason to Call” insights with primary and secondary drivers and supporting context.
• Transparent Resolution Outcomes: Delivered call-resolution classification with confidence scores and contextual evidence.
• Proven Model Accuracy: Achieved 80%+ accuracy for intent and 70%+ accuracy for resolution.
• Operational and Revenue Uplift: Reduced repeat calls, improved agent benchmarking, and unlocked revenue opportunities.