AI Model Validation Lead

We are looking for a passionate and experienced AI professional to develop, manage and execute our AI Model Validation processes across the Prudential Group.

The AI Model Validation Lead will build a team and the required processes that provide confidence in our AI systems ability to deliver business benefits with all risks manage appropriately. The team will validate internally built use cases as well as vendor and off the shelf solutions. The team will execute validation in internal systems (DataBricks, GCP etc.) and also manage independent third party assessments.

The position plays a critical role in scaling AI at Prudential. With more AI systems being deployed, appropriate automations and technical support to streamline AI governance processes.

The successful candidate must have strong technical abilities in AI and machine learning.

Key Responsibilities

AI Model Validation Technical Assessment

• Establish a high-level standardized validation framework that comprehensively assesses accuracy, fairness, robustness, and reliability across model types and use cases.

• Curate test and validation datasets that allow for adequate testing on AI models ensuring.

• Integrate risk-based scoring frameworks to classify models by impact and sensitivity, guiding the depth of validation required.

• Design testing frameworks for individual AI models that are appropriate for each model's risk profile.

• Expand validation coverage to include adversarial testing, edge-case analysis, challenger models and stress testing for high-risk deployments.

Automated and Streamlined Model Evaluation

• To support scalability, develop a library of tests to be reused across similar use cases (for example standard tests for OCR, cross sell models etc.)

• Where possible, embed validation checkpoints into CICD pipelines and the SDLC to enable continuous, scalable oversight of model performance.

• Develop and make available modular evaluation tools tailored for Traditional AI, Generative AI, and Agentic AI systems, ensuring flexibility and reuse.

• Develop processes to detect model degradation such as drift detection, and build alerting mechanisms to monitor model behaviour post-deployment and trigger re-validation when needed.

Stakeholder management

• Work with stakeholders across the organization, including data scientists, engineers, business leaders and LBUs to ensure sufficient input and buy-in for decisions and appropriate and clear requirements are defined.

• The ability to fluently discuss both technical and business concepts with stakeholders.

• The ability to lead and gain the commitment to change using influence.

• Establish and maintain external relationships with vendors.

Other Responsibilities

• Stay up to date with industry trends and technical developments across the AI ecosystem

• Participate in industry forums and working groups that advance AI adoption.

• Stay abreast of Prudential's AI Objectives and AI Strategy

Requirements

• Significant experience developing and deploying AI and machine learning solutions into production environments.

• Solid understanding of AI and machine learning evaluation method

• At least 10 years experience in data science or AI in financial services

• Excellent communication and interpersonal skills, with the ability to build relationships

• Strong experience with Azure DataBricks

• Experience with Google Cloud (VertexAI) is a nice to have

• Strong experience with Dev/Data/ML/AI Ops (a preference for experience with ML or AI Ops)

• Experience in API development