AI Trust and Governance Architect

Primary Duties:

• Primary Duties
• AI Assurance Architecture
• Develop platforms and frameworks for evaluating and benchmarking AI assurance
• Create systems for assessing LLM agents and RAG across functional, non-functional, and risk dimensions
• Establish architectural patterns for Responsible AI, focusing on bias detection, explainability, and safety validation
• Construct reusable assurance components that facilitate Business Assurance, Risk Assurance, and Reliability
• Security, Reliability, and Governance
• Design AI testing and validation processes addressing security, privacy, prompt injection, and adversarial robustness
• Incorporate red teaming, threat simulation, and chaos-style validation into AI systems
• Define governance frameworks for model usage, ensuring auditability, traceability, and compliance
• Guarantee that AI systems adhere to enterprise standards for resilience, fault tolerance, and observability
• Platform Engineering Facilitation
• Develop AI assurance platforms that support automated test execution, reporting, and insights
• Facilitate integration with CI/CD pipelines to establish AI quality gates
• Collaborate with QE engineering teams to weave AI assurance into the Software Development Life Cycle (SDLC)
• Guide teams in identifying and mitigating AI risks from an engineering standpoint
• Core Platforms, Frameworks, and Tools
• Frameworks for LLM and AI evaluation, including PromptFoo, DeepEval, and custom LLM evaluation harnesses
• Tooling for prompt RAG and agent validation, comprising prompt testing frameworks, retrieval accuracy validators, and agent workflow evaluators
• Responsible AI and model risk tooling, including Fairlearn, SHAP, Explainable AI libraries, and toxicity and bias scanners
• Security and adversarial testing tools for AI systems like PyRIT and Garak
• AI red teaming and threat simulation frameworks featuring automated red team scripts and adversarial test suites for LLMs and agents
• Automation frameworks for AI assurance and quality engineering, such as Galileo
• Observability solutions for AI behavior and drift, including Langfuse, Arize, Evidently, and custom telemetry dashboards
• Client-Focused Leadership
• Collaborate with product and engineering teams to uncover AI Assurance opportunities and shape strategic roadmaps
• Facilitate client workshops, RFPs, and solution presentations
• Mentor engineers on AI, ML, and Generative AI best practices and emerging technologies
• Transform complex AI concepts into easily understandable business narratives

Technical Qualifications:

• Essential Qualifications
• 13 years of software engineering experience, with at least 3 years dedicated to AI and a strong focus on architectural ownership
• Hands-on experience with AI, ML systems, LLM evaluation, and assurance frameworks
• Experience in AI red teaming, model risk management, or AI auditing tools
• Deep understanding of Responsible AI, AI risks, and governance principles
• Background in security testing, adversarial testing, and reliability engineering
• Proficiency in Python automation frameworks and cloud computing platforms
• Desirable Skills
• Familiarity with regulatory and compliance requirements for AI systems
• Experience in performance engineering, chaos engineering, or resilience testing for AI
• Contributions to internal platforms, frameworks, or standards

Preferred Competencies:

Domain->Strategy->Communication Design->Design strategy for the respective domain,Domain->Information Security->Governance Risk and Compliance, Audits, Workflow,Foundational->Governance_COBIT->Assurance,Technology->AI-Generative AI->Conversational AI Platform,Technology->AI-Generative AI->Generative AI - Basic,Technology->AI-Generative AI->Prompt Engineering,Technology->AI-Responsible AI->Responsible AI