Introduction:
The first advanced AI audit-specific certification, ISACA Advanced in AI Audit™ (AAIA™), builds on the skills validated by
higher-level audit certifications to empower experienced IT auditors to face the AI challenges and become tomorrow’s
AI audit leaders.
Course Information:
◦ 2 days
◦ Approximately 12 hours
Exam Duration:
◦ 90 questions
◦ Must be completed in 2.5 hours
Required Prerequisite:
◦ Must possess a CISA, CPA or CIA to be eligible for certification
CPE Requirements:
• A minimum of 10 hours of CPE/year in the AI domain.
• CPE can be applied to other certifications as well as part of the 20 annual /120 three-year requirement.
• No additional three-year requirement.
Program Content:
Domain 1. AI Governance and Risk
A. AI Models, Considerations, and Requirements
I. Types of AI
• Generative
• Predictive
• Narrow
• General
II. Machine Learning/AI Models
• Basic models
• Neural networks
III. Algorithms
• Classes of algorithms
• Additional AI considerations (technical terms and
concepts relevant to the IS auditor)
IV. AI Lifecycle Overview
• Plan and design
• Collect and process data
• Build and/or adapt model(s)
• Test, evaluate, verify, and validate
• Make available for use/deploy
• Operate and monitor
• Retire/decommission
V. Business Considerations
• Business use cases, needs, scope, and objectives
• Cost-benefit analysis
• Return on investment
• Internal vs. cloud hosting
• Vendors
• Shared responsibility
B. AI Governance and Program Management
I. AI Strategy
• Strategies
• Opportunities
• Vision and mission
• Value alignment
II. AI-Related Roles and Responsibilities
• Categories, focuses, and common examples
III. AI-Related Policies and Procedures
• Usage policies
IV. AI-Related Policies and Procedures
• Skills, knowledge, and competencies
V. Program Metrics
• Examples of metrics with objectives and definitions
C. AI Risk Management
I. AI-Related Risk Identification
• AI threat landscape
• AI risks
• Challenges for AI risk management
II. Risk Assessment
• Risk assessment
• Risk appetite and tolerance
• Risk mitigation and prioritization
• Remediation plans/best practices
III. Risk Monitoring
• Continuous improvement
• Risk and performance metrics
D. Privacy and Data Governance Programs
I. Data Governance
• Data classification
• Data clustering
• Data licensing
• Data cleansing and retention
II. Privacy Considerations
• Data privacy
• Data ownership (governance and privacy)
III. Privacy Regulatory Considerations
• Data consent
• Collection, use, and disclosure
E. Leading Practices, Ethics, Regulations, and Standards for AI Standards, Frameworks, and Regulations Related to AI
• Best practices
• Industry standards and frameworks
• Laws and regulations
I. Ethical Considerations
• Ethical use
• Bias and fairness
• Transparency and explainability
• Trust and safety
• IP considerations
• Human rights
2. AI Operations
A. Data Management Specific To AI
I. Data Collection
• Consent
• Fit for purpose
• Data lag
II. Data Classification
III. Data Confidentiality
IV. Data Quality
V. Data Balancing
VI. Data Scarcity
VII. Data Security
• Data encoding
• Data access
• Data secrecy
• Data replication
• Data backup
B. AI Solution Development Methodologies and Lifecycle
I. AI Solution Development Life Cycle
• Use case development
• Design
• Development
• Deployment
• Monitoring and maintenance
• Decommission
II. Privacy and Security by Design
• Explainability
• Robustness
C. Change Management Specific To AI
I. Change Management Considerations
• Data dependency
• AI model
• Regulatory and societal impact
• Emergency changes
• Configuration management
D. Supervision of AI Solutions
I. AI Agency
• Logging and monitoring
• AI observability
• Human in the Loop (HITL)
• Hallucination
E. Testing Techniques for AI Solutions
I. Conventional Software Testing Techniques
• A/B testing
• Unit and integration testing
• Objective verification
• Code reviews
• Black box testing
II. AI-Specific Testing Techniques
• Model cards
• Bias testing
• Adversarial testing
F. Threats and Vulnerabilities Specific To AI
I. Types of AI-Related Threats
• Training data leakage
• Data poisoning
• Model poisoning
• Model theft
• Prompt injections
• Model evasion
• Model inversion
• Threats for using vendor supplied AI
• AI solution disruption
II. Controls for AI-Related Threats
• Threat and vulnerability identification
• Prompt templates
• Defensive distillation
• Regularization
G. Incident Response Management Specific To AI
I. Prepare
• Policies, procedures, and model documentation
• Incident response team
• Tabletop exercises
II. Identify and Report
III. Assess
IV. Respond
• Containment
• Eradication
• Recovery
V. Post-Incident Review
3. AI Auditing Tools and Techniques
A. Audit Planning and Design
I. Identification of AI Assets and Controls
• Inventory objective and procedure
• Inventory and data gathering methods
• Documentation
• Surveys
• Interviews
II. Types of AI Controls
• Examples including control categories, controls, and
explanations
III. Audit Use Cases
• Large language models
• Audit process improvement
• Generative AI
• Audit-specific AI applications
IV. Internal Training for AI Use
• Key components for auditor knowledge
• Practical skills development
B. Audit Testing and Sampling Methodologies
I. Designing an AI Audit
• AI audit objectives
• Audit scoping and resources
II. AI Audit Testing Methodologies
• AI systems overall testing
• Financial models
III. AI Sampling
• Judgmental sampling
• AI sampling
IV. Outcomes of AI Testing
• Reduce false positives
• Reduce workforce needs
• Outliers
C. Audit Evidence Collection Techniques
I. Data Collection
• Training and testing data
• Unstructured and structured data collection
• Extract, transform, and load
• Data manipulation
• Scraping
II. Walkthroughs and Interviews
• Design interview questions
III. AI Collection Tools
• Using AI to collect logs
• AI agents to create outputs
• Voice to speech
• Optimal character recognition
D. Audit Data Quality and Data Analytics
I. Data Quality
• Optimization
II. Data Analytics
• Sentiment analysis
• Run data analytics
III. Data Reporting
• Reports
• Dashboards
E. AI Audit Outputs and Reports
I. Reports
• Report types (examples and details)
• Advisory reports
• Charts and visualizations
II. Audit Follow-up
• Automated follow-up
III. Quality Assurance
CPE Credits: 12
Level: Intermediate – Advanced
Field of Study: Auditing
Advance Preparation: Must possess a CISA, CPA or CIA to be eligible for certification
