Responsible AI in action, Part 3: Tools to help
This is the third in a series of Responsible AI (RAI) articles:
- Part 1: Get started
- Part 2: Complete an impact assessment
- Part 3 (this article): Tools to help
Introduction
As we increasingly rely on AI assistance in our work and personal lives, there needs to be a practical approach for assessing and mitigating the risks that come with the technology. From privacy concerns to algorithmic bias, there are many factors to consider when assessing and mitigating these risks. Responsible AI (RAI) frameworks and processes are becoming more established, but tooling is an emerging focus for both open source and commercial platforms. The purpose of this article is to raise awareness about tools and resources that are available to help support us as we design, develop, and deploy AI safely and responsibly. To read more about Responsible AI, please see my previous article “Responsible AI in action, Part 1: Get started.”
Challenges
Tools are needed to advance the responsible design, development, and deployment of AI systems. Despite the existence of building blocks and recommendations, the overall landscape is fragmented, making it difficult to establish an end-to-end workflow. Among the challenges:
- There is no one tool that can holistically assess potential AI harms.
- Tools often do not map directly to a Responsible AI principle or standard, such as fairness or transparency, making it difficult to select an appropriate tool or understand how to apply the tool effectively.
- Components and libraries built to support distinct aspects of RAI practice often require additional effort for them to be used in complementary ways.
- Tools vary in their specificity and may require exploration before selection of the best fit for a given AI scenario.
The content presented below is based on experience and learning from a Microsoft organization comprised of 1700 employees and multiple product teams delivering both internal and external applications. In the past year, the organization has seen a dramatic shift in adoption of Generative AI (GenAI)–based solutions compared to traditional Machine Learning (ML) solutions. Today 66 percent of this organization’s AI solutions are GenAI solutions. The suggestions below are based on the tools and techniques available to the teams in this organization.
RAI metrics
Because AI is sociotechnical, with AI outputs typically relying on statistical methods applied to information about people and the physical world, it is inevitable that failures sometimes occur. Teams developing and deploying AI responsibly must work to first identify, and then measure and mitigate, the potential harms of AI outputs to individuals and society. This is iterative work throughout the lifespan of an AI system, with no “one-and-done” tool or solution.
From early-stage impact assessment to the pre-deployment testing and red teaming of AI systems, surfacing potential biases, errors, or other RAI issues for measuring and mitigation is essential to accountability. Building the right set of metrics for measuring and mitigating potential harms is likely to be an iterative process as teams conduct testing and observe an AI system in production. For sociotechnical systems, measuring for model accuracy alone is not enough. Responsible AI measurement involves considerations of both technical attributes, like accuracy and security, and socio-technical attributes such as fairness and its many dimensions, including representational harms, quality of service, and allocation harms; reliability and safety; and explainability.
Getting started with metrics
A starter set of metrics to consider for either traditional Machine Learning (ML) or Generative AI (GenAI) systems could be platform, usage, and operational level metrics. For instance:
- Resource consumption: GPU, CPU, memory utilization, model versions, response times.
- Response/request tracking: number of unique/active users, number of requests, number of prompts, token usage.
- Operational metrics: capacity (quota), cost, latency, throttling.
Note: In Azure, platform telemetry is available with Azure Monitor. See Monitoring Azure OpenAI Service and Monitor Azure Machine Learning.
Add a layer of metrics to measure the potential for RAI harms identified by cross-disciplinary teams in their testing and red teaming efforts for the given AI system. Metrics may need to be tailored because of the uniqueness of an AI system. Consider:
- For ML systems: metrics related to performance, including precision, recall, accuracy, F1 scores and — especially important for ML systems — detections for model and data drift (performance over time).
- For GenAI systems: number of input prompts blocked, number of responses blocked, number of jailbreak prompt attempts, metrics to measure content categories that are being filtered (for example hate, sexual violence, self-harm, politics) and metrics related to quality of the generated output such as groundedness, relevance, and similarity.
- Feedback: sentiment, percent of upvotes/downvotes.
Metrics are the backbone for evaluating how well an AI system is meeting intended RAI objectives. But it is not just about collecting data. Teams need to experiment and iterate with a process to define, collect, and report metrics that are consistent and reliable. This will facilitate deployment of an AI system at scale. Keep in mind that human oversight is always required for accountability in responsible, human-centered AI.
RAI risk-management cycle
The cornerstone of an RAI practice is a cycle of identifying, measuring, and mitigating potential harms and risks. RAI tools can be conceptually organized as useful within these risk stages of the risk-management cycle.
- Identify: This is the first stage and includes activities to identify potential types of potential harms that could occur in an AI system. Ideally issues are identified early during system design, but issues may also be identified during development or after the system has been deployed. For additional guidance on identifying potential RAI harms, see Identify.
- Measure: These are activities to identify and detect the occurrence of an RAI event. They can include defining metrics that are relevant for a scenario, improving monitoring capabilities to detect incidents, or running tests at scale to measure the extent of a potential harm. For additional guidance on measuring RAI harms, see Measure.
- Mitigate: This stage includes activities to reduce risk after an issue has been identified. Examples of mitigations can include managing communications, conducting an effective analysis, and implementing improvements. For additional guidance on how to mitigate RAI harms, see Mitigate.
The tools included in the following section are mapped to these risk stages.
Tools for identifying, measuring, and mitigating AI harms
Some RAI tools listed below are applicable to both ML and GenAI domains. The RAI tools for ML alone focus on model creation, data preparation, and model training. Tools listed for GenAI focus on model selection, prompt engineering, and model output of generated images, text, audio, or video.
The tables below are organized into tools for both domains as well as tables primarily for ML and GenAI.
Table 1: RAI tools helpful for both ML and GenAI systems
Table 2: RAI tools helpful for ML systems (Azure platform tools)
Table 3: RAI tools helpful for ML systems (open source)
Table 4: RAI tools helpful for GenAI systems (Azure platform tools)
Recommended practices
Based on the experience of the Microsoft organization profiled earlier, the following are recommended best practices for ensuring the effectiveness and safety of AI solutions:
- Track metrics over time: As new models are trained, ensure that regressions have not been introduced. For Machine Learning solutions, create automated pipelines to identify data and model drift.
- Use a defense-in-depth strategy: To mitigate potential RAI harms, leverage platform features when available (such as safety systems and blocklists) in combination with application-level mitigations (such as best practices for UI/UX design and meta-prompt engineering). Consider a combination of tools, such as classifiers and blocklists, for content filtering.
- Conduct use-case reviews: Use-case review is the most impactful RAI exercise for Machine Learning systems because it confirms whether feature selection and results are appropriate. For Machine Learning, select the right use case and model, and then monitor performance regularly to detect drift. See Model monitoring with Azure Machine Learning (preview).
- Assemble a diverse red team: When assembling participants for a red team, include people with different life experiences, demographics, and subject-matter expertise. Keep in mind that red teaming can expose participants to offensive materials, so consider making participation voluntary. See this red-teaming planning guide for more.
By following these recommendations, our Microsoft organization profiled earlier believes that AI systems can be more effective and safer.
Wrapping up
The pace of AI is moving quickly, and RAI-specific tools are continuing to evolve. SDKs and platform extensions will be developed, components will integrate and allow for more customization, and measurements will improve for domains such as finance, insurance, and healthcare. RAI components will become easier to deploy, share, and combine into automated workflows for scalability.
AI systems affect many people directly and indirectly, in positive and negative ways. Responsible AI can help teams to build and deploy AI products in ways that minimize harm. If you have the talent and passion to develop AI solutions, please consider these recommendations and join us in the commitment to innovate responsibly.
Useful links
Here are resources to help:
- A curated collection of links addressing RAI practices: RAI Getting Started (GitHub repo).
- Responsible AI Dashboard and Scorecard (video).
- RAI tools and guidance including Responsible AI Maturity Model, HAX Toolkit, RAI Toolbox, and more (link here: https://aka.ms/rai).
- Frontier Model Forum: What is Red Teaming?
Acknowledgments
Special thanks to Stanley Lin, Kris Bock, Mickey Vorvoreanu, and Kathy Walker for their contributions to and collaboration on this article.
Kate Baroni is on LinkedIn.
