Making AI Trustworthy with Capsa

At Themis AI, we're at the forefront of crafting technologies that underpin trustworthy AI. Our tools are engineered to gauge the robustness of a model, assessing when it can reliably produce accurate outcomes. In this blogpost, we show how Capsa allows you to assess and mitigate risks in your model and do so automatically and at scale. 

In April 2019, the European Commission's High-Level Expert Group on AI unveiled an important document that would shape the ethical and legal landscape of artificial intelligence. The EU "Ethics Guidelines for Trustworthy Artificial Intelligence" is a comprehensive framework aimed at establishing a solid foundation for AI development and deployment within the European Union and beyond. In fact, the upcoming EU AI Act - the World’s first comprehensive legislative framework for AI - is expected to define certain AI systems as "high-risk" and subject them to more stringent regulations and oversight. For those models, ensuring risk assessment and mitigation at scale will be absolutely key to guarantee compliance. 

According to the EU Guidelines, trustworthy AI must adhere to a three-fold criterion:

1. Laws: Trustworthy AI should operate within the bounds of all relevant laws and regulations. This means AI systems must respect and uphold the legal frameworks that govern their use, ensuring compliance with data protection, intellectual property, and other pertinent laws.

2. Ethics: Beyond legality, AI systems must also be ethical. This implies a commitment to upholding a widely recognized and accepted set of moral principles and values (e.g., promoting fairness, respecting human rights, and ensuring transparency and accountability).

3. Robustness: The third pillar of trustworthy AI is robustness. AI systems need to be reliable and accurate across various contexts, situations, and scenarios. They should not falter under unexpected conditions but rather perform consistently and dependably, irrespective of the real-world situations they may encounter. They should also be able to signal uncertainty and low confidence to allow humans to take control when needed. 

In this ever-evolving landscape of AI development, the role of cutting-edge tools becomes increasingly vital. Capsa, built by the engineers at Themis AI, stands at the forefront of this technological revolution. Capsa provides model-agnostic, quantifiable insights into the core components that are indispensable for the development and deployment of robust AI systems.

How does Capsa ensure the robustness of AI and ML systems? Capsa allows engineers and designers to determine the risk of their models by assessing different levels of uncertainty.

Existing approaches for estimating uncertainty in neural networks incur significant computational costs. These methods, most stemming from the domain of Bayesian deep learning, have traditionally depended on repeatedly running or sampling a neural network to gauge its level of confidence. This process consumes both time and memory, making it hardly suitable for high-risk, rapid decision-making scenarios. 

In contrast, Capsa stands out as a model-agnostic, easily implementable platform tailored for uncertainty evaluation. Our Python library encompasses a range of methods cleverly packaged as easy-to-use "wrappers," making them applicable and scalable across any model. 

At any stage of model development (e.g., pre-training, post-training, fine-tuning, during training, during inference), Capsa automatically converts models into an uncertainty-aware version, able to provide uncertainty estimates alongside every output across different dimensions. More specifically, Capsa allows your model to autonomously evaluate the following risk factors:

1. Vacuitic Uncertainty ("representation bias”): when the model has been trained and tested on insufficient, incomplete data vis-a-vis its area of deployment, it will lack the necessary knowledge to deliver reliable results. In other words, a biased training and testing data set leads to high uncertainty if deployed to make assessments on inputs that are not well represented. This risk factor is particularly difficult to spot when it affects latent features. Latent features are those variables that are not directly and explicitly represented. In machine learning, latent feature imbalance can lead to biases, even in balanced classes. 

2. Aleatoric (data) Uncertainty: this type of uncertainty comes from noisy or ambiguous data points that do not follow the expected pattern in the training set. Here is a representation of this type of uncertainty:

3. Epistemic (model) Uncertainty: this type of uncertainty arises from regions where the model lacks sufficient training data. In such regions, the model cannot offer reliable predictions due to the absence of similar training data, and therefore it will not be confident in its predictions. 

To detect these uncertainties, Capsa provides a variety of broad classes of wrappers (e.g., Sculpt, Vote, Neo, etc.). Each individual wrapper encompasses a range and class of uncertainty estimation algorithms that you can select and tune to your specific needs. Our wrapper technology ensures that the AI model continues to produce uncertainty estimates with minimal effort. Here is an illustration of what Capsa can do:

In conclusion, by measuring uncertainty, we can calculate a model's robustness, namely its ability to perform reliably and accurately in diverse scenarios. Capsa empowers organizations and AI developers to take a significant step toward the ultimate goal of ensuring that AI models can be truly trusted. In alignment with the guidelines of the EU AI Act, Capsa emerges as a tangible, turnkey solution addressing AI robustness concerns. It pioneers the industry's proactive response to the pressing demands for reliable and trustworthy AI.