Robust and Trustworthy Deep Learning (Part 1)

‍

‍
In April 2023, Sadhana Lolla, machine learning scientist at Themis AI, delivered a lecture centered around the theme of "Robust and Trustworthy Deep Learning." During her presentation, she unveiled the cutting-edge technological advancements in progress at Themis AI. Over the next few weeks, we will publish a sequence of blog posts that will showcase the main points of her talk. For more information on the lecture see MIT Introduction to Deep Learning.

The Unprecedented Opportunities and Challenge of Artificial Intelligence 

Over the past few years, we witnessed the tremendous growth of artificial intelligence in safety-critical domains such as robotics, autonomous vehicles, and healthcare. This surge in AI integration has revolutionized the way tasks are performed and decisions are made, bringing forth both unprecedented opportunities and challenges. In the field of robotics, AI has enabled the development of advanced autonomous systems that can navigate complex environments, assist in hazardous operations, and enhance manufacturing efficiency. These robots are now capable of executing intricate tasks with a level of precision and adaptability previously unattainable. In medicine and healthcare, AI's impact has been profound. Diagnostic tools powered by AI can analyze medical images, such as X-rays and MRIs, with remarkable accuracy, aiding clinicians in early disease detection and treatment planning. Additionally, AI is accelerating drug discovery by analyzing vast amounts of biological data to identify potential drug candidates. This speeds up the research and development process and could lead to the discovery of new treatments for various diseases. Finally, self-driving cars and drones are being developed with AI algorithms that can perceive their environment, make real-time, split-second decisions, and potentially navigate without human intervention.

These tools and innovations are revolutionary and promise to considerably improve our lives. However, there is a gap between innovation and its practical deployment in real-life settings, particularly in safety-critical domains. The popular press is full of reports that extensively cover the shortcomings and limitations of artificial intelligence. These stories underscore the instances where AI systems have fallen short of expectations or encountered unforeseen challenges. They shed light on the complexities that arise when cutting-edge technology meets real-world complexities, often showcasing scenarios where AI's performance has failed catastrophically,  including instances where AI algorithms have made unfair, biased decisions, misinterpreted medical data, or struggled in unanticipated scenarios.

These failures can and should be tackled as soon as possible if we want AI to develop and flourish. We at Themis AI aim to solve these issues by creating a safe and trustworthy AI technology to be applied and deployed in the real world. 

Our approach at Themis AI is grounded in the belief that bias and uncertainty are the two big challenges to developing safe and robust deep learning systems. In this post we will discuss the origin of bias in machine learning and how bias can generate unfairness. In the next blog post, we will tackle the problem of uncertainty in machine learning. Finally, in our last post of the series, we will explain how Themis is innovating in these areas to finally bring innovation to industries worldwide.

Bias as an Ethical Risk Factor

Bias is an ethical risk factor in machine learning because it may lead to results that are unfair and discriminatory such as denying loans based on applicants’ race or systematically failing to recognize women’s faces. Therefore, addressing these biases is crucial for creating robust and trustworthy AI systems.

Bias in artificial intelligence can occur at various stages of the AI life cycle. Bias occurs primarily in the data we use to train, test and evaluate our models: 

• Sampling bias happens when we over or under sample from some regions of our data distribution. This may be due to the fact that there are gaps in the data we have access to and hence it is difficult to build data sets that represent all relevant classes equally. 
• Selection bias emerges where the training data distribution fails to match real-world distribution. In this case, when training data sets are built, data is not selected in a way that is representative of how the model will be used and applied in the real world.  
• Evaluation bias occurs when the evaluation metrics and data set adopted to evaluate a model are not representative. This bias prevents us from getting an accurate picture of the performance of the model in the real world, and thus avoiding failure.  
• Distribution shift is a type of bias that occurs when a model is based on data that is obsolete and the model is not updated. 

Eventually these biases may lead to inaccurate and unfair model deployments. An example of this bias is seen in commercial facial detection systems, where we witness significant accuracy gaps between different demographics. If these facial recognition  tools are adopted in high-risk scenarios (e.g., the judicial system), they will produce inaccurate outputs, likely harming specific demographic groups. 

To mitigate class imbalance (i.e. classes being unevenly represented), three approaches are commonly used:

1. Sample re-weighting: Samples are selected from the data set inversely proportional to the incidence of the class. Hence, underrepresented classes are sampled more frequently to ensure the model sees both classes equally.
2. Loss re-weighting: The contribution of each sample to the total loss function is adjusted. Samples from underrepresented classes contribute more to the loss function, encouraging the model to focus on learning those classes better.
3. Batch selection: Randomly choosing data points from each class to create batches with an equal number of data points per class, ensuring all classes are equally represented during training.

While these techniques can help address class imbalance, they may not completely solve all forms of bias in the data. Other biases might still be present and need to be carefully considered and addressed in machine learning tasks. For example, in machine learning, latent feature imbalance can lead to biases, even in balanced classes. Latent features are those variables that are not directly and explicitly represented. Feature imbalance is not easily detected by standard bias mitigation techniques. 

The approach we favor is to automatically learn latent features and use this representation to de-bias the model. Variational autoencoders (VAEs) offer a way to learn latent features probabilistically. They sample from a learned latent space and then decode the latent vectors back into the original input space, minimizing the reconstruction loss during training. The goal is to have similar input samples map to close points in the latent space and dissimilar samples map to distant points, enabling us to identify biased features and mitigate them more effectively.

The Debiasing Variational Autoencoder (DB-VAE) is an algorithm that automatically utilizes the latent features learned by a Variational Autoencoder (VAE) to address bias in data. The VAE is trained to learn latent representations that are important for classification without explicit guidance. The algorithm then calculates a probability distribution based on the features of each data item in the latent space.  

Using this distribution, DB-VAE can oversample from sparser areas of the data set and undersample from denser areas. By doing so, it addresses biases in the data set, such as homogeneous skin colors in dense regions, and diverse skin colors and illuminations in sparse regions.

The DB-VAE algorithm approximates the latent space using histograms for individual latent variables to address bias in data. By discretizing the continuous distribution, it calculates probabilities and adjusts the probability of sampling data points based on the inverse of the joint approximated distribution. Then parameter Alpha controls the level of debiasing, thus allowing adaptive resampling during training.

This debiasing process enables fair and unbiased model training, and it has been demonstrated to successfully debias commercial facial detection algorithms, making it a foundational aspect of Themis's work. This debiasing algorithm can also be applied to various machine learning tasks, including autonomous driving, large language models, and healthcare recommendation algorithms, where bias is widespread and can be mitigated using the same approach.