What Is an AI ‘Black Box’?

Black box algorithms make it very difficult to understand how AIs work, but the situation isn’t quite black and white.

The opposite of a black box is sometimes referred to as a glass box. An AI glass box is a system whose algorithms, training data and model are all available for anyone to see. But researchers sometimes characterize aspects of even these as black box.

That’s because researchers don’t fully understand how machine-learning algorithms, particularly deep-learning algorithms, operate. The field of explainable AI is working to develop algorithms that, while not necessarily glass box, can be better understood by humans.

Why AI black boxes matter

In many cases, there is good reason to be wary of black box machine-learning algorithms and models. Suppose a machine-learning model has made a diagnosis about your health. Would you want the model to be black box or glass box? What about the physician prescribing your course of treatment? Perhaps she would like to know how the model arrived at its decision.

What if a machine-learning model that determines whether you qualify for a business loan from a bank turns you down? Wouldn’t you like to know why? If you did, you could more effectively appeal the decision, or change your situation to increase your chances of getting a loan the next time.

Black boxes also have important implications for software system security. For years, many people in the computing field thought that keeping software in a black box would prevent hackers from examining it and therefore it would be secure. This assumption has largely been proved wrong because hackers can reverse-engineer software – that is, build a facsimile by closely observing how a piece of software works – and discover vulnerabilities to exploit.

If software is in a glass box, then software testers and well-intentioned hackers can examine it and inform the creators of weaknesses, thereby minimizing cyberattacks.

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators and Everything We Know About OpenAI’s ChatGPT.

Saurabh Bagchi, Professor of Electrical and Computer Engineering, Purdue University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Services Marketplace – Listings, Bookings & Reviews

Entertainment blogs & Forums