A year after the Jeopardy match, Ferrucci left to form Elemental Cognition. It has so far been funded by Bridgewater Associates, a hedge fund created by Ray Dalio that manages roughly $160 billion, and three other parties. Elemental Cognition operates on Bridgewater’s campus, in lush woodland overlooking a lake in Westport, Connecticut.

Not long after Watson’s triumph, AI was transformed. Deep learning, a means of teaching computers to recognize faces, transcribe speech, and do other things by feeding them large amounts of data, emerged as a powerful tool, and it has been applied in ever-more ways.

Over the past couple of years, deep learning has produced striking progress in language understanding. Feeding a particular kind of artificial neural network large amounts of text can produce a model capable of answering questions or generating text with surprising coherence. Teams at Google, Baidu, Microsoft, and OpenAI have built ever-larger and more complex models that are progressively better at handling language.

And yet, these models are still bedeviled by a lack of common sense. For instance, Ferrucci’s team gave an advanced language model the story involving Ferdanando and Zoey, and asked it to complete the sentence “Zoey moves her plant to a sunny window. Soon….”. Failing to grasp the notion that plants thrive in sunlight, it generated a series of bizarre endings based purely on statistical pattern matching: “she finds something, not pleasant,” “fertilizer is visible in the window,” and “another plant is missing from the bedroom.”

CLARA aims to go further by combining deep learning techniques with more old-fashioned ways of building knowledge into machines, through explicit logical rules, like the fact that plants have leaves and need light. It uses a statistical method to recognize concepts like nouns and verbs in sentences. It also has a few pieces of what’s known as “core knowledge,” like the fact that events happen in time and cause other things to happen.

Knowledge about specific subjects is crowdsourced from Mechanical Turkers and then built into CLARA. This might include, for example, that light causes plants to thrive and windows allow light in. In contrast, a deep learning model fed the right data might be able to answer questions about botany correctly, but it might not.

It would take a long time to hand-craft every possible piece of common sense knowledge into the system, as previous efforts to build knowledge engines by hand have sadly demonstrated. So CLARA combines the facts it’s given with deep learning language models to generate its own common sense. In the plant story, for example, this might allow CLARA to conclude for itself that being in a window helps make plants green.

CLARA also gathers common sense by interacting with users. And if it comes across a contradiction, it can ask which statement is most often true.

“It’s a very challenging enterprise, but I think it’s an important vision and goal,” says Roger Levy, a professor at MIT who works at the intersection of AI, language, and cognitive science. “Language is not just a set of statistical associations and patterns—it also connects with meaning and reasoning, and our common sense understanding of the world.”

It’s hard to say how much progress Ferrucci has made towards giving AI common sense, in part because Elemental Cognition is unusually secretive. It recently published a paper arguing that most efforts at machine understanding fall short, and should be replaced by ones that ask deeper questions about the meaning of text. But it hasn’t published details of its system, or released any code.

Keep Reading

illustration of a head

Scaling such a complex system beyond simple stories and basic examples will likely prove tricky. Ferrucci seems to be looking for a company with deep pockets and a large number of users to help. If people could be persuaded to help a search engine or a personal assistant build common sense knowledge, that could accelerate the process. Another possibility Ferrucci suggests is a program that asks students questions about a piece of text they have read, to both check they understand it and build its own knowledge base.

The psychological oasis.