Will artificial intelligence let us talk to the animals?

Much of the excitement in artificial intelligence over the past decade has come from the achievements of neural networks, systems based on analogies to how the human brain processes information through collections of neurons. Deep learning puts data through multiple layers of a neural network, giving rise to the chatbot ChatGPT. However, studies on sperm whales, elephants, and marmosets used early forms of artificial intelligence, namely decision trees and random forests.

A decision tree is a classification algorithm that looks like a flow chart. For example, it might ask if the frequency of the sound being emitted is higher than a certain value. If so, it might ask if the call lasted a certain length of time, and so on, until it determines whether the call matches the acoustic variables it was trained to look for using a human-labeled dataset. A random forest is a collection of many decision trees, each constructed from a randomly selected subset of the data.

Kurt Fristrup, an evolutionary biologist at Colorado State University who wrote the random forest algorithm for the Elephant Project, says tree-based algorithms have several advantages for this type of work. First, they can use less information than is needed to train a neural network—even thousands of hours of recordings of animal sounds is still a relatively small data set. Additionally, because of the way tree-based algorithms break down variables, they are less likely to be discarded by mislabeled or unlabeled data.

Random forests also provide a way to verify that similar calls match: different calls showing the same characteristics should end up in the same “leaf” of a single tree. “Since there are about a thousand of these trees, you can get a pretty fine-grained measure of how similar two calls are by how often they fall on the same leaf,” Fristrup said.

It’s also easier to figure out how the random forest algorithm reaches a specific conclusion than deep learning, which can produce answers that leave scientists scratching their heads about how the model made its decisions. “Deep learning models allow us to obtain, even easily, a variety of results that we wouldn’t be able to obtain any other way,” Fristrup said. But if scientists don’t understand the reasoning behind it, they may not understand “what we would learn if we went into it through the old, less efficient and less computationally intensive path of random forests,” he said.

Nonetheless, many researchers are attracted by the ability of neural networks to generalize from relatively small sets of labeled data and discover patterns by examining large amounts of unlabeled data.

Machine learning expert Olivier Pietquin is the company’s director of artificial intelligence research Planet Species Projectan international team based in Berkeley, California, is using artificial intelligence to decode the communications of animal species. He hopes to exploit the ability of neural networks to generalize from one data set to another, training the model using not only large numbers of sounds from different animals, but also other acoustic data, including human speech and music.

The hope is that computers can use this understanding to deduce some basic characteristics of sounds and then specifically identify the characteristics of animal vocalizations. This is the same way that an image recognition algorithm trained on pictures of faces learns some basic features of pixels that first describe ovals and then eyes. The algorithm can then use this basic knowledge to recognize cat faces, even though human faces make up the majority of its training data.

“We could imagine using speech data and hoping it could be transferred to any other animal that has a vocal tract and vocal cords,” Pitkun said. For example, the whistle produced by a flute may be very similar to a bird whistle, and a computer can make inferences from it.

Models trained in this way can be used to identify which sounds convey a message and which sounds are just noise. However, figuring out what the calls might mean still requires people to observe the animals’ behavior and label what the computers recognize. Recognizing speech, which researchers are currently working toward, is just the first step toward understanding speech. “Understanding is really a difficult step,” says Peter Quinn.