Where is humanity’s final frontier?

The last frontier of mankind! AI cannot understand comics, creating an iron job for cartoonists.

A large part of the job of AI engineers is to "lead the way" for AI, and then tell humans that this and that job of yours will be replaced by AI.

Humanity has always seemed to be in a weak position with no power to fight back. Maybe the future will be like what some pessimists imagine. We have AI drivers, AI salesmen, and AI poets, but humans He himself became a poor beggar.

To this day, for the first time, they discovered that there is a job that AI will not be able to surpass humans for a period of time, and this victory belongs to the second dimension - the job that will not be replaced by AI is cartoonist.

If you can’t even understand the comics, why are you talking about destroying the world?

The reason why cartoonists will not be replaced by AI is simple, because a professor at the University of Maryland conducted a study and ultimately found that AI cannot understand comics at all.

The above is a very simple four-frame cartoon, which is very easy for humans to understand: the kitten was thinking about creative materials, then discovered the puppy, and asked the puppy to tell a joke, and the puppy said "You are beautiful" caused the kitten to get very angry.

Actually, in the last picture, the puppy is not in the shot, and "You are beautiful" was originally a compliment. It needs to be connected with the "joke" in the previous picture to explain the fact that the kitten emotions.

It is simply too difficult for AI to understand the information presented outside the screen.

In an experiment at the University of Maryland, researchers built a data set consisting of 1.2 million comic frames, and extracted the text in each frame. Using the LSTM model, they hope that AI can Get a coherent understanding of comics.

A lot has been introduced before about LSTM (Long Short-Term Memory Network). The characteristic of this model is that it incorporates the concept of memory and can process and predict longer elements of time series. Although it performs well in long texts, machine translation, etc., LSTM is completely defeated when it comes to reading comics.

After extensive training, the researchers gave the AI ??a set of comics it had never seen before and asked the AI ??to understand and predict the text information or picture content in the next picture. As a result, the AI's performance was a mess. The accuracy of human predictions can usually reach 80%.

Visual storytelling? Why bother with artificial intelligence

Strictly speaking, comics are "visual narratives" - hiding information in images. It is also a visual narrative. It is much easier for AI to understand movies than comics. The protagonists of movies are people, and people’s faces all look the same. It is very simple to train AI to read facial expressions and recognize emotions. , not to mention that the movie will have a detailed script.

But the biggest feature of comics is that they are not visually coherent. Just like the four-panel comic above, the puppy is still in the picture in the third picture, but not in the fourth picture. Humans can quickly understand that a puppy named Calm Dog leaves calmly after just one word. But for AI, it is really difficult for strong artificial intelligence to read this kind of information beyond pictures and text.

Secondly, the painting and narrative styles of different comics are very different, which is also a difficulty for AI training. In a simple four-frame comic, the scene in each frame is the same, but in other comics, one frame may be a fighting scene, and the next frame may be an angry face. The AI ??can understand four-frame comics, but it's still confusing to read comics that have a sense of camera switching. As for the style of painting, different cartoonists have very different ways of depicting human faces. If it is used to understand by AI, it will be more difficult.

Another point is that visual narrative is based on the two concepts of "logic" and "common sense". For example, the kitten says to tell a joke, and the puppy says, "You are so beautiful." To understand this plot, you need the basic logic of "You said I'm beautiful, it's a joke = you said I'm ugly." Another example is the common mouse stalk in Doraemon, which also requires the basic common sense that "cats are usually not afraid of mice." These things are very simple for humans, but AI does not have these common sense and logical concepts, and we cannot instill these concepts into the brain of AI like compiling an encyclopedia.

The great player in the Go world died in one episode of The Legend of Zhen Huan

In fact, combining AI’s victory in the field of Go and its failure in comics, we can see that AI is completely Performance in the domain of information is completely different from performance in the domain of incomplete information.

Perfect information is originally a category in economics, which means that participants can understand all the information in the entire market. Here we can look at the dataset as a job. In the work of Go, all information can be summarized into data sets: the rules of the game and the playing methods of each move. But in comics, the most we can do is make detailed annotations on the screen and extract all the text information. Logical relationships, common sense, and other things that can only be understood but cannot be expressed in words cannot be provided to AI.

The worst thing AI does is read between the lines.

By analogy, AI plays a role in dramas, idioms, the secret chess of Four Kingdoms (a military flag game that involves deception), including falling in love, etc. Everything is full of incomplete information, deception and counter-deception, and interpretation. Imagery, common sense, and logic games all don't do well.

From this point of view, AI is a bit like the early Sophon in "The Three-Body Problem". It cannot hide its own thoughts, nor can it understand the concepts of concealment and deception. Therefore, we really don’t have to be afraid of the victory of AI. It will be the least popular colleague in the office and the passerby who died in the harem episode. The excellence of a certain ability cannot make up for its imperfect information. Short board. What's more, things like imagery, analogy, irony, and metaphor are the tools humans are best at.

I believe that the best future world will look like when humans and AI each perform their duties and do what they are best at. In things like visual storytelling, which AI is particularly bad at, they can still provide a lot of help to humans.

For example, using generative adversarial neural networks to create character images, using supervised learning + convolutional networks to color line drawings, and even developing a comic reading app that automatically enlarges text. These are not fantasies, but reality that is happening right now. When these complicated mechanical tasks are taken over by AI, we can devote more to what we are good at: telling more stories using the incomplete information environment to keep the world as interesting as it should be.