“It is difficult to create a universal ethical model for machines.”
Yeah, they got it. In the field of artificial intelligence there are two disciplines that, although very complex, have gained great strength: on the one hand, vision – recognition and image processing – and, on the other, natural language processing. Very complex algorithms are needed to automate this process that the human brain naturally performs in machines. For example: machine translation, question and answer systems, text summarization systems, grammatical correctors... But we are getting better and better results.
Yeah, but it's not just that. After all, most of the information that is generated in the world is encoded in our language. This big data exists in human language and in digital form. Therefore, this source is the most productive to access any information. In order to make the most of the information we have digitized and to carry out a variety of analyses, natural language processing is necessary. This is how we will benefit from this information.
Yeah, that's right. In order for machines to learn the language of humans, they are trained with texts created by humans: news, novels... These texts look at our behaviors; they are a reflection of our racist and exclusionary society. Therefore, machines learn exclusionary behaviors.
In machine translation, for example, it is seen very clearly in the professions: often the engineer considers them directly as men and the nurse considers them as women. The same was true of religious minorities: a system that extracted synonyms or semantic relations, for example, sometimes linked Islam to terrorism. Google also developed a service to detect objects in photos, which in some cases classified black people as gorillas.
In reality, bias or exclusionary attitudes come from training texts that we need to “clean up”. The fact is that there are many minorities and many ethical conflicts that sometimes intersect. It is not feasible to create millions of purified texts and a universal ethical corpus that has no bias in any social problem. It’s ethically difficult, not just technologically.
Ethical models can be created by paying attention to a specific problem, and we are working on it. But the algorithms we create for machine learning can also amplify the bias inherent in texts. Both factors must be taken into account.
Now, mainly neural language models are used for machines to learn natural language, and in these neural language models they try to distinguish between different degrees of abstraction of language: morphological, syntactic, semantic and pragmatic. The latter is where emotions come in. A scale of six categories has been proposed to detect emotions: angry, happy, sad... What happens is that in spoken language, gestures and intonation must also be taken into account to detect emotions. Therefore, image processing is also required. It requires a neural architecture that combines three disciplines: image recognition, speech technology and language comprehension. That's what they're already investigating.
It's harder to detect the irony. To do this, the system should pay attention to the whole and the global context, what we call “knowledge of the world”. To understand jokes, the machine must first learn some cultural references. Integrating all this knowledge of the world into neural networks is very complex. There's no neural network that makes good jokes, for example. But that will come, too.
For example, the automatic translator Elia that we have developed, when we want to translate a phrase, must take a phrase and announce how it is said in another language. It therefore has two variables: input and output. A word can be introductory, and some neurons have to predict how to translate it. These neurons operate with bits: they receive a number at the input and give a number at the output. In binary language, 0 or 1.
Yes, a formula. You have a series of values in the input, and in order to provide the output value, the input values are combined. But for every input word, you need a bunch of interconnected neurons. And because of the complexity of language, millions of neurons, millions of mathematical formulas, interconnected, are ultimately needed. Like biological neurons. Hence the name of the neural networks. These neural architectures can be organized in layers. In a certain way, morphology is learned in the first layer; in the second, syntax; in the third, semantics
And in order to get better results, we need to adjust these formulas. But how do you adjust a million formulas? It cannot be done by hand. We use optimization algorithms until we explore automatically and find the parameters that give the best results in training.
It has had a long evolution. In the beginning, methodologies based on language rules were used: “If in a sentence this word appears in this position, and has a pronoun next to it...” But the language is complex and there are too many rules, so the results were not good. Then they thought that statistical models would be the most appropriate. The results improved slightly. in 2010, Mikolov, a Google researcher, proposed a technique to better represent the linguistic characteristics of words. And they started using deep neural networks.
But in 2017 and 2018, two innovations were proposed by other Google researchers to better learn the complexity of language. These innovations have consolidated today’s paradigm: the knowledge of a language is concentrated in a huge neural linguistic model that can be adapted to perform specific tasks of language processing. This has led to a boom in language technologies.
Yeah, it's a big limit. We are working on a project with the HiTZ centre and Vicomtech to obtain the largest database that the Basque language has ever had. we've reached a corpus of 400 million words. In English, they drive billions.
It is clear that the most important thing to strengthen the digital vitality of a language is the creation of content. But then we need digital tools to generate interaction in this language: automatic translators, automatic subtitling, news recommendations... To consume content, language technologies are necessary.
Communication has been completely transformed by digitalization. At this time, you can read the tweet written by a Russian journalist in your own language. But for this, languages need technological support. Otherwise, they will be out of the game in the short term. So, yes, it is important that language policies have a clear digital strategy. The Russian, however, does not have a very clear strategy.
In my opinion, it is essential to be sovereign in technologies that are critical to the survival of our language. Above all, to be able to mark strategic lines and not depend on anyone. Google offers some services – translators, search engines, transcription systems, etc. – and I often listen to people in fear: “If Google doesn’t come out, we’re out!” But perhaps the most important thing is not that Google does it, but that we acquire the knowledge to do it ourselves. In fact, to have this knowledge is to be sovereign. This will give us the ability to deal with any situation without relying on large corporations.
Most of the innovations have been made by Google researchers, but they have made algorithms and other resources available to everyone. In the Basque Country we have benefited greatly from these resources. We have three teams: HiTZ Center, Vicomtech and Orai NLP Technologies. So we have the ingredients to achieve technological sovereignty. But strong public funding is needed, because the market will never save the weak.
Buletina
Bidali zure helbide elektronikoa eta jaso asteroko buletina zure sarrera-ontzian