Words and images. Two artificial intelligence experiments help us understand the frontiers of this technology that is a candidate to change the modus operanti of companies, institutions and political subjects. The first is from Google and is called “verse by verse”. It is in English and is a poetry generator. Better said to allow anyone to create poems in the style of 22 famous poets. Even better as it looks like a little game is a plastic example of how some of these machine learning algorithms work. In this case, Google engineers trained the model by feeding the algorithm the complete work of each author. The system “has read” everything but has not understood anything. He has learned the style, the scheme, let’s say the sound, of the poetry of the single poet. In practice, a sentence is written and the system, through a semantic technology module, suggests an option of verses based on the selected poets and the “semantic” meaning of the sentence. The result is weird but effective. Because it is likely. It looks like a poem.
Dall-E 2 uses similar logic but applied to images. The new project of OpenAI, the non-profit organization founded by Elon Musk, and Sam Altman is a sort of automated illustrator. It starts from the caption, from a description of the image you want to obtain, the AI understands the message, looks for the elements and composes the illustration. Text-to-image models are typically trained on large datasets taken directly from the web, which can introduce a variety of problems. Technically DALL-E 2 is an assembly line. A model called Clip (Contrastive Language-Image Pre-training) maps a textual caption to a representation space, after which another map model called Glide statistically relates text and image, that is, studies the hierarchies of the elements. And one this textual encoding along with a visual encoding – an image – that captures the semantic information mapped by the caption. In a nutshell, the system uses a method that allows text and image to be statistically similar, that is, it identifies to which part of the text the relative image corresponds. Finally, a technique called diffusion deals with putting “grounded” what has been learned. The use of this technique in the field of generative artificial intelligence is the real novelty of Dall-E 2. It is a small step, but this is how this discipline is done.
Google has also moved along the same lines, presenting Imagen a few days ago. Here, too, it is an artificial intelligence system capable of creating images starting from a textual description: Imagen, according to the Google Research Brain Team, would be able to offer “an unprecedented degree of photorealism and a deep level of understanding of the language. “. The Google Research Brain Team also points out that Imagen has inherited the “social prejudices and limitations of large language models and may present harmful stereotypes and representations”.