Where does technology end and where does art begin? He seems to want to answer this question all the work of Daito Manabe, interaction designer, programmer, engineer and deejay. Japanese, 46, is one of the best known media artists in the world and his work moves on the borders between analog and digital, between real and virtual. He will also happen tonight, in Rome, where he will present Dissonant Imaginarywithin the Eni Gazometro area in occasion of Videocittàthe vision festival conceived by Francesco Rutelli and directed by Francesco Dobrovich.
The theme of the event, the technological and ecological transition, is fully represented in the work of Manabe, who investigates the relationship between sound and images. It does this by resorting to science: Dissonant Imaginary arises from the analysis of the activity of the human visual cortex in reaction to sounds. To obtain the data, several subjects, immersed in the darkness, listened to specific sounds and abstract music. The evolutions resulting from the different sound stimulations were then decoded by means of functional magnetic resonance imaging (fMRI). “It is not a simple electroencephalogram – Manabe explains – but a much more complex technology that generates images ”, which are then processed through an algorithm developed together with Yukiyasu Kamitani, expert in computational neuroscience and professor of computer science at Kyoto University.
Brian Eno: “With NFTs even artists can become little capitalist assholes”
by Bruno Ruffilli
The performance two about forty minutes and is divided into two parts: the first shows the data of the 3D scan of the brain, the brain activity and the brain decoding process. “For this project I used over 25 hours of fMRI data and I have also obtained permission to use the data of other parties. ”, Explains Manabe. The second part focuses on the process of reconstructing the images that have formed in the mind while listening to sounds or music, and in particular in some areas of the brain called the high visual cortex, low visual cortex and Brodmann area 39. And here the game is no longer machine learning, but human creativity: “This is an arbitrary interpretation. I have to compose music for listening in the fMRI device and imagine what I hear from the music when I listen to it. I usually use the code to convert music to images. but for this project I only need to imagine ”.
A low organ note will perhaps be dark, a trill will have bright colors, as well as a hiss, the human voice will vary among a thousand shades. But assigning a color and a shape to a sound is an artistic operation such as that of visually representing the stars of the James Webb telescope: “The impression that the major chords are bright and the minor chords are dark is determined by the cultural background,” observes Manabe. “Believe it or not, some people think of a dark dark color when they hear a major chord.”
Therefore, for the Japanese artist there is no absolute language of sounds, because the brain activity of a human being in reaction to music is influenced by external and personal factors. This finding makes it though more difficult is the realization of a project like Neuralink by Elon Musk, which explores the possibility of connecting our brains to the Internet without the intermediation of a computer and a keyboard, and moves into the same territory explored by Manabe. “It is one of the most interesting projects of the moment. I’m not sure what will be possible with their technology, but I believe the initial experiments will consist of convert the movement of the hands in text or graphics. I wish I could insert a Neuralink device into my brain and make an art project ”.
Dissonant Imagery it has already made the rounds of electronic and avant-garde music festivals, starting with Sonar in Barcelona. He arrives after experiments with drones and dance, after the videos with Björk and OkGo, after the collaboration with Iannis Xenakis, after the video games and commercials. For Manabe an idea comes true: “For years I have dreamed of being able to extract images from my brain when I listen to music. The current technology is not perfect, but I am sure that one day the decoding algorithms will be even more precise ”. The effect, meanwhile, is alienating, halfway between Quark and a futurist nightclub: “I don’t need perfect technology for my art. What I’m looking for in technology is inspiration, ”she explains.
What if inspiration were replaced by artificial intelligence? “I’m sure gods will be invented tools to compose music easily, but this will never cancel the effort of man or the contribution of the composer. The more we go forward in discoveries and inventions, the more we realize how great human beings and nature are compared to artificial intelligence “.
Digital art
Madonna’s latest provocation: 3 NFTs together with Beeple
by Bruno Ruffilli
