“Humans haven’t necessarily made the best choices for our world.” – Interview with Peter Mantello on Emotionalized Artificial Intelligence

In February 2024, a collaboration with colleagues from Ritsumeikan Asia Pacific University on the topic of Emotionalized Artificial Intelligence (EAI) started. Professor Peter Mantello (Ritsumeikan Asia Pacific University) leads a 3-year project funded by the Japan Society for the Promotion of Science, on which c:o/re is a partner, that will compare attitudes in Japan and in Germany on EAI in the workspace. This explorative pathway contributes to the c:o/re outlook on Varieties of Science. You can find the whole project description on our website here.
In the interview below, Peter Mantello explains what EAI is, how the project will consider AI ethics and why the comparison of German and Japanese workplaces is particularly insightful. We thank him for this interview and look forward to working together.

Peter Mantello
c:o/re short-term
Senior Fellow (11-17/2/2024)
Peter Mantello is an artist, filmmaker and Professor of Media Studies at Ritsumeikan Asia Pacific University in Japan. Since 2010, he has been a principal investigator on various research projects examining the intersection between emerging media technologies, social media artifacts, artificially intelligent agents, hyperconsumerism and conflict.
What is Emotionalized Artificial Intelligence (EAI)? What does this formulation entail differently than ‘Emotional’ AI?
Emotional AI is the commercial moniker of a sub-branch in computer science known as affective computing. The technology is designed to read, monitor, and evaluate a person’s subjective state. It does this by measuring heart rate, respiration rate, skin perspiration levels, blood pressure, eye movement, facial micro-expressions, gait, and word choice. It involves a range of hardware and software. This includes cameras, biometric sensors and actuators, big data, large language models, natural language processing, voice tone analytics, machine learning, and neural networks. Emotionalized AI appears in two distinct forms: embodied (care/nursing robots, smart toys) and disembodied (chatbots, smartphone apps, wearables, and algorithmically coded spaces).
I think the term ’emotionalized’ AI better encompasses the ability of AI not to just read, and recognize human emotion but also to simulate and respond in an empathic manner. Examples of this can be found in therapy robots, chatbots, smart toys, and holograms. EAI in allows these forms of AI to communicate in a human-like manner.
What is emotionalized AI used for and for what is it further developed?
Currently, emotionalized AI can be found in automobiles, smart toys, healthcare (therapy robots/ doctor-patients conversational AI) automated management systems in the workplace, advertising billboards, kiosks and menus, home assistants, social media platforms, security systems, wellness apps and videogames.
What forms of ethical work practices and governance do you have in mind? Are there concrete examples?
There are a range of moral and ethical issues that encompass AI. Many of these are similar to conventional usages of AI, such as concerns about data collection, data management, data ownership, algorithmic bias, privacy, agency, and autonomy. But what is specific about emotionalized AI is that the technology pierces through the corporeal exterior of a person into the private and intimate recesses of their subjective state. Moreover, because the technology targets non-conscious data extracted from a person’s body, they may not be aware or consent to the monitoring.
Where do you see the importance of cultural diversity in AI ethics?
Well, it raises important issues confronting the technology’s legitimacy. First, the emotionalized AI industry is predominantly based in the West, yet the products are exported to many world regions. Not only are the data sets used to train the algorithms limited to primarily Westerners, but they also rely largely on famed American sociologist Paul Eckman’s ‘universality of emotions theory’ that suggests there are six basic emotions and are expressed in the same manner by all cultures. This is untrue. But thanks to a growing number of critics who have challenged the reliability/credibility of face analytics, Eckman’s theory has been discredited. However, this has not stopped many companies from designing their technologies on Eckman’s debunked templates. Second, empathetic surveillance in certain institutional settings (school, office, factory) could lead to emotional policing, where to be ‘normal’ or ‘productive’ will require people to be always ‘authentic’, ‘positive’, and ‘happy’. I’m thinking of possible dystopian Black Mirror scenarios, like in the episode known as “Nosedive”.
Third, exactly what kind of values do we want AI to have – Confucian, Buddhist, Western Liberal?
Do you expect to find significant differences between the Japanese and German workplace?
Well, it’s important to understand the multiple definitions of the workplace. Workplaces include commercial vehicles, ridesharing, remote workspaces, hospitals, restaurants, and public spaces, not just brick-and-mortar white-collar offices.
Japan and Germany share common work culture features, but each society also has historically different attitudes to human resource management relationships, what constitutes a ‘good’ worker, loyalty, corporate responsibility to workers, worker rights and unions, and precarity. The two cultures also differ in how they express their emotions, raising questions about the imposition of US and European emotion analytics in the Japanese context.

How will the research proceed?
The first stage of the research will be to map the ecology of emotion analytics companies in the West and East. This includes visits to trade show exhibits, technology fairs, start-up meetings, etc. The second stage will be interviews. The third stage will include a series of design fiction workshops targeted to key stakeholders. Throughout all of these stages, we will be holding workshops in Germany and Tokyo, inviting a interdisciplinary mix of scholars, practitioners, civil liberties advocates and industry people.
What do you think will be the most important impact of this project?
We are at a critical junction point in defining and deciding how we want to live with artificial intelligence. Certainly, everyone talks about human-centric AI but I don’t know what that means. Or if that’s the best way forward. Humans haven’t necessarily made the best choices for our world. If we try to make AI in our own image, it might not turn out right. What I hope this project brings are philosophical insights that will better inform the values we need to encode into AI, so it serves the best interests of everyone, especially, those who will be most vulnerable to its influence.
What inspired you to collaborate with c:o/re?
My inspiration to collaborate with c:o/re stems from my growing interest in phenomenological aspects of human-machine relations. For the past three years, my research has focused primarily on empirical studies of AI. The insights gained from this were very satisfying, albeit they also opened the door to larger, more complex questions that could only be examined from a more theoretical and philosophical perspective. After a chance meeting with Alin Olteanu at a semiotic conference, I was invited to attend a c:o/re workshop on software in 2023. I realized then that KHK’s interdisciplinary and international environment would be a perfect place for an international collaborative research project.
Lecture Series Summer 2024: Lifelikeness

Due to the great interest, the lecture series of the summer semester 2024 will once again be held on the topic of “Lifelikeness”.
Various speakers, including the sociologist Hannah Landecker (University of California, Los Angeles) and the historian of science Friedrich Steinle (TU Berlin), will be guests at the KHK c:o/re and shed light on “Lifelikeness” from different disciplinary perspectives.
Please find an overview of the dates and speakers in the program.
The lectures will take place from May 8 to July 3, 2024 every second Wednesday from 5 to 6.30 pm in presence and online.
An exception is the lecture by Hannah Landecker, which she will give as part of the interdisciplinary conference “Politics of the Machines” on Tuesday, April 23, 2024 from 5:30 to 7 p.m. in the Super C- Generali Saal.
If you would like to attend the lectures, please send a short email to events@khk.rwth-aachen.de.

Program: PoM Conference in Aachen

Programmable biosensors, life-like robotics and other artificial models – the present and the future are dominated by new phenomena in the life sciences. How can the challenges, opportunities and uncertainties associated with these advances be addressed?
The transdisciplinary conference series “PoM – Politics of the Machines”, which will take place from April 22 to 25, 2024 at the Super C at RWTH Aachen University (Templergraben 57, 52062 Aachen) under the title “Lifelikeness & beyond”, will explore this question. At the interface of science and art, the conference aims to stimulate reflection on the comprehensive connections that shape our perception of the world.
International researchers and practitioners from various fields of science, technology and art will come together to discuss socio-cultural concepts of the future, the interaction between human and machine and ideas of the living and non-living in different formats.
The main program from 22 to 25 April will take place in Aachen in the Super C of the RWTH Aachen University and in the LOGOI Institute.
Super C: Templergraben 57, 52062 Aachen
LOGOI Institute: Jakobstraße 25a, 52064 Aachen
You can register with this form.
Further information on the schedule can be found in this program.
You can find a longer version with all abstracts in this program.

On Thursday, April 25, Dr. Jürgen Kippenhan will give a talk on “Artificial intelligence and the sensory structures of human speech, thought and action” as part of the “POM Conference” at LOGOI, Jakobstraße 25a, 52064 Aachen.


As part of the conference, the choreographic centre PACT Zollverein in Essen will realize the accompanying programme ‘life.like’ on 26 and 27 April 2024, which consists of six artistic positions in the form of performance, installation, discourse and sound.

‘Lifelikeness & beyond’ is the fourth edition of the “Politics of the Machines” conference series, founded by Laura Beloff (Aalto University Helsinki) and Morten Søndergaard (Aalborg University Denmark) and organized in collaboration with RWTH Aachen University, LOGOI Institute for Philosophy and Discourse and PACT Zollverein in Essen.
Theodore von Kármán Fellowship to Professor Reiner Grundman

Reiner Grundmann, Professor of Science and Technology Studies (STS) at the University of Nottingham, has been awarded the Theodore von Kármán Fellowship by RWTH Aachen University.

Professor Holger Hoos (Chair for Methodology of Artificial Intelligence), Professor Frank Piller (Chair of the Institute for Technology and Innovation Management) and KHK c:o/re Director Professor Stefan Böschen jointly applied for the fellowship. The fellowship thus strengthens interdisciplinary cooperation in the field of artificial intelligence (AI).
The fellowship enables Reiner Grundmann to spend seven weeks at the Käte Hamburger Kolleg: Cultures of Research (c:o/re) at RWTH Aachen University, where he will work on a project entitled “Communication Unbound: The Discourse of Artificial Intelligence” from April to May 2024. It investigates the discourse on forms of AI based on large language models and the challenges they pose to society. His current work focuses on the relation between knowledge and decision making, with a special interest in the role and nature of expertise in contemporary societies. To present the outcomes of this fellowship, Reiner Grundmann will give a public university lecture on May 15, 2024, 5-6.30 pm, at KHK c:o/re, Theaterstr. 75.
RWTH Kármán-Fellowships are funded by the Federal Ministry of Education and Research
(BMBF) and the Ministry of Culture and Science of the German State of North Rhine
Westphalia (MKW) under the Excellence Strategy of the Federal Government and the
Länder.


How a well-crafted brain model has influenced research

HANS EKKEHARD PLESSER
On the tenth anniversary of the publication of a model of a cortical microcircuit by Potjans and Diesmann (2014), 14 experts in computational neuroscience and neuromorphic computing will meet at the KHK c:o/re, at RWTH Aachen, to discuss their experiences in working with this model.
This event is a unique opportunity to gain insights on the effect that the Potjans-Diesmann model is having on computational neuroscience as a discipline. In light of the success of the model, the participants will reflect on why active model sharing and re-use is still not common practice in computational neuroscience.

Hans Ekkehard Plesser
Hans Ekkehard Plesser is an Associate Professor at the Norwegian University of Life Sciences.
His work focuses on simulation technology for large-scale neuronal network simulations and reproducibility of research in computational neuroscience.
Computational neuroscience, the field dedicated to understanding brain function through modelling, is dominated by small models of parts of the brain designed to explain the results of a small set of experiments, for example animal behavior in a particular task. Such ad hoc models often set aside much knowledge about details of connection structures in brain circuits. This limits the explanatory power of these models. Furthermore, these models are often implemented in low-level programming languages such as C++, Matlab, or Python and are shared as collections of source code files. This makes it difficult for other scientists to re-use these models, because they will need to inspect low-level code to verify what the code actually does. One might thus say that these models are formally, but not practically FAIR (findable, accessible, interoperable, reusable).
The model of the cortical microcircuit published by Potjans and Diesmann (2014) pioneered a new approach, quite different to previous practice. For good reasons, this new approach made the model remarkably popular in computational neuroscience. Based on a well-documented analysis of existing anatomical and physiological data, Potjans and Diesmann provided a bottom-up crafted model of the neuronal network found under one square millimeter of cortical surface. Their paper describes the literature, data analysis and data modeling on which the model is based, and provides a precise definition of the model.

In addition to their theoretical definition of the model, they created an implementation which is executable on the domain-specific high-level simulation tool NEST. They complemented this implementation with thorough documentation on how to work with the model. They even created an additional implementation of the model in the PyNN language, so that the model can be executed automatically on a wide range of neuronal simulation tools, including on neuromorphic hardware systems.
These efforts have led to a wide uptake of the model in the scientific community. Hundreds of scientific publications have cited the Potjans-Diesmann model. Several groups in computational neuroscience have integrated it in their own modelling efforts. The model has also had a key role in driving innovation in neuromorphic and GPU-based simulators by providing a scientifically relevant standard benchmark for correctness and performance of simulators.
References
Potjans, T. C., Diesmann, M. 2014. The cell-type specific cortical microcircuit: Relating structure and activity in a full-scale spiking network model. Cerebral Cortex 24(3): 785-806.
van Albada S. J., Rowley A. G., Senk, J., Hopkins, M., Schmidt, M., Stokes, A.B., Lester, D.R., Diesmann, M., and Furber S.B. 2018. Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model. Frontiers in Neuroscience. 12:291.
Objects of Research: Sarah R. Davies


For today’s edition of the “Objects of Research” series, c:o/re Senior Fellow Sarah R. Davies gives an insight into her desk set up. As a professor of Technosciences, Materiality, and Digital Cultures, her work focuses on the intersections between science, technology, and society, with a particular focus on digital tools and spaces.
“I guess many academics would share some varient of this image: a careful arrangement of computer equipment, coffee, notepads, pens, and the other detritus that lives on (my) desk.
For me it’s important that the technical equipment is shown in conjunction with the paper notebook and pens. I’m fussy about all of these things – it’s distracting when my computer set-up isn’t what I’m used to, and I need to use very specific pens from a particular store – but ultimately my thinking lives in the interactions between them.
My colleagues and I are working on an autoethnographic study of knowledge production, and notice that (our) creative research work often emerges as we move notes and ideas from paper to computer (and back again).”

Would you like to find out more about our Objects of Research series at c:o/re? Then take a look at the pictures by Benjamin Peters, Andoni Ibarra, Hadeel Naeem, Alin Olteanu, Hans Ekkehard Plesser, Ana María Guzmán, Andrei Korbut, Erica Onnis, Phillip H. Roth, Bart Penders and Dawid Kasprowicz.
Objects of Research: Dawid Kasprowicz


Presenting the next chapter of the “Objects of Research” series, c:o/re Postdoc and Fellow Program Coordinator, Dr. Dawid Kasprowicz, whose main research fields include theory and history of embodiment, phenomenology, human-robot-Interaction, philosophy of computer simulation, joins us with a picture of his fundamental object for his research practice:
“This is a notebook my Mom gave me. She had it as a kind of leftover from a shopping tour and she thought that it might be of use for my work. And of course, she was right. And as you know, research always starts with a good question that attracts attention.”

Would you like to find out more about our Objects of Research series at c:o/re? Then take a look at the pictures by Benjamin Peters, Andoni Ibarra, Hadeel Naeem, Alin Olteanu, Hans Ekkehard Plesser, Ana María Guzmán, Andrei Korbut, Erica Onnis, Phillip H. Roth and Bart Penders.
Get to know our Fellows: Guillaume Yon

Get to know our current fellows and gain an impression of their research.
In a new series of short videos, we asked them to introduce themselves, talk about their work at c:o/re, the impact of their research on society and give book recommendations.
You can now watch the sixth video of Dr. Guillaume Yon, historian of economics, on our YouTube channel:
Check out our media section or our YouTube channel to have a look at the other videos.
Workshop “Art, Science, the Public”

On 16 February and 17 February 2024, the workshop “Art, Science, the Public” took place at the KHK c:o/re in cooperation with the project “Computer Signals: Art and Biology in the Age of Digital Experimentation“, a research collaboration between artists, biologists and humanities scholars, in which c:o/re director Gabriele Gramelsberger has been involved since a long time.
Together with representatives and colleagues from the research group “Computer Signals”, PACT Zollverein and RWTH Knowledge Hub, different formats and practices of science communication, in particular those that experiment with artistic forms, were discussed. The aim of the workshop was to exchange ideas and best practice examples on the interface between science and art and the associated communication challenges.


A special highlight was the sound work by Valentina Vuksic, a transdisciplinary associate of the project “Computer Signals”. During the workshop, Valentina set up an installation format in which the archive of sounds, produced by the research project, could be explored.

In the evening, the workshop was concluded with a live performance by Valentina, in which she presented artistic formats that stem from straightforward audifications of computational processes with little aesthetic consideration taken at first, and yet, took on a double life as musical works outside of their context.
The electromagnetic, electric and mechanical recordings originate from the research infrastructure of the biological laboratory at UT Austin by Hans Hofmann and the underwater observatory RemOS in Kongsfjorden, Spitsbergen by Philipp Fischer (Alfred-Wegener-Institut for polar and marine research). The audio material stays unprocessed; it is merely re-arranged and layered. The sonic works set out from digital data generation as part of scientific procedures to take a specific course outlined by a series of sonic extracts.


Here you can listen to excerpts from Valentina’s work that she presented during the evening:
Photos and videos by Jana Hambitzer
Header picture: RemOs1, Archiv Stereometrie (15. 9. 2012 – 16. 6. 2020), 2022. Detailansicht Fotoinstallation Ausstellung «Daten lauschen» im Deutschen Schifffahrtsmuseum, Bremerhaven 2022. Fotodruck auf Polycarbonatplatten, 135.168 Bildpaare, 2.32 x 1.59 x 60 m. Fotografie: Marc Latzel.
Objects of Research: Bart Penders


Here comes the new edition of our “Objects of Research” series. c:o/re Senior Fellow Dr. Bart Penders provides an insight into his research work and introduces an important tool for this:
“As part of the work I do at KHK c:/ore, as well as extending beyond that, I collect empirical data. In my case, that data consists of records of interviews with scientists and others. Those records can be notes, but they can also be integral recordings of the conversations.
Relying on technology for the production of data is what scientists do on a daily basis. With that comes a healthy level of paranoia around that technology. Calibrating measurement instruments, measurement triangulation, and comparisons to earlier and future records all help us to alleviate that paranoia. I am not immune and my coping mechanism has been, for many years, to take a spare recording device with me.
This is that spare, my backup, and thereby the materialisation of how to deal with moderate levels of technological paranoia. It is not actually a formal voice recorder, but an old digital music player I have had for 15 years, the Creative Zen Vision M. It has an excellent microphone, abundant storage capacity (30 gigabytes) and, quite importantly, no remote access options. That last part is quite important to me, because it ensures that the recording cannot enter the ‘cloud’ and be accessed by anyone but me. Technologically, it is outdated. It no longer serves its original purpose: I never listen to music on it. Instead, it has donned a new mantle as a research tool.”

Would you like to find out more about our Objects of Research series at c:o/re? Then take a look at the pictures by Benjamin Peters, Andoni Ibarra, Hadeel Naeem, Alin Olteanu, Hans Ekkehard Plesser, Ana María Guzmán, Andrei Korbut, Erica Onnis and Phillip H. Roth.