Being in the third year of our Fellowship Program, c:o/re is accumulating a remarkable variety of perspectives revolving around its main focus, research on research.

Questions tackled in this lively research environment are highly interesting and exciting and, as such, complex. The meeting of distinct research cultures may stir curiosity but may also leave one wondering what is the other even talking about… What are they studying?

To offer an insightful glimpse into the lively dialogues here, bridging and reflecting on diverse academic cultures, we have started the blog series “Objects of Research”.
We asked current and former c:o/re fellows and academic staff to show us an object that is most relevant to their research in order to understand how they think about their work.

In 12 contributions, we were able to witness the personal connections researchers have to objects that shape their work. We now invite you to visit the individual contributions and explore the world of research once again.

Hans Ekkehard Plesser

“For the past two decades, I have had a leading role in developing the neuronal network simulator NEST. This high-quality research software can improve research culture by providing a foundation for reliable, reproducible and FAIRly sharable research in computational neuroscience. Together with colleagues, I work hard to establish “nest::simulated()” as a mark of quality for research results in the field. Collaboration in the NEST community is essential to this effort, and many great ideas have come up while sharing a cup of coffee.“

Dawid Kasprowicz

“This is a notebook my Mom gave me. She had it as a kind of leftover from a shopping tour and she thought that it might be of use for my work. And of course, she was right. And as you know, research always starts with a good question that attracts attention.”

Sarah R. Davies

“I guess many academics would share some varient of this image: a careful arrangement of computer equipment, coffee, notepads, pens, and the other detritus that lives on (my) desk.

For me it’s important that the technical equipment is shown in conjunction with the paper notebook and pens. I’m fussy about all of these things – it’s distracting when my computer set-up isn’t what I’m used to, and I need to use very specific pens from a particular store – but ultimately my thinking lives in the interactions between them.

My colleagues and I are working on an autoethnographic study of knowledge production, and notice that (our) creative research work often emerges as we move notes and ideas from paper to computer (and back again).”

Phillip H. Roth

“I use mechanical pencils (like the one in the photo) to highlight, annotate, question, clarify, or reference things I read in books. This helps me digest the arguments, ideas, and discourses I deal with in my historical and sociological research. I also have software for annotating and organizing PDFs on my iPad as well as a proper notebook for excerpting and writing down ideas. However, I’ve found that the best way for me to connect my reading practices with my thoughts is through the corporeal employment of a pencil on the physical pages of a book.”

Bart Penders

“As part of the work I do at KHK c:/ore, as well as extending beyond that, I collect empirical data. In my case, that data consists of records of interviews with scientists and others. Those records can be notes, but they can also be integral recordings of the conversations.

Relying on technology for the production of data is what scientists do on a daily basis. With that comes a healthy level of paranoia around that technology. Calibrating measurement instruments, measurement triangulation, and comparisons to earlier and future records all help us to alleviate that paranoia. I am not immune and my coping mechanism has been, for many years, to take a spare recording device with me.

This is that spare, my backup, and thereby the materialisation of how to deal with moderate levels of technological paranoia. It is not actually a formal voice recorder, but an old digital music player I have had for 15 years, the Creative Zen Vision M. It has an excellent microphone, abundant storage capacity (30 gigabytes) and, quite importantly, no remote access options. That last part is quite important to me, because it ensures that the recording cannot enter the ‘cloud’ and be accessed by anyone but me. Technologically, it is outdated. It no longer serves its original purpose: I never listen to music on it. Instead, it has donned a new mantle as a research tool.”

Erica Onnis

“When asked about the fundamental object for my research practice, I immediately thought of my computer, which seemed the obvious answer given that I read, study, and write on it most of the time.

Upon further reflection, however, I realized that on my computer, I just manage the initial and final phases of my research, namely gathering information and studying on the one hand, and writing papers on the other.

Yet, between these two phases, there is a crucial intermediate step that truly embodied the essence of research, for me: the reworking, systematization, organization, and re-elaboration of what I have read and studied, as well as the formulation of new ideas and hypothesis. These processes never occur on the computer but always on paper.

Therefore, the essential objects for my research are notebooks, sticky notes, notepads, pens, and pencils.”

Ana María Guzmán

“As I research Hegel’s logic and how he understands life as a logical category necessary to make nature intelligible, I work closely with his texts. On the other hand, the stickers on my laptop remind me of the need to look at reality and regularly question the relevance of my research for understanding current social phenomena. In this sense, I think I remain a Hegelian, because for Hegel one can only fully understand an object of research by looking at both its logical concept and how it appears in reality. However, I think that in order to look at current political and social phenomena, we need to go beyond Hegel’s racist and sexist ideas, which are all around his ideas on social organization. And none of this would be possible without a good cup of coffee and/or a club mate!”

Alin Olteanu

“The 3D replica of my teeth that stands on my desk reminds me of two important things. First, a model is what we make of it. The epistemic value of modelling lies in interpretation, which depends on but is not defined by representation. I make something very different of (a replica of) teeth than a dentist and an archaeologist do.

Secondly, and not any less important, this replica reminds me to smile, and I hope that it might inspire colleagues to smile, too, when they see it on my desk.
To tell a smile from a veil, as Pink Floyd ask us to, we need to know that a smile is infinitely more important than scientific modelling. If scientific modelling does not lead to smiling, it is of no value. A smile is a good metonymy to be reminded by.”

Andrei Korbut

“There is a joke about which faculty is cheaper for the university. Mathematics is very cheap because all they need is just pencils and erasers. But philosophy is even cheaper because they don’t even need erasers.

My favorite and indispensable object is the rOtring 600 mechanical pencil. It shows that social science is closer to mathematics than to philosophy. Of course, social scientists often need more than pencil and eraser: they have to collect and process data from the real world. But this processing is greatly facilitated by the ability to write and erase your observations.

In my work, I deal with the transcripts of human-machine communication, and I use the rOtring 600, which has a built-in eraser, a lot. It’s useful not only because of the eraser, but also because it’s designed to stay on the table and not break, even in very demanding circumstances like the train journey. And it gives me the feeling that I am making something tangible with it, because it reminds me of engineers or designers producing blueprints for objects and machines.”

Hadeel Naeem

“A pen and a notebook are essential for my research. They help me think. It’s not at all about the words I write. I rarely read them again. Scribbling is just an act that helps me stack ideas on top of each other and do all the complicated thinking and connection-building.

I also turn to scribbling in my notebook when I am stuck in the writing process. There is often a time after the first rough draft of the paper when some ideas stop flowing smoothly or don’t fit very well with the main argument. I turn to the notebook and start writing the main ideas, deliberating how they support each other.

This is all especially interesting since a lot of my research is about extended cognition, which is the idea that we sometimes employ external resources such that part of our thinking happens outside our body (in these resources).”

Andoni Ibarra

“Spending a few weeks in Argentina, in front of my desk, a Post Office building. A nice futuristic architectural concept, degraded by its construction materials, support of a communication antenna, appropriated by pigeons as a dovecote: a hybrid object.”

Benjamin Peters

“By saying that I study ‘artful intelligence’, which I mean only as a half joke, I take seriously the propositions to my career as a media scholar that…

1. As the first image suggests, human artfulness can be found all around, such as this snapshot of a wall on a side street not far from the Cultures of Research at the RWTH.

2. Sometimes architectural masterpieces that represent more than the sharp angles of twentieth-century modernism are all about us, such as this bus stop on the way to Cultures of Research in Aachen. Any study of science and technology has to ask, what does it mean? Sources do not speak for themselves.

3. Sometimes artificial intelligence is best found in letting people be people, such as a doodle here in a sketchbook. Straight lines do not always precipitate straightness.

4. I study how science, technology, and artificial intelligence has been understood in different times and places, such as this remote-controlled robot that failed in the immediate aftermath of the Chernobyl explosion in 1986 in Soviet Ukraine, which helps unstiffen, enliven, and sober our imagination of what may already be the case today and could be the case tomorrow.”

Thank you for joining us on this journey. We look forward to share more insights and stories with you!

In February 2024, a collaboration with colleagues from Ritsumeikan Asia Pacific University on the topic of Emotionalized Artificial Intelligence (EAI) started. Professor Peter Mantello (Ritsumeikan Asia Pacific University) leads a 3-year project funded by the Japan Society for the Promotion of Science, on which c:o/re is a partner, that will compare attitudes in Japan and in Germany on EAI in the workspace. This explorative pathway contributes to the c:o/re outlook on Varieties of Science. You can find the whole project description on our website here.

In the interview below, Peter Mantello explains what EAI is, how the project will consider AI ethics and why the comparison of German and Japanese workplaces is particularly insightful. We thank him for this interview and look forward to working together.

Peter Mantello

c:o/re short-term
Senior Fellow (11-17/2/2024)

Peter Mantello is an artist, filmmaker and Professor of Media Studies at Ritsumeikan Asia Pacific University in Japan. Since 2010, he has been a principal investigator on various research projects examining the intersection between emerging media technologies, social media artifacts, artificially intelligent agents, hyperconsumerism and conflict.

What is Emotionalized Artificial Intelligence (EAI)? What does this formulation entail differently than ‘Emotional’ AI?

Emotional AI is the commercial moniker of a sub-branch in computer science known as affective computing. The technology is designed to read, monitor, and evaluate a person’s subjective state. It does this by measuring heart rate, respiration rate, skin perspiration levels, blood pressure, eye movement, facial micro-expressions, gait, and word choice. It involves a range of hardware and software. This includes cameras, biometric sensors and actuators, big data, large language models, natural language processing, voice tone analytics, machine learning, and neural networks. Emotionalized AI appears in two distinct forms: embodied (care/nursing robots, smart toys) and disembodied (chatbots, smartphone apps, wearables, and algorithmically coded spaces). 
I think the term ’emotionalized’ AI better encompasses the ability of AI not to just read, and recognize human emotion but also to simulate and respond in an empathic manner. Examples of this can be found in therapy robots, chatbots, smart toys, and holograms. EAI in allows these forms of AI to communicate in a human-like manner. 

What is emotionalized AI used for and for what is it further developed?

Currently, emotionalized AI can be found in automobiles, smart toys, healthcare (therapy robots/ doctor-patients conversational AI) automated management systems in the workplace, advertising billboards, kiosks and menus, home assistants, social media platforms, security systems, wellness apps and videogames. 

What forms of ethical work practices and governance do you have in mind? Are there concrete examples?

There are a range of moral and ethical issues that encompass AI. Many of these are similar to conventional usages of AI, such as concerns about data collection, data management, data ownership, algorithmic bias, privacy, agency, and autonomy. But what is specific about emotionalized AI is that the technology pierces through the corporeal exterior of a person into the private and intimate recesses of their subjective state. Moreover, because the technology targets non-conscious data extracted from a person’s body, they may not be aware or consent to the monitoring. 

Where do you see the importance of cultural diversity in AI ethics?

Well, it raises important issues confronting the technology’s legitimacy.  First, the emotionalized AI industry is predominantly based in the West, yet the products are exported to many world regions. Not only are the data sets used to train the algorithms limited to primarily Westerners, but they also rely largely on famed American sociologist Paul Eckman’s ‘universality of emotions theory’ that suggests there are six basic emotions and are expressed in the same manner by all cultures. This is untrue. But thanks to a growing number of critics who have challenged the reliability/credibility of face analytics, Eckman’s theory has been discredited. However, this has not stopped many companies from designing their technologies on Eckman’s debunked templates. Second, empathetic surveillance in certain institutional settings (school, office, factory) could lead to emotional policing, where to be ‘normal’ or ‘productive’ will require people to be always ‘authentic’, ‘positive’, and ‘happy’. I’m thinking of possible dystopian Black Mirror scenarios, like in the episode known as “Nosedive”. 
Third, exactly what kind of values do we want AI to have – Confucian, Buddhist, Western Liberal? 

Do you expect to find significant differences between the Japanese and German workplace?

Well, it’s important to understand the multiple definitions of the workplace. Workplaces include commercial vehicles, ridesharing, remote workspaces, hospitals, restaurants, and public spaces, not just brick-and-mortar white-collar offices. 
Japan and Germany share common work culture features, but each society also has historically different attitudes to human resource management relationships, what constitutes a ‘good’ worker, loyalty, corporate responsibility to workers, worker rights and unions, and precarity. The two cultures also differ in how they express their emotions,  raising questions about the imposition of US and European emotion analytics in the Japanese context.

How will the research proceed?

The first stage of the research will be to map the ecology of emotion analytics companies in the West and East. This includes visits to trade show exhibits, technology fairs, start-up meetings, etc. The second stage will be interviews. The third stage will include a series of design fiction workshops targeted to key stakeholders. Throughout all of these stages, we will be holding workshops in Germany and Tokyo, inviting a interdisciplinary mix of scholars, practitioners, civil liberties advocates and industry people. 

What do you think will be the most important impact of this project?

We are at a critical junction point in defining and deciding how we want to live with artificial intelligence. Certainly, everyone talks about human-centric AI but I don’t know what that means. Or if that’s the best way forward. Humans haven’t necessarily made the best choices for our world. If we try to make AI in our own image, it might not turn out right. What I hope this project brings are philosophical insights that will better inform the values we need to encode into AI, so it serves the best interests of everyone, especially, those who will be most vulnerable to its influence. 

What inspired you to collaborate with c:o/re?

My inspiration to collaborate with c:o/re stems from my growing interest in phenomenological aspects of human-machine relations. For the past three years, my research has focused primarily on empirical studies of AI. The insights gained from this were very satisfying, albeit they also opened the door to larger, more complex questions that could only be examined from a more theoretical and philosophical perspective. After a chance meeting with Alin Olteanu at a semiotic conference, I was invited to attend a c:o/re workshop on software in 2023. I realized then that KHK’s interdisciplinary and international environment would be a perfect place for an international collaborative research project.   

On 16 February and 17 February 2024, the workshop “Art, Science, the Public” took place at the KHK c:o/re in cooperation with the project “Computer Signals: Art and Biology in the Age of Digital Experimentation“, a research collaboration between artists, biologists and humanities scholars, in which c:o/re director Gabriele Gramelsberger has been involved since a long time. 

Together with representatives and colleagues from the research group “Computer Signals”, PACT Zollverein and RWTH Knowledge Hub, different formats and practices of science communication, in particular those that experiment with artistic forms, were discussed. The aim of the workshop was to exchange ideas and best practice examples on the interface between science and art and the associated communication challenges.

A special highlight was the sound work by Valentina Vuksic, a transdisciplinary associate of the project “Computer Signals”. During the workshop, Valentina set up an installation format in which the archive of sounds, produced by the research project, could be explored.

In the evening, the workshop was concluded with a live performance by Valentina, in which she presented artistic formats that stem from straightforward audifications of computational processes with little aesthetic consideration taken at first, and yet, took on a double life as musical works outside of their context.

The electromagnetic, electric and mechanical recordings originate from the research infrastructure of the biological laboratory at UT Austin by Hans Hofmann and the underwater observatory RemOS in Kongsfjorden, Spitsbergen by Philipp Fischer (Alfred-Wegener-Institut for polar and marine research). The audio material stays unprocessed; it is merely re-arranged and layered. The sonic works set out from digital data generation as part of scientific procedures to take a specific course outlined by a series of sonic extracts.

Here you can listen to excerpts from Valentina’s work that she presented that evening

Photos and videos by Jana Hambitzer

Header picture: RemOs1, Archiv Stereometrie (15. 9. 2012 – 16. 6. 2020), 2022. Detailansicht Fotoinstallation Ausstellung «Daten lauschen» im Deutschen Schifffahrtsmuseum, Bremerhaven 2022. Fotodruck auf Polycarbonatplatten, 135.168 Bildpaare, 2.32 x 1.59 x 60 m.  Fotografie: Marc Latzel.