Käte Hamburger Kolleg: Cultures of Research

NIKITA BRAGUINSKI

As a c:o/re fellow, I had the uniquely advantageous opportunity to develop and test, in an environment dedicated to the study of science, my ideas about how AI and data can influence music research. Members of the Kolleg and its fellows, many of whom are philosophers of science, offered a very rich intellectual circle that inspired me to look at the datafication and technologization of future music research from many new angles. With its intensive and diverse program of talks, lectures, and conferences, the Kolleg also offered ideal opportunities for testing approaches in front of an attentive, thoughtful, critical and friendly audience. Below, I  present brief overviews of the main ideas that I discussed during three talks I gave at the Kolleg.

Nikita Braguinski

Nikita Braguinski studies the implications of technology for musicology and music. In his current work, he aims to discuss challenges posed to human musical theory by recent advances in machine learning.

My first presentation, entitled “The Shifting Boundaries of Music-Related Research: Listening Logs, Non-Human-Readable Data, and AI”, took place on January 16, 2024 during an internal meeting of Kolleg fellows and members. I focused on the promises and problems of using data about music streaming behavior for musical research. Starting from the discussion of how changing technologies of sound reproduction enabled differing degrees of observing listener behavior, I discussed the current split between academic and industrial music research, the availability of data, the problems of current industry-provided metrics such as “danceability”, and the special opportunities offered by existing and future multimodal machine learning (like the systems that use the same internal encoding for both music and text). I also offered examples of descriptive statistics and visualizations made possible by the availability of data on listener behavior. These visualizations of large listening datasets, which I was able to create thanks to my access to the RWTH high performance computing cluster, included, among others, an illustration of how users of online streaming services tend to listen to new recordings on the day of their release, and an analysis of the likeliness of different age groups to listen to popular music from different decades (with users from the age group 60-69 having almost the opposite musical preferences of the age group 10-19).

Discussing my talk, c:o/re colleagues drew parallels to other academic disciplines such as digital sociology and research on pharmaceutical companies. The topic of addictiveness of online media that I touched upon was discussed in comparison to data-gathering practices in gambling, including the ethics of using such data for research. The political significance of music listening and its connection to emotions was also discussed in relation to the danger of biases in music recommender systems.

My second presentation, entitled “Imitations of Human Musical Creativity: Process or Product?”, took place during the conference “Politics of the Machines 2024. Lifelikeness and Beyond”, which c:o/re hosted. I focused on the question of what AI-based imitations of music actually model – the final product (such as the notation or the audio recording) or the processes that lead to the creation of this product.

In this presentation, I discussed:

1) The distinction between process and product of artistic creation, which, while especially important for discussions on the output of generative AI, currently receives little scholarly attention;

2) How several theories in the humanities (notably, formalism, psychoanalytic literary theory, and the line of AI skepticism connected to the so-called Chinese room argument) stress the importance of the process in artistic creation and cognition;

3) That current endeavors in generative AI, though impressive from the point of view of the product, do not attempt to imitate the processes of creation, dissemination, and reception of art, literature, or music, nor do they imitate historical, cultural, or economic environments in which these processes take place;

4) Finally, because the data on which generative AI systems operate carries traces of past processes, the product of these systems remains connected to the processes, even if no conscious effort is made by the creators of these systems to imitate the processes themselves.

A conference participant commented that for commercial companies avoiding the imitation of all these processes is a deliberate strategy because their imitation has to be cheaper than the original process-based artifact.

My third presentation at the Kolleg, “Life-Like Artificial Music: Understanding the Impact of AI on Musical Thinking”, took place on June 5, 2024 as a lecture in the c:o/re Lifelikeness lecture series. Here, I addressed the likeliness (or unlikeliness) of major shifts in the musicological terminology to result from the academic use of AI . Starting with an overview of various competing paradigms of musical research, I drew attention to possible upcoming problems of justifying the validity of currently existing musicological terminology. The salient point here is that AI systems based on machine learning are capable of imitating historical musical styles without recourse to explicitly stated rules of musical theory, while humans need the rules to learn to imitate those styles. Moreover, the ability of machine learning systems to learn internal structures of music directly from audio (skipping the notation stage on which most of human music theory operates) has the potential to question the validity and usefulness of musical theory, as currently taught.

Having stated these potential problems, I turned to a current example, a research paper [1] in which notions of Western music theory were compared to the internal representations learned by an AI system from music examples. Using this paper as a starting point for my argument, I asked whether it could be possible in principle to also use such an approach to come up with new, maybe better, musicological terminology. I pointed to the problems of interpreting the structures learned by machine learning systems and of the likely incompatibility of such structures (even if successfully decoded) with the human cognitive apparatus. To illustrate this, I referred to the use, by beginner players of the game of Go, of moves made by AI systems. Casual players are normally discouraged from copying the moves of professional human players because they cannot fully understand these moves’ underlying logic and thus cannot effectively integrate them into their strategy.

In the following discussion, one participant drew attention to the fact that new technologies often lead to a change in what is seen as a valid research contribution, devaluing older types of research outcomes and creating new ones. Another participant argued that a constant process of terminological change takes place in disciplines at all times and independently of a possible influence of a new technology, such as machine learning.

Overall, my c:o/re fellowship offered, and continues to offer, an ideal opportunity to develop and discuss new ideas for my inquiry into the future uses and problems of AI and data in music research, which have resulted, in addition to the three presentations mentioned above, in talks given at the University of Bonn, Maastricht University, and at a music and AI conference at the University of Hong Kong.

[1] N. Cosme-Clifford, J. Symons, K. Kapoor and C. W. White, “Musicological Interpretability in Generative Transformers,” 4th International Symposium on the Internet of Sounds, Pisa, Italy, 2023

It has been more than a month since the KHK c:o/re hosted the PoM conference “Lifelikeness & beyond”. As this sizeable and, while still new, already renown conference produced many lively discussions in a creative interrogation of the dialog between life sciences and technology studies, we want to share our retrospective reflections on it through a series of focused posts. In two interviews, the artists shared with us insights into their work and creative process. Here, we reflect on the board game Algorithms of Late Capitalism together with Karla Zavala Barreda and Adriaan Odendaal.

What can we learn from the contingency of the community of the living and the non-living? What insights on contingency may transpire from embedding life and non-life within each other? How are factuality and fiction mediated by the imagination in the pursuit for new forms of collective action and of creating collectivities?

Algorithms of Late-Capitalism: The Board Game
by Karla Zavala Barreda and Adriaan Odendaal

During the PoM conference, Karla Zavala Barreda and Adriaan Odendaal from the research & design studio internet teapot hosted a series of guided play-sessions of their new board game “Algorithms of Late-Capitalism”.

In 2021, they conducted a series of experimental workshops as part of the New New Fellowship that brought diverse groups of international participants together to co-design a board game. The purpose of this project was to use board game co-design as a medium through which participants can collectively explore questions around more pluralistic and desirable technological futures. Over the course of several workshop sessions, participants contributed ideas and reflections to the creation of the game, framed by concepts drawn from pluriversal ontological design, intersectional feminism, and digital materialism.

In Algorithms of Late-Capitalism, players become members of a community of cyborgs, reigned over by the first Sentient Machine Cult. This cult has given rise to a formative new algocracy in which society is governed by the organizational logic of rigid data structures and opaque algorithms. The players-as-cyborgs are confronted with a rule-system that places them in a position of systematic exclusion and increasing marginality.
The board game affords different ways of playing: players can integrate themselves into this society by following the formal rules and competing against each other to conform to the logic of the Sentient Machine Cult’s algocracy; or they can subversively coordinate their efforts and attempt to change the system by introducing new rules and winning conditions. By discovering ways to play collaboratively instead of competitively, players are encouraged to explore alternative, convivial, caring, and inherently pluralistic technological futures – as well as possible pathways towards these futures.

By playing the game, conference attendees were able to explore reflections, questions, and ideas encoded into the game fiction and mechanics by the different cohorts of game co-designers.

How did the idea of developing the game come up?

Karla: We have been exploring the medium of board game design for a couple of years, both designing prototypes and playing them. On the other hand, we have also been hosting and co-creating zines, so when the New New Fellowship opportunity came up, we thought of it as a chance to merge game design with co-creation methodologies. We also believe that design can foster critical reflection and social transformation. So we wanted participants to think about the absurdities of the technology in our present and through this lens imagine better futures. As technology users we all have an expertise to share. We want to open the barriers to technology design, so that everyone can share their experiences and perspectives to help improve things. Through the board game design, we wanted to ask: What can be reimagined to make more inclusive and desirable technological futures?

What is the goal of the game?

Adriaan: The goal is to create an open space for people to contribute to and enrich the process of thinking about technological futures. The game is an exploration of how we can benefit from collaborative processes, instead of following the imperatives of market-driven competition. We want people to explore  these critical and conceptual points through low-barrier and playful mediums. Board games are also very social objects, they create social spaces where people can connect and start discussions. By playing, people engage with more inclusive imaginaries of better technological futures. When we think of digital technologies, for example AI, what probably comes to mind is widespread services such as ChatGPT. Big tech companies’ imaginaries dominate the discussions of what technology is and can be. But, through co-creating the board game we explored alternative imaginaries.

Karla: It’s important to empower the broader public to imagine what technology can be and understand that they should have a say in what technologies get deployed in their cities and societies at large. As a society we negotiate culturally how technology works, as such public participation should be fostered. This was the goal of the co-creation workshops that brought this game to life, to give non-technical public the tools to think of important questions around our increasingly digitized and mediatized societies.

What would be the ideal technological future for you?

Adriaan: There should be more diversity in technology. Smaller, weirder, experimental things. I would wish for a future where technology is curious and diverse and not dominated by a few companies that copy each other.

Karla: A future where  communities understand how technology works and have a say in the technologies that impact their lives. To me, especially understanding that technology is socially constructed is important, what we think as a community of certain technologies matter. Technology carries values and worldviews, there should be more variety and creative imaginaries around it.

How should things continue with your game?

Karla: We will soon publish it as a print to play version. Our aim is that the game can be used as means to open conversations about technology and its role in our social and intimate lives in diverse settings: from schools to university students and even policy making.

The board game is currently available as a free print-to-play version online. You can also follow Karla’s and Adriaan’s work on Instagram.

Would you like to gain further impressions of the PoM conference in Aachen? Then take a look at our interview with Chris Dupuis as well as our recap of the conference days and the accompanying program of art and performances.

Photos by Jana Hambitzer

It has been more than a month since the KHK c:o/re hosted the PoM conference “Lifelikeness & beyond”. As this sizeable and, while still new, already renown conference produced many lively discussions in a creative interrogation of the dialog between life sciences and technology studies, we want to share our retrospective reflections on it through a series of focused posts. In two interviews, the artists shared with us insights into their work and creative process. Here, we reflect on the performance Dead people are like things on Facebook in conversation with Chris Dupuis.

What can we learn from the contingency of the community of the living and the non-living? What insights on contingency may transpire from embedding life and non-life within each other? How are factuality and fiction mediated by the imagination in the pursuit for new forms of collective action and of creating collectivities?

Dead People Are Liking Things On Facebook
by Chris Dupuis

What happens to our online self after we die? How might this material be used by others, and to what effect? Does this material serve as a valid means of remembering people? Do we remember them as they were or as they wanted to be?

Chris Dupuis asked himself these questions as part of his interactive lecture performance. In this interview, he provides insights into the background to his work and how he deals with death in social media.

Could you please introduce yourself?

Chris: I’m a Canadian writer, curator, and performance maker, based in Brussels.

What is your performance about?

Chris: “Dead People Are Liking Things On Facebook” is a lecture performance where I scroll through the profiles of Facebook friends who have died, discuss how I knew them, and what meaning can be taken from their online afterlives. The show was catalyzed in 2016 when I was scrolling through Facebook and noticed that my friend Will had “liked” Coca-Cola. In one way, this wasn’t strange, as Will actually liked Coca-Cola in real life. Will was a well-known Toronto DJ and queer club promoter in the early 2000s. He was famously sober, but thought everyone needed at least one bad habit, and so at some point, he decided Coke would be his vice. At the same time, it was strange that he had “liked” it in 2016 since at that point he had been dead for six years. How was this possible? The show started with me searching for the answer to that question.

What do you want to show with your performance?

Chris: I think a lot of the experience is up to the audience to interpret. I’m not really trying to “show” anything or make any specific claims. It’s more about raising a series of questions about mortality, social media, and the construction of identity for us to consider together.

What do you want to happen to your online presence after your death?

Chris: Despite having toured this show for several years and being preoccupied with these questions the whole time, I haven’t actually made any decisions about it. But assuming I have an average lifespan, the Internet and human connectivity will probably look radically different than it does now, so it’s difficult for me to imagine what I’ll be concerned with then.

How do you think social media platforms will deal with this type of situation in the future?

Chris: When all of these social media platforms and tech companies were starting out, they weren’t considering where they would be in twenty years. They were thinking about surviving the next six months. As they’ve gradually come to control so much of our lives and our public discourse, I think that some of them (though not all of them) have genuine concerns with how to navigate the future with the power they wield. At the same time, there’s also a question of how many of these companies will be around in the future or whether they will be replaced by AI versions that allow us to live online in very different ways, particularly as they may intersect with VR. What does seem clear is that there needs to be some level of government intervention to regulate these companies as they develop increasingly powerful tools.

Would you like to gain further impressions of the PoM conference in Aachen? Then take a look at our interview with Karla Zavala Barreda and Adriaan Odendaal as well as our recap of the conference days and the accompanying program of art and performances.

Photos by Jana Hambitzer

ANA MARÍA GUZMÁN OLMOS

What do synthetic cells have to do with programmed biosensors? What ideas about life are embedded in their development? What are the boundaries between the living and the non-living? How have these boundaries been reshaped in our current (post)digital condition? How can science and the humanities collaborate with the arts to address these complex questions? From April 22 to 25, c:o/re, in collaboration with the Politics of the Machines conference series, c:o/re co-organized an interdisciplinary conference to address these and many other questions.

The Politics of the Machines conference series was founded in 2018 by Laura Beloff (Aalto University, Helsinki) and Morten Sondergaard (Aalborg University, Copenhagen), and now counts Hassan Choubassi (The International University of Beirut) and Joe Elias (The International University of Beirut) as part of its executive committee. The conference series aims to open up open discussions on how perception is being reshaped through interaction with emerging technologies. Previous conferences in this series were hosted by the Aalborg University, Copenhagen, the Lebanese International University, Beirut, the Universität der Künste, Berlin and Beaconhouse National University, Lahore… Now interrogated under the scope of Politice of the machines, lifelikeness is a central interest at c:o/re and, as such, the theme of our last two lecture series and the last Call for Fellowships.

Beyond the limits of the living and non-living

Lifelikeness refers to the imitation of living behavior in technological development. Originating in the field of robotics and human-machine interaction (Abubshait, 2020), the concept has been expanded to refer to more recent developments in the fields of synthetic biology and artificial life. One way to approach the concept of likeness is to consider the concept of imitation as an operation of representation from the living into its modeling and operationalization in technology. This could be called a representational model. The representation can take place on the level of aesthetics (looking like a living thing) or of operation (having a function that only a living thing would have). In this perspective, many discussions on lifelikeness center on the question of how similar a technology should be to the living thing it imitates to qualify as life-like. The uncanny valley is a classic problematization arising in these discussions: if the technology is very similar to living, we may develop an aversion to the technical object instead of the expected sense of familiarity (Kim et. al., 2022: 628).

As cells have become machines (Landecker, 2007; Damiano et. al. 2020), the question of what it means to model life expanded beyond these classic interrogations. If life itself has become entangled with technological operations, and digital algorithms operate on the principles of living organisms, what exactly is being represented in the models that inspire new technological developments? Where is the life and where is the machine? Arguably, the model of representation or imitation may not be sufficient to explain the complexity in which the living and the non-living have become entangled. This disruption of the boundaries between the living and the non-living leads to questioning how social life is affected by technologically mediated decisions, and what kind of politics are necessary to deal with these new conditions of living. For example, algorithms in preventive technologies decide whose life is to be cared for and whose life is to be considered a threat to life (Amaro, 2022). Such questions and issues have inspired the title of the conference “Llifelikeness & beyond”.

The conference was organized into 11 tracks that addressed topics such as death in the age of the lifelike, more than human XR, models in the research of the living, the concept of the organism and environmental approaches to ecological engagement, and the multiple imaginaries of our bodies in our present. Discussions covered various topics such as: a redefinition of immersion based on experiences of radicalization in the conditions of an extended reality, how to decolonize the discourses that approach the environment, what is the sound produced by the environment and its microscopic invisible fields, the concept of the organism and its appearance in political and social explanatory models, the redefinition of the concept of death in the age of the lifelike, and many others.

From Enzymes to Magic

The two keynotes, by Hannah Landecker (University of California in Los Angeles) and, respectively Manuela de Barros (University of Paris 8), brought together the broadly multidisciplinary approaches of the conference.

Hannah Landecker delivered an inspiring keynote entitled “Distilled, Extruded, Suspended: Lessons in Lifelikeness from the Metabolism of Mass Production”. Landecker looked at the history of the industrialization of metabolism in the twentieth century, a history of transforming ideas of synthesis through the market demands for rubber and oil. This lecture invited participants to rethink the technological transformation of our world, a world that remains invisible to us. Questions about the division between the synthetic and the natural, and how they shape our perception of the world – in this case through the development of an industrially produced taste – emerged during the Q&A session. We remark professor Landecker having presented her ongoing working, which will be part of the book American Metabolism (under contract with Harvard University Press), on which she is currently working.

In her insightful keynote, Manuela de Barros highlighted the ways in which technology has been interfacing with the concept of magic since the Renaissance. De Barros also discussed how we form beliefs and expectations about what technology should provide to society, thus endowing it with a magical power to take care of social problems. She also encouraged revisiting animistic ideas of non-Western cultures, as helpful for reorienting the way the Western modernity inhabits this world. The talk inspired questions on how to approach non-Western worldviews, and how new forms of romanticizing these ways of life might do more harm than good in finding ways to live together. Such discussions are covered in de Barros’ Magie et technologie (UV Editions, 2017).

Algorithms of Late Capitalism, Dead people on facebook, and Workshops!

With an important concern for art and artistic research, the discursive program was accompanied by two workshops: Sibling-cenes: Building Narratives PostAnthropocene, facilitated by artist H C-(M), and Changed but equivalent: rewinding mental states in complex systems, facilitated by Lebanese writer Rayyan Dabbous. A performance playing with the concept of lecture performance by Chris Dupuis explored notions of death, as shaped by the existence of our digital persona. PoM: Lifelikeness & beyond also offered playing sessions of “Algorithms of Late Capitalism: The Board Game”, a playful critical engagement with internet literacy, developed by Karla Zavala Barreda and Adriaan Odentaal from the internet teapot, together with the many participants of a series of workshops in which the game was developed.

With about 130 talks and lecture-performances, 2 workshops, 1 performance, 1 exhibition and a board game, as well as an artistic event together with the arts and performance center PACT Zollverein in Essen. The conference brought together researchers from many different fields, from philosophy to biology and astronomy, from STS to art. It brought together, at RWTH Aachen, researchers from 30 different countries and 5 continents. With this series of blog posts, we hope to recapitulate some of the thoughts and resonances generated during PoM Aachen, and to further explore how to open boundaries of research beyond the division of living and the non-living.

References

Amaro, Ramon (2022). The Black Technical Object. On Machine Learning and the Aspiration of Black Being. London: Sternberg Press.

Abubshait, A., Weis, P.P. & Wiese, E. (2021). Does Context Matter? Effects of Robot Appearance and Reliability on Social Attention Differs Based on Lifelikeness of Gaze Task. Int J of Soc Robotics 13, 863–876.

Damiano L and Stano P (2020) On the “Life-Likeness” of Synthetic Cells. Front. Bioeng. Biotechnol. 8:953.

De Barros, Manuela (2017). Magie et technologie. Paris: UV Éditions.

Kim, J.S., Kang, D., Choi, J., Kwak, S.S. (2022). Effects of Realistic Appearance and Consumer Gender on Pet Robot Adoption. In: Cavallo, F., et al. Social Robotics. ICSR 2022. Lecture Notes in Computer Science(), vol 13818. Springer, Cham.

Landecker, Hannah (2007). Culturing Life: how cells became technology. Cambridge; London: Harvard University Press.

Would you like to gain further impressions of the PoM conference in Aachen? Then take a look at our interviews with Chris Dupuis and Karla Zavala Barreda and Adriaan Odendaal as well as our recap of the accompanying program of art and performances.

Photos by Jana Hambitzer