we are experience designers

my ai promise

11-07-2023, merijn denkers

exploring ai: a personal journey


If you talk with someone for more than half an hour, the conversation will inevitably turn to the topic of Artificial Intelligence. Okay, I realise that this is driven by my own fixation. The greengrocer might experience it differently, but as a creative professional, you cannot escape the subject. The emotions range from wonder and enthusiasm to fear and suspicion.

Especially in professional circles, I sense a strong fear of missing out, coupled with a fear that AI will take over our jobs. "And it's happening faster than you think" is often the addition. Of course, it will radically change creative professions. It is similar to the arrival of graphic software that brought the work of graphic designers from the drawing board to the screen in the 1980s. Monks must have scratched their heads in the Middle Ages when the first bibles appeared in larger editions. What else can you do with all your time? Brew beer?

I, too, was skeptical at first. But after a few months of studying practical applications of AI, I have reassured myself with the conclusion that as experience designers, we will not be quickly replaced by Artificial Intelligence. The parameters are too much subject to the human scale for that. In an experience, all senses are stimulated, and AI cannot determine or fill that emotion. Computers cannot take over intuitive and associative thinking. Immersion cannot be captured in hypernetworks, no matter how intelligent they are. The initial emotion and information that a scenography must communicate are determined by scientists, project managers, conservators, and copywriters, in agreement with scenographers, lighting designers, graphic designers, sound designers, performers, and many other experts. This cannot be replaced by Large Language Models or other forms of AI. It is not only too complex; it is primarily too human. But then what to expect?

Embracing AI and shaping the future of museum experiences
Artificial Intelligence will totally change the museums and experiences scenery. The challenge is to transform our way of thinking towards it. It is already an indispensable tool in our design process, but it will also completely transform the visitor experience. Storytelling in museums and experiences has been pre-scripted until now. As a visitor, you can get answers to questions, but the questions have already been devised. The transfer has already been written. Interactive installations follow a strict flowchart, films are written frame by frame, audio is linearly designed and plays in a loop. The narrative is determined. However, in 10 years, this will be a thing of the past. Get used to the idea.

Hosts, stewards, and volunteers can, for example, be replaced by a friendly and hyperrealistic virtual personal assistant who can answer any question in a dynamic and organic conversation. Do you speak Finnish? Then the host switches to Finnish. Or would you prefer Mandarin? No problem. A recent example of a development in that direction is the introduction of Nvidia's ACE, a cloud engine designed to bring digital avatars to life. Through speech recognition, text-to-speech, and Natural Language Understanding (Large Language Model), it can animate an avatar, making it respond directly and naturally to your voice and movement. As if you're talking to a real person. You program an avatar with a story, and all a visitor has to do is talk to the avatar. Any question is possible. Even off-topic. Everything in real-time and in the form of a hyperrealistic virtual animated person. Or a monkey. Or a cartoon character. If you don't like the character, you ask if it can change into something else. Maybe they have different insights?

AI in practice: exploring the limits of artistic expression
Total immersion will move along with this. At Tinker, we are already testing and applying this behind the scenes. For example, we're developing a concept for a new project, and I can give you a glimpse of it. How can you develop a visual language that conveys a positive message about the value of nature but also warns future generations about the risks of polluting forms of energy generation? I've been inspired by allegorical classical painting. These paintings are filled with symbolism that needs to communicate complex messages. Art historians are still unraveling the symbolism.

"An allegory is a literary or artistic work in which characters, events, or objects are used symbolically to represent abstract ideas or principles. The purpose is to communicate a deeper, hidden meaning than what is presented on the surface." (Thanks, GPT!)

For this project, I provided GPT with input to generate descriptions for new allegorical still lifes with the positive message that also contains the warning. The result is a complete description of the symbolism and a prompt for a text-to-image generator. I copy-paste the prompt to Midjourney, and the results are incredible! Aesthetic masterpieces. Intimidating perfect but with mutated nuances that make you pause and wonder what you're actually seeing.

Imagining Alien Worlds using AI
Another example is an autonomous side project, a study exploring the possibilities of AI. In this project, I create (together with my buddy AI) glimpses into alien worlds with organisms never seen before. I use GPT-4 to generate datasets about the organic world and compositions of environments. I use these sets in Google Sheets to generate prompts, which I then use in GPT-4 to generate fictional organisms and prompts for Midjourney. The images generated can seamlessly be enhanced using Photoshop's generative AI. The results are stunning! It's like being the first to look into a new world, like a virtual explorer. Only David Attenborough's voice is missing, but with the right AI tools and a bit of study time, that can also be tackled.

I expect that the public's value of palpable cultural heritage in museums and physical experiences will only increase. In a world where you constantly have to question if something is real or fake, it will feel like a breath of fresh air to look at something that is physically and guaranteed real. Something that provides material meaning to our history. To your history. The information and interaction that provide context to the objects will become much smarter. More fluid. More dynamic. Less predictable and therefore more exciting. And above all, more enjoyable and interesting because it aligns better with the individual. This applies to all cultural expressions. Live concerts, theatre, and visual arts will gain even more significance. Simply because it’s wonderful to be human and to be reminded of that sometimes.

It's an exciting time as a creative professional. The past six months have been a rollercoaster for me. Besides the bewildering speed of developments, AI dilemmas such as climate impact, bias, copyright, privacy, and the power of tech companies are quite a mind-boggling experience. For now, I reassure myself that we can solve electricity and bias, copyright is something only boomers worry about (welcome to 2023!), and privacy and the power of Big Tech need to be regulated at the European level (which I have no control over). That's how I make it okay for myself... The creative doors that have opened for me as a designer are simply too enticing not to engage with. Artists must have felt the same way when the first portraits were captured on sensitive plates in the 19th century. You just had to sit still for half an hour, and you had a portrait! Now, you give a creative prompt, and within seconds, you receive the most incredible images that no one has ever seen before. And we're only at the beginning...

Human artistic voice will never be replaced. You still have to develop good taste to create beautiful things. Your sense of wonder is still safe. I will only use AI to stimulate and enrich that sense of wonder, I solemnly promise that.