Dark surreal artwork featuring numerous distorted, grimacing human faces clustered tightly in a shadowy background.

Synthetic Dystopia

人工合成废托邦
Machine learning artworks & visual experiments
Category
Machine Learning
Location
New York, Us
YEAR
2022/05

Statement

Synthetic Dystopia is a series of Machine learning artworks and visual experiments based around the theme of urbicide. The project attempts to restore our estranged knowledge of urbicide, and challenge our generation to re-sensitize our perspective on urban destructions, by visualizing urban decays through a series of speculative digital artworks using machine learning algorithms.

Project Involvement

This is my final capstone project for Interactive Media Arts at New York University. Thanks to the amazing resources from Derrick Schultz and the assistance from ML artist AvantContra, the project received positive feedbacks from IMA and ML art community in China. I was invited to present and showcase my work at the Artiverse Talk livestreamed on Bilibili.

Credit

Produced by Damon Chen
Special Thanks to:
AvantContra
Daniel Shiffman
Derrick Schultz
HuemanInstrumenti
William Chen
Roopa Vasudevan
Julie LizardoTora Thuland

resources

Synthetic Dystopia PaperSynthetic Dystopia Presentation Slide
Project Purpose

Urbicide >

Urbicide means "violence against the city". The most obvious form of urbicide is the violence upon the city fabric, namely the physical destruction and demolition of the city and its architecture. Over the years, the term urbicide continues to evolve in broader terms. The destruction of urban foundation and urban life would be the destruction of a heterogeneous collection of identities, beliefs, traditions, cultures, and memories of its inhabitants.

Emotional Desensitization >

Desensitization to violence is a form of habituation that results in a diminished negative response and sentiments to violence, after repeated exposure to violence through digital media.
‍
People increasingly become indifferent to war and destruction in urban settings or even develop adhesive emotions. Instead of preventing further destruction and violence, a copious amount of communities either deliberately or indirectly increase the tension between the adversaries in such situations, and sometimes even provoke new forms of hostility, thereby creating a vicious cycle of urbicidal violence.

Machine Learning >

There are two reasons why I am using machine learning as the approach to create artworks:
‍
First, I am using AI as a tool or medium for creative production. Machine learning could be considered as a toolkit that facilitates image generation, animation, and manipulation. The process could be thought of as a human artist collaborating with AI assistants.

Second, I believe AI has an abstract representation and symbolic meaning in the theme of urbicide. For this project, I have created a fictional context where AI is the artist or the dreamer. In other words, I am using the lens of a fictional AGI to understand urbicide.
Act I : Decay Converter

Interior space ⥦ Decayed space >

Before

After

Before

After

Overview >_

The Decay Converter converts an interior space into its decayed form. The converter is capable of producing the ongoing, intermediate stage of decay, and projected destruction in the future, based on the input of the predecay image, and vice versa. It is a combination of the current decay that is happening and projected destruction in the future. The set of generated images aims to impart a sense of distress and imminence to the audience, by prophecizing the outcome of our surroundings in a state of urban decay.

Overview >.

The Decay Converter is presented as a data sculpture or data painting. Ideally, It would be a digital frame or a projection mapping installation set in the room of where the decay is generated. The use of particle systems is an analogy for machine learning art. Since each pixel of a generated image represents apiece of data in a ML model, it is constantly transforming, twisting, turning, and pushing into infinity, just like a particle in a vast and turmoil structure.

Technical Detail >

Model: MUNIT (Multimodal Unsupervised Image-to-Image Translation)/Cycle GAN

Program: Colab Notebook by Derrick Schultz, Unity Audio Reactive VFX (inspired by Yuma Yanagisawa

Dataset: 3343 images of interior space, 3890 images of decayed space

Dataset Source: 4750 from Flicker, 2283 from Google, 200 from my own albums

decayed example (ITP classroom) >

Before

After

Empty conference room with rows of black chairs and tables under fluorescent lights.
Empty classroom with multiple rows of tables and chairs facing whiteboards and a large TV screen.
Empty classroom with long tables and chairs arranged in rows facing a whiteboard.
Dark, abandoned conference room with long tables and black chairs scattered across a damaged floor.

decayed example (CApstone presentation room) >

Before

After

Empty room with rows of brown chairs facing a dark wall with a telescope and tripod on wooden floor.
Empty dark classroom with rows of black chairs facing a blackboard and a teacher's desk.
Empty room with rows of red chairs facing a projector screen and a small table with equipment on a worn wooden floor.
Empty lecture room with rows of chairs facing a podium and a large screen.

decay converter data sculpture >

Abstract art installation composed of numerous small reflective square tiles arranged in a swirling, dynamic pattern.
Dimly lit empty conference room with a long table, chairs, and large windows.
Modern conference room with black tables and chairs arranged in a U-shape, a large flatscreen TV on a stand, and a whiteboard wall on the right.
Act II : TImeline (evolution and extinction) of human architecture

Machine Generated Architectural Sketches >

Black and white sketch of a modern multi-story building with balconies and large windows.
Grid of 25 detailed black and white line drawings depicting various traditional and modern Japanese buildings.
Grid of 25 detailed black ink architectural sketches of various buildings and houses on a white background.

Overview >_

The project utilizes a fictional context from the perspective of an extraordinary artificial intelligence, set in the future, where humans ceased to exist. As the last conscious being in the long dilapidated cities, the AI is dreaming about the past ,contemplating the anti-climactic finale of human civilization. The AI is reminiscing on the violence that once took place, the indifference that it witnessed, and lamenting the cultural decay and loss of humanity.

Overview >.

The series of ML artwork are architectural sketches in the style of scientific illustration. These timelines are the impressions of how the machines study humans’ past, though these generated illustrations like the ones we see in encyclopedias or anatomy books.

Technical Detail >

Model: StyleGan2 ADA, StyleClip, Disco Diffusion
‍
Program: StyleGan2 ADA PyTorch Colab, Notebook by Derrick Schulz, StyleClip (StyleGan3 + CLIP) Colab Notebook by  nshepperd, Disco Diffusion v5.2 Colab Notebook by Katherine Crowson

Dataset: 570 processed architectural sketches

Dataset Source: @jlron, @yann.leroy, @i_am_yeono, @its_leepd, @maxiaowei_artist

"Archisketch" (custom trained styelgan model) >

Latent space linear interpolation (z:0.8) >

Truncation Interpolation (ψ=0.8-2.8, seed 470) >

Timeline of human architecture >

Detailed black-and-white ink drawing of a traditional two-story Japanese house surrounded by foliage.
Black and white pen sketch of a row of old, weathered buildings with varied architectural styles and details.
Detailed black and white ink sketch of an industrial building facade with pipes, windows, and structural supports.
Black and white sketch of three connected storefront buildings with varying roof and window styles.
Detailed black ink sketch of a multi-story industrial building with antennas and external staircases.
Black and white sketch of a multi-level modern building with various windows and structural details.
Detailed black and white sketch of a two-story building with storefront windows and awnings.
Artistic digital rendering of a modern building with curved glass facades and a traditional roof section.
Metallic abstract sculpture resembling a distorted, multi-layered building facade with windows and balconies.
Detailed sketch of a modern two-story building with large windows and exterior staircases under bright lighting.
Futuristic metallic sculpture with sharp, abstract architectural elements and reflective surfaces.
Sketched modern multi-story building with large windows and a power pole in front.
Futuristic architectural sketch of a multi-level building with modern and angular design elements, featuring large windows and varying textures.
Abstract digital artwork resembling a futuristic, fragmented building with protruding structures and illuminated windows.
Futuristic industrial building with complex pipes and modern skyscrapers in the background.
Futuristic, abstract building with complex metallic and glass structures, featuring a mix of geometric and layered design elements.
Digital artwork of a futuristic, weathered industrial building with intricate mechanical and metallic details.
Futuristic industrial building with metallic textures and complex architecture isolated on white background.
A weathered, industrial-style concrete building with exposed pipes and a utility pole in front, set against a white background.
Futuristic digital artwork of a multi-story urban building blending modern and traditional architecture with detailed textures and light reflections.
A digitally rendered dilapidated multi-story building overgrown with plants and graffiti, showing signs of decay and abandonment.
Abandoned industrial building ruins overgrown with greenery and vines.
Futuristic multi-level building with integrated greenery and industrial design elements.
Two weathered, abandoned buildings with broken windows and overgrown green plants on and around them.
Abandoned futuristic building partially covered with greenery and plants.
A futuristic, concrete building overgrown with tall green plants and vines against a white background.
A dilapidated abandoned building overgrown with green plants and moss, isolated on a white background.

Timeline of human architecture (causes of urbicide) >

Detailed black and white sketch of a two-story building with balconies, windows, and dense foliage surrounding it.
Modern building with curved, cantilevered upper section and adjacent greenery in an urban setting.
Abstract futuristic sculpture resembling a metallic, fluid form with embedded translucent textured rectangular panels on a reflective surface.
Abandoned, weathered concrete building with overgrown plants and cracked pavement surrounding it.
Abstract digital sculpture resembling a futuristic, organic architectural form on a textured surface.
Futuristic architectural model of a building with greenery and wooden textures on a layered base.
Abstract architectural structure blending organic, porous pink forms with modern metallic and glass elements on a light background.
Sketch of a two-story building with multiple windows, doors, and roller shutters.
Futuristic concrete building with smooth, curved architectural design and geometric detailing.
Futuristic architectural building with smooth, flowing curves and reflective metallic surfaces.
Futuristic, weathered building with layered concrete structures, graffiti, and sparse greenery.
Abstract architectural structure resembling a modern building with layered, beige, and textured surfaces on a barren ground.
Rustic, weathered wooden house with multiple levels and red vines growing around it.
Abstract futuristic building with organic mushroom-like structures and red dripping accents.
Detailed black and white sketch of a traditional two-story Japanese house with tiled roof and wooden exterior.
Digital artwork of a stylized multi-story house combining wooden and concrete architectural elements with a curved roof.
Intricately detailed traditional Asian-style building with layered roofs and wooden accents.
Old traditional Asian-style building with curved tiled roofs and weathered wooden walls isolated on white background.
Ancient wooden Asian-style house partially submerged in still water with detailed carvings and a small rock in the foreground.
Model of a partially destroyed traditional Asian-style building with visible wooden beams, crumbling walls, glowing embers, and smoke rising from the structure.
Futuristic, densely-packed multi-level building combining traditional and modern architecture styles.
Black ink sketch of a traditional two-story Japanese house with tiled roof and detailed wooden features.
Abstract symmetrical digital artwork resembling a pair of traditional Asian-style buildings with intricate wood-like textures on a white background.
Intricately detailed miniature of a traditional East Asian wooden house with ornate carvings and glowing lanterns.
Intricate digital rendering of an ancient Asian-style building with layered rooftops and detailed textures on a white background.
Intricately detailed ancient Chinese-style temple with a flowing water feature and green vegetation around it.
Ancient temple engulfed in flames and smoke, partially damaged with glowing fire inside.
Futuristic, heavily mechanized building combining traditional Asian architectural elements with large mechanical gun turrets.
Act III: ML Art

artificial nightmares 4K >

Experimenting with PYTTI TOOl's 3D Animation

Overview >_

Act III of Synthetic Dystopia incorporates collections of ML artworks in either image or animation format. The image collection consists of two styles of artwork:
‍
1. Graphic depictions of imageries of the decayed urban environment.
2. Abstract representations and symbols for the deep irony in the false grandiosity of the human civilization and its silent decay.

Technical Detail >

Model: Disco Diffusion, PYTTI Tools

Program: Disco Diffusion v5.2 Colab Notebook by Katherine Crowson, PYTTI Colab Notebook by sportsracer48

Urbicide (Pink Floyd - Speak To Me / Breathe) 4K >

Experimenting with Disco Diffusion 3D Animation
Inspired by HuemanInstrumenti

The Hollow men (T.S. Eliot - The Hollow Men) 4K >

Stroyboard & experimentations

Dark surreal landscape filled with humanoid scarecrow figures made of tattered material under a cloudy sky and rocky hills in the background.
Black and white surreal artwork featuring three distorted human faces intertwined with twigs and branches.
Surreal grayscale scene with distorted humanoid figures in a nightmarish composition, featuring a large head, skeletal limbs, and shadowy entities.
Surreal dark artwork showing grotesque humanoid figures with elongated limbs confronting each other across a cracked mirror.
Dark, surreal painting of distorted humanoid figures in a grim, eerie scene with a large haunting face on the right.
Dark surreal scene with mysterious cloaked figures and a large cracked mask-like face blending into the textured background.
Dark, surreal scene of distorted humanoid figures in tattered clothing gathered in a dimly lit space, evoking a nightmarish atmosphere.
Close-up of a dark, fantastical eye surrounded by textured, glossy tentacles and spiral shell-like structures.
Abstract organic structure resembling intertwined tentacles with glowing, eye-like orbs and a central bright light.
Ancient column supporting a swirling wooden vortex with a glowing galaxy at its center.
Surreal scene with a massive eye-shaped wooden structure floating above a smoky, fiery landscape with small humanoid figures beneath it.
Surreal figures with skeletal faces and glowing eyes emerging from swirling, cloud-like textures.
Dark surreal scene of ghostly, hooded figures with tangled gray hair and haunting, distorted faces blending into a misty background.
Dark, surreal scene with numerous robed figures gathered around a fiery, volcanic landscape with glowing lava and smoky skies.
Surreal scene with towering, abstract humanoid figures and a crowd of smaller figures in a smoky, orange-lit environment.
Surreal scene with towering humanoid figures overlooking a crowded gathering in a barren landscape under a cloudy sky.
Crowd of distorted humanoid figures standing in a dark, surreal wasteland with tall, eerie structures under a cloudy sky.
Surreal desert scene with humanoid cacti figures having eerie faces, twisted branches, and a misty background.
Surreal landscape with large skull-like structures featuring mechanical, spiral eyes and eerie humanoid figures scattered on a barren ground.
Surreal scene with large, decayed bird-like heads entwined with roots and branches, and distorted, elongated human limbs amidst a barren, smoky landscape.
Surreal, dark fantasy scene featuring grotesque, creature-like forms with tangled, root-like textures and large hollow eyes on a muted background.
Dark surreal artwork featuring three distorted and grotesque human-like faces with exaggerated wrinkles and deformed features on a textured background.
Post-apocalyptic scene with tall, dilapidated buildings and overgrown vegetation under a cloudy sky.
Dark, abandoned cityscape with tall, dilapidated buildings under a gloomy sky and scattered orange street lights.

Explaination

The ML animation The Hollow Men was based on T. S. Eliot’s poem of the same name. The “stuffed men” in the poem allude to the indifferent common folks, like scarecrows, with motions following the wind. I used the imageries of” stuffed men” from the poem to symbolize the people who echo others’ words, who fear speaking the truth, and in turn, preserve the emotional desensitization cycle.

Explaination

Several pieces of my work depict crowds of people, not in commotion but in acquiescence, as the city in the background burns and explodes and blood spills and all kinds of atrocities resume. This matches the end of the poem as the poet predicts: As the emotional desensitization grows and the crowd of indifferent people amass, and becomes a “kingdom”, then “the world would end not in a bang but in a whimper”.

Final Video

Synthetic Dystopia VR Virtual Gallery >

Unreal Engine RealTime Gameplay >

Screenshots >

Modern art gallery with wooden floor displaying dark, surreal paintings on both walls and illuminated square artwork at the far end.
Modern art gallery with wooden floor, white walls, black-and-white photographic portraits, architectural images, and white benches.
Modern art gallery with polished wooden floor, white walls, and various dark, surreal paintings on display.
Modern art gallery room with wooden floors, white benches, and surreal dark-toned paintings displayed on walls.