Machine Learning PARTICLE SYSTEM

ML Data Sculpture & VR Experience
Category
Machine Learning
Location
NYU IMA, New York, Us
YEAR
2021/12

Statement

This is an experimental project investigating on the fusion between machine learning and immersive experience. The data sculpture is essentially a Niagara particle system in Unreal Engine driven by StyleGan latent spacewalk Video. This project is my first step into the realm of machine learning, which laid the groundwork for my Capstone project “Synthetic Dystopia”.

Credit

Created by Damon Chen
Inspired by: Refik Anadol, Yuma Yanagisawa, AvantContra
Project Summary

overview >

This project is a way to generate digital art with the help of StyleGan and machine learning algorithms. I am using the Niagara particle system in Unreal Engine with the particles mapping the position and color of the StyleGan latent spacewalk video I generated from Runway.

Intention >

I believe each picture generated by StyleGan or any machine learning algorithm could be a unique artwork or texture. I built three scenes to demonstrate how these machine-generated images with distinctive styles and subject matters with some simple touch could be transformed into powerful and dramatic artworks.

Future Inprovement>

My original plan is to control the video input in UE, also the color and position of the particles, in real-time (using the google drive link streaming blueprint in UE). So the viewers could generate their own StyleGan video and their unique particle system directly.

StyleGan Latent Spacewalk Video from Runway >

Custom Trained texture model >

Runway GalaxyGan model >

Runway Portraitgan model >

ML Data sculpture in Unity >

UE Realtime Gameplay >