Back to the Past and Present
A creative machine learning project created as the final project for Machine Learning for Artists and Designers (Fall 2024) at NYU Shanghai. The work blends AI-generated imagery, coded prompts, and physical computing to explore nostalgia, emotion, and the relationship between technology, memory, and time.
Developed Using
p5.js, ComfyUI
Materials
CNC-carved 5mm wooden boards, laser-cut acrylic, rotary encoders with knobs, Arduino, iPad (display), machine learning–generated imagery
Instructor
Gottfried Haider
System Flow
Physical Input (Arduino)
Two rotary encoders send continuous values via serial communication
Encoder 1: Temporal context (1940s → 2020s)
Encoder 2: Emotional mood (warm, dreamy, melancholic, nostalgic, etc.)
↓
p5.js Data Mapping
Raw encoder values are mapped to predefined time-based keywords and mood descriptors
Selected decade + emotional tone are combined into a structured text prompt
↓
Prompt Construction
p5.js dynamically generates a prompt describing:
Era-specific objects and aesthetics
Mood-based lighting, color, and atmosphere
Prompt updates continuously based on physical interaction
↓
AI Image Generation (ComfyUI)
The prompt is sent to ComfyUI using a predefined workflow
A Stable Diffusion (SDXL) pipeline processes the prompt:
CLIP text encoding
Sampling and latent image generation
Image decoding and compositing
↓
Output & Display
Generated images are returned to p5.js and displayed on screen
Images refresh at timed intervals, creating an evolving visual narrative
