Deep Learning Art

I develop new approaches to use Deep Learning in creative and useful ways. Often I use unconventional methods such as Deep Neural Cellular Automata or Modern Hopfield Networks to create images or develop methods for image modification.

Deep Image Prior using fractal Automata

Deep Image Prior using Deep Cellular Automata. The network consists of an encoder that creates a seed from a fixed noise vector. From there I alternate automata and upsampling steps, like in my “fractal”-Automata. The network is trained to only create one image as in “Deep Image Prior” by Dymitry Ulyanov. The network’s inductive bias guides the look of the art. In the case of using the same deep cellular automata update step followed by upsampling in an alternative fashion, this inductive bias is a fractal and recursive look.

Deep Neural Cellular Automata for Style Transfer

This method developed by me uses CLIP guidance or style transfer to train cellular automata to transform images to the desired style. The cellular automata can be imaged to “live” on the image and transform it in an iterative way. Contrary to conventional deep learning methods this work is very sparse on parameters, only a few thousand parameters in the form of the cellular automata update rule can be used to learn stylistic transformations that work on images outside the training set.

StyleGan3 Latent Optimization

Here the latent values of StyleGan3 were optimized to fit a texture and content loss. The results turned out to be surprisingly interesting for such a simple optimization as the model, which was only trained on faces, tries to fit a certain texture.