quaintitative

I write about my quantitative explorations in visualisation, data science, machine and deep learning here, as well as other random musings.

For more about me and my other interests, visit playgrd or socials below


Categories
Subscribe

Style Transfer X Generative Art

Style transfer, where you transfer an artistic style to a selected image is great fun.

But I thought that I would go one step further. Instead of just applying style transfer to some boring photo or image, I thought I would try to dynamically apply it to art generated that is continuously generated in the browser. For the generative art piece, I used the piece I generated previously and covered here.

What I did was to capture the image from this piece every few seconds, and apply a style transfer model (trained on the Great Wave by Hokusai) on the generated art image.

The code here is pretty self explanatory for those who are interested.

I only had a little trouble with converting the captured image into something that the style transfer model could work with. This little code snippet does the trick.

function learn(){

    $('#outimage').empty();
    $('#styled').empty();
    $('#styled2').empty();
    saveFrames('_', '_', 1, 25, function(data) {
    // console.log(data[1].imageData);
    capturedImage = createImg(data[1].imageData, runModel);
    capturedImage.parent('outimage');
    });
}

First, I clear out all the images from previous runs of the style transfer model with jQuery’s empty() function. Then I use p5.js’ saveFrames function to capture the image on screen. This is in imageData (base64 binary) format so it won’t work if you just feed it into the style transfer mode. What I did was to first create a DOM image element from this binary. This can then be fed into the style transfer model, which is called using the runModel function.

function runModel(){
    console.log('Running model');
    style1.transfer(capturedImage, function(err, result) {
    createImg(result.src).parent('styled');
    createImg(result.src).parent('styled2');
    });
}

Check out the actual demo, running on p5.js and ml5.js, here.


Articles

AI and UIs
Listing NFTs
Extracting and Processing Wikidata datasets
Extracting and Processing Google Trends data
Extracting and Processing Reddit datasets from PushShift
Extracting and Processing GDELT GKG datasets from BigQuery
Some notes relating to Machine Learning
Some notes relating to Python
Using CCapture.js library with p5.js and three.js
Introduction to PoseNet with three.js
Topic Modelling
Three.js Series - Manipulating vertices in three.js
Three.js Series - Music and three.js
Three.js Series - Simple primer on three.js
HTML Scraping 101
(Almost) The Simplest Server Ever
Tweening in p5.js
Logistic Regression Classification in plain ole Javascript
Introduction to Machine Learning Right Inside the Browser
Nature and Math - Particle Swarm Optimisation
Growing a network garden in D3
Data Analytics with Blender
The Nature of Code Ported to Three.js
Primer on Generative Art in Blender
How normal are you? Checking distributional assumptions.
Monte Carlo Simulation of Value at Risk in Python
Measuring Expected Shortfall in Python
Style Transfer X Generative Art
Measuring Market Risk in Python
Simple charts | crossfilter.js and dc.js
d3.js vs. p5.js for visualisation
Portfolio Optimisation with Tensorflow and D3 Dashboard
Setting Up a Data Lab Environment - Part 6
Setting Up a Data Lab Environment - Part 5
Setting Up a Data Lab Environment - Part 4
Setting Up a Data Lab Environment - Part 3
Setting Up a Data Lab Environment - Part 2
Setting Up a Data Lab Environment - Part 1
Generating a Strange Attractor in three.js
(Almost) All the Most Common Machine Learning Algorithms in Javascript
3 Days of Hand Coding Visualisations - Day 3
3 Days of Hand Coding Visualisations - Day 2
3 Days of Hand Coding Visualisations - Day 1
3 Days of Hand Coding Visualisations - Introduction