quaintitative

I write about my quantitative explorations in visualisation, data science, machine and deep learning here, as well as other random musings.

For more about me and my other interests, visit playgrd or socials below


Categories
Subscribe

Setting Up a Data Lab Environment - Part 1 - I ❤️ Docker

The reason I started using Docker was because I was trying to run Jupyter notebooks on Amazon Web Services (AWS). As that’s still something useful to know if all you want is something lightweight, and without the added steps of needing to install Docker, I’ve covered that in 2 previous posts:

It’s a lot easier to do these. But to just use Docker for the two tasks above is frankly a waste.

So we what we will do in this series is to get a -

Docker So let’s go through some basics in Docker first.

I won’t go through the full extent of what it can do or how to install Docker (just Google!), but will just run through 2-3 key commands that will help get you an understanding of what Docker does, and get you through the series.

What Docker does is quite simple. It creates a container with a barebones OS, and adds layers of software on top, depending on what you need.

Each combination of OS and software is called an image. You can use images built by others, or roll your own. Or roll your own starting with an image built by someone else.

So once you have an image, all you need to do is first get the image by pulling it.

docker pull jupyter/minimal-notebook

Then run it. The -p flag is essential to link port 8888 on your own computer to the port 8888 in the Docker container.

docker run -p 8888:8888 jupyter/minimal-notebook

When you do this, the terminal will spurt out a series of logs, and you will not be able to do anything further in the terminal. To have Docker run in a detached mode, which will allow you to continue on in the terminal, just add a -d flag.

docker run -p -d 8888:8888 jupyter/minimal-notebook

Now you have an instance of a jupyter notebook running in a container, which you can access the usual way - localhost:8888.

Usually what we would do would be to install a base installation of Anaconda, and start adding on other libraries and packages as and when needed.

With Docker, we don’t really have to do this. There are pre-defined images available that you can use, as seen in the image below which I took from here.

For more of the most common docker commands, I find this to be a good cheat sheet.


Articles

AI and UIs
Listing NFTs
Extracting and Processing Wikidata datasets
Extracting and Processing Google Trends data
Extracting and Processing Reddit datasets from PushShift
Extracting and Processing GDELT GKG datasets from BigQuery
Some notes relating to Machine Learning
Some notes relating to Python
Using CCapture.js library with p5.js and three.js
Introduction to PoseNet with three.js
Topic Modelling
Three.js Series - Manipulating vertices in three.js
Three.js Series - Music and three.js
Three.js Series - Simple primer on three.js
HTML Scraping 101
(Almost) The Simplest Server Ever
Tweening in p5.js
Logistic Regression Classification in plain ole Javascript
Introduction to Machine Learning Right Inside the Browser
Nature and Math - Particle Swarm Optimisation
Growing a network garden in D3
Data Analytics with Blender
The Nature of Code Ported to Three.js
Primer on Generative Art in Blender
How normal are you? Checking distributional assumptions.
Monte Carlo Simulation of Value at Risk in Python
Measuring Expected Shortfall in Python
Style Transfer X Generative Art
Measuring Market Risk in Python
Simple charts | crossfilter.js and dc.js
d3.js vs. p5.js for visualisation
Portfolio Optimisation with Tensorflow and D3 Dashboard
Setting Up a Data Lab Environment - Part 6
Setting Up a Data Lab Environment - Part 5
Setting Up a Data Lab Environment - Part 4
Setting Up a Data Lab Environment - Part 3
Setting Up a Data Lab Environment - Part 2
Setting Up a Data Lab Environment - Part 1
Generating a Strange Attractor in three.js
(Almost) All the Most Common Machine Learning Algorithms in Javascript
3 Days of Hand Coding Visualisations - Day 3
3 Days of Hand Coding Visualisations - Day 2
3 Days of Hand Coding Visualisations - Day 1
3 Days of Hand Coding Visualisations - Introduction