This guide shows how to install Docker Desktop on Windows, set up a Docker Hub account, run Jupyter Notebook with PySpark, and build your own custom Docker images with Python scripts.

1. Install Docker Desktop on Windows

• Download Docker Desktop from Docker Desktop for Windows.
• Install and restart your system. Ensure WSL 2 or Hyper-V is enabled.

2. Verify Installation

docker --version
docker run hello-world

3. Create a Docker Hub Account

• Go to Docker Hub and sign up.
• You’ll use this account to pull and push images.

4. Pull a Python Base Image

docker pull python:3.11-slim

5. Build a Custom Docker Image with Python Script

Create a file named Dockerfile with the following content:

# Dockerfile
FROM python:3.13
COPY test_script.py /
CMD ["python", "./test_script.py"]

Create a file named test_script.py in the same folder:

# test_script.py
print("Hello from inside Docker!")
print("Python is working correctly in this container.")

Build and run the image:

docker build -t sthithapragna/test_image .
docker run sthithapragna/test_image

6. Explore Jupyter PySpark Notebook Image

Official Jupyter + PySpark image: PySpark Notebook on Docker Hub

docker pull jupyter/pyspark-notebook
docker images

7. Run Jupyter Notebook with PySpark

docker run -p 8888:8888 jupyter/pyspark-notebook:latest

Access the notebook in your browser using the token URL provided.

8. Map Local Files into Container

docker run -p 8888:8888 -v C:\Users\sthithapragna\spark\:/home/sthithapragna/work/ jupyter/pyspark-notebook:latest

9. Run with Bash (Interactive)

docker run -it -p 8888:8888 -v C:\Users\sthithapragna\spark\:/home/sthithapragna/work/ jupyter/pyspark-notebook:latest bash

10. Advanced Run (Extra Resources)

docker run -it --rm --shm-size=1g --ulimit memlock=-1 \
  -p 8888:8888 \
  -v C:\Users\sthithapragna\spark\:/home/sthithapragna/work/ \
  jupyter/pyspark-notebook:latest

You now have both custom Docker images (running test_script.py) and ready-to-use PySpark Jupyter notebooks running on Windows 🚀