March 18, 2022

Dockerizing Django with Numpy and Gunicorn

Hi all.

There are many tutorials about dockerizing Django on the web. But at the project I was working on, we used libraries like Numpy and Pillow, and alpine Linux cannot compile this because of missing dependencies.

Our Stack:

First of all, let's create basic Dockerfile

FROM python:3.6.4-alpine
ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["manage.py", "runserver"]

T works if you are not using c compiled libraries like Numpy.
The thing is, we need to add compilers to alpine. I think running these processes on one layer is better.
Entire Dependency install Layer :

RUN apk add --no-cache jpeg tiff-dev openjpeg-dev postgresql-dev && \
    apk add --no-cache \
    --virtual=.build-deps \
    g++ zlib-dev freetype-dev lcms2-dev \
    gcc libc-dev linux-headers tk-dev tcl-dev harfbuzz-dev \
    fribidi-dev build-base py-pip rsyslog cython file binutils \
    musl-dev python3-dev cython && \
    ln -s locale.h /usr/include/xlocale.h && \
    pip install --upgrade pip && \
    pip install -r requirements.txt --no-cache-dir && \
    rm -r /root/.cache && \
    find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
    find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
    rm /usr/include/xlocale.h && \
    apk --purge del .build-deps

Compilers for Alpine

I installed some packages like g++ and linux-headers virtual and cleaned them after pip install because I need these packages for postscripts of some dependencies, and size is the matter on the container.

-t, --virtual NAME    Instead of adding all the packages to 'world', create a new 
                        virtual package with the listed dependencies and add that 
                        to 'world'; the actions of the command are easily reverted 
                        by deleting the virtual package

That means those packages are not added to global packages when you install. So if I need GCC to compile a program, but once the program is compiled, I don't need GCC more. So you can remove this virtual package. You can remove your virtual install packages command like this.

apk add --virtual mypacks gcc vim
apk del mypacks

Postgres

I'm running Postgresql on my local machine, not Docker. And I need to connect it. You can use host.docker.internal this mapping is direct routes your machines localhost.

So I added this env variable

ENV DB_HOST=host.docker.internal

and used this on settings.py.

Gunicorn

I decided to use Gunicorn for running processed with n workers. So I added Gunicorn to my requirements.txt and added these two lines for running processes.

ENV WORKER_COUNT=2
CMD ["sh", "-c", "gunicorn --bind :8000 --workers ${WORKER_COUNT} project.wsgi:application"]

Using WORKER_COUNT env variable for the default value. I'm overriding this value every time I need it.

Running

I decided to create a Makefile to run this container with n workers. So I created Makefile with this command.

docker-run:
 docker run -e worker_count=${worker} -p 127.0.0.1:8000:8000 project${version}

Dockerfile

FROM python:3.6.4-alpine
ADD . /app
WORKDIR /app
# Need to Update at Settings.py
ENV DB_HOST=host.docker.internal
# Default Worker Count is 2
ENV WORKER_COUNT=2
# Install dependencies layer.
RUN apk add --no-cache postgresql \
                        jpeg tiff-dev openjpeg-dev \
                        libpq postgresql-libs postgresql-dev && \
    apk add --no-cache \
    --virtual=.build-deps \
    g++ jpeg-dev zlib-dev freetype-dev lcms2-dev \
    gcc libc-dev linux-headers tk-dev tcl-dev harfbuzz-dev \
    fribidi-dev zlib-dev build-base py-pip rsyslog cython file binutils \
    musl-dev python3-dev cython && \
    ln -s locale.h /usr/include/xlocale.h && \
    pip install --upgrade pip && \
    pip install -r requirements.txt --no-cache-dir && \ 
    rm -r /root/.cache && \
    find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
    find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
    rm /usr/include/xlocale.h && \
    apk --purge del .build-deps
EXPOSE 8000
CMD ["sh", "-c", "gunicorn --bind :8000 --workers ${WORKER_COUNT} project.wsgi:application"]