Dockerizing Django with Numpy and Gunicorn
Hi all.
There are many tutorials about dockerizing Django on the web. But at the project I was working on, we used libraries like Numpy and Pillow, and alpine Linux cannot compile this because of missing dependencies.
Our Stack:
- Django
- Numpy
- Pillow
- Postgres
First of all, let's create basic Dockerfile
FROM python:3.6.4-alpine
ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["manage.py", "runserver"]
T works if you are not using c compiled libraries like Numpy.
The thing is, we need to add compilers to alpine. I think running these processes on one layer is better.
Entire Dependency install Layer :
RUN apk add --no-cache jpeg tiff-dev openjpeg-dev postgresql-dev && \
apk add --no-cache \
--virtual=.build-deps \
g++ zlib-dev freetype-dev lcms2-dev \
gcc libc-dev linux-headers tk-dev tcl-dev harfbuzz-dev \
fribidi-dev build-base py-pip rsyslog cython file binutils \
musl-dev python3-dev cython && \
ln -s locale.h /usr/include/xlocale.h && \
pip install --upgrade pip && \
pip install -r requirements.txt --no-cache-dir && \
rm -r /root/.cache && \
find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
rm /usr/include/xlocale.h && \
apk --purge del .build-deps
Compilers for Alpine
I installed some packages like g++
and linux-headers
virtual and cleaned them after pip install because I need these packages for postscripts of some dependencies, and size is the matter on the container.
-t, --virtual NAME Instead of adding all the packages to 'world', create a new
virtual package with the listed dependencies and add that
to 'world'; the actions of the command are easily reverted
by deleting the virtual package
That means those packages are not added to global packages when you install. So if I need GCC to compile a program, but once the program is compiled, I don't need GCC more. So you can remove this virtual package. You can remove your virtual install packages command like this.
apk add --virtual mypacks gcc vim
apk del mypacks
Postgres
I'm running Postgresql on my local machine, not Docker. And I need to connect it. You can use host.docker.internal this mapping is direct routes your machines localhost.
So I added this env variable
ENV DB_HOST=host.docker.internal
and used this on settings.py.
Gunicorn
I decided to use Gunicorn for running processed with n workers. So I added Gunicorn to my requirements.txt and added these two lines for running processes.
ENV WORKER_COUNT=2
CMD ["sh", "-c", "gunicorn --bind :8000 --workers ${WORKER_COUNT} project.wsgi:application"]
Using WORKER_COUNT
env variable for the default value. I'm overriding this value every time I need it.
Running
I decided to create a Makefile to run this container with n workers. So I created Makefile with this command.
docker-run:
docker run -e worker_count=${worker} -p 127.0.0.1:8000:8000 project${version}
Dockerfile
FROM python:3.6.4-alpine
ADD . /app
WORKDIR /app
# Need to Update at Settings.py
ENV DB_HOST=host.docker.internal
# Default Worker Count is 2
ENV WORKER_COUNT=2
# Install dependencies layer.
RUN apk add --no-cache postgresql \
jpeg tiff-dev openjpeg-dev \
libpq postgresql-libs postgresql-dev && \
apk add --no-cache \
--virtual=.build-deps \
g++ jpeg-dev zlib-dev freetype-dev lcms2-dev \
gcc libc-dev linux-headers tk-dev tcl-dev harfbuzz-dev \
fribidi-dev zlib-dev build-base py-pip rsyslog cython file binutils \
musl-dev python3-dev cython && \
ln -s locale.h /usr/include/xlocale.h && \
pip install --upgrade pip && \
pip install -r requirements.txt --no-cache-dir && \
rm -r /root/.cache && \
find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
rm /usr/include/xlocale.h && \
apk --purge del .build-deps
EXPOSE 8000
CMD ["sh", "-c", "gunicorn --bind :8000 --workers ${WORKER_COUNT} project.wsgi:application"]