Recently for a project, I needed to build a Python, Django, PostgreSQL, NGINX Development Virtual Machine(VM). Below are the steps that I followed to build this VM in VMware Fusion on MacbookPro (MBP). This post is as much about sharing my build experience as documenting the steps for my future use and potential automation.
The guidance for this procedure came from How To Install and Configure Django with Postgres, Nginx, and Gunicorn.
Installation
Ubuntu Server OS
The steps were very similar to the ones I covered in my prior post OpenStack: Quick Install using DevStack.
Download Ubuntu 12.04 "Precise Pangolin" x86_64 Minimal CD ISO Image mini.iso.
Start VMware Fusion and select Virtual Machine Library in Windows option on VMware Fusion toolbar. This will bring up Virtual Machine Library window showing all the Virtual Machine already available.
Click
Add button and select New. This will bring up New Virtual Assistance Window showing Create New Virtual Machine.Click Continue without disc as we will be using the downloaded ISO image. This will bring up Installation Media section.
Select Use operating system installation disc or image and click on arrows next to Choose a disc or disc image.... Select the Ubuntu ISO image and then click Continue.
Choose Operating System section should show Linux as Operating System and Ubuntu 64-bit as Version. Click Continue.
The Finish section will show Virtual Machine Summary, Click Finish. Select the location where we want to save the VM file and name the file.
A console window will be launched and OS install will start. Answered the prompts during the install process. Once update and reboot completes, a login prompt will appear. Log in to VM.
OpenSSH Server
After login, install OpenSSH Server to enable access to Ubuntu VM over SSH.
$sudo apt-get install openssh-server
Check whether SSH process is running.
$service ssh status
Either note down the IP address of VM from login screen (shown above) or using ifconfig command to be able to SSH into the VM remotely.
$ssh anil@172.16.191.158
You may need to remove SSH key if there is a fingerprint mismatch between the VM and remote client.
$ssh-keygen -R 172.16.191.158
Update Packages
You need to make sure all installed packages are current. Download any package updates and install.
anil@django:~$ sudo apt-get update [sudo] password for anil: Hit http://us.archive.ubuntu.com precise Release.gpg Get:1 http://us.archive.ubuntu.com precise-updates Release.gpg [198 B] ...
At this point VM is ready for installation of Python Virtualenv, Django, PostgreSQL, NGINX, and Gunicorn.
Python Virtualenv
Virtualenv is Virtual Python Environment builder to create separate Python environments. This enables to keep installations, dependencies, versions and permissions separate for different applications across different virtual environments.
Install python-virtualenv.
anil@django:~$ sudo apt-get install python-virtualenv Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: python-pip python-setuptools The following NEW packages will be installed: python-pip python-setuptools python-virtualenv ...
Now, we need to create a virtual environment for our project (in this case, lendcafe) where we can install Python and Django packages.
anil@django:~$ sudo virtualenv /opt/lendcafe New python executable in /opt/lendcafe/bin/python Installing distribute...........................................................done. Installing pip...............done.
You can name anything you like for your virtualenv.
Django
Django is a Python web framework. It enables rapid development of common web application tasks and adheres to DRY principle (Don't Repeat Yourself).
To install Django, first we need to activate virtualenv.
anil@django:~$ source /opt/lendcafe/bin/activate (lendcafe)anil@django:~$
The activate script modifies shell prompt to show the currently active environment.
Install Django
(lendcafe)anil@django:~$ sudo pip install django [sudo] password for anil: Downloading/unpacking django Downloading Django-1.6.2.tar.gz (6.6Mb): 6.6Mb downloaded Running setup.py egg_info for package django warning: no previously-included files matching '__pycache__' found under directory '*' warning: no previously-included files matching '*.py[co]' found under directory '*' Installing collected packages: django Running setup.py install for django changing mode of build/scripts-2.7/django-admin.py from 644 to 755 warning: no previously-included files matching '__pycache__' found under directory '*' warning: no previously-included files matching '*.py[co]' found under directory '*' changing mode of /usr/local/bin/django-admin.py to 755 Successfully installed django Cleaning up...
PostgreSQL
PostgreSQL is an open source object-relational database.
Deactivate the virtual environment.
(lendcafe)anil@django:~$ deactivate anil@django:~$
Install Python dependencies for PostgreSQL
anil@django:~$ sudo apt-get install libpq-dev python-dev Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: ...
Install PostgreSQL
anil@django:~$ sudo apt-get install postgresql postgresql-contrib Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: ...
NGINX
NGINX is an open source HTTP server and reverse proxy. It is known for high performance and low resource utilization. Instead of relying on threads to handle requests, it uses event-driven asynchronous architecture.
Install NGINX
anil@django:~$ sudo apt-get install nginx Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: ...
Gunicorn
Gunicorn is a Python WSGI HTTP Server.
Activate virtualenv
anil@django:~$ source /opt/lendcafe/bin/activate (lendcafe)anil@django:~$
Install Gunicorn within virtualenv
(lendcafe)anil@django:~$pip install gunicorn Downloading/unpacking gunicorn Downloading gunicorn-18.0.tar.gz (366Kb): 366Kb downloaded Running setup.py egg_info for package gunicorn ...
Configuration
PostgreSQL
The default superuser for PostgreSQL is called postgres. We need to login as this user first.
anil@django:~$ sudo su - postgres [sudo] password for anil: postgres@django:~$
The shell prompt should now show starting with postgres@....
Create a user. Answer the prompts. I decided to not make this user superuser, allow this new user to create databases or new roles.
postgres@django:~$ createuser -P Enter name of role to add: anil Enter password for new role: Enter it again: Shall the new role be a superuser? (y/n) n Shall the new role be allowed to create databases? (y/n) n Shall the new role be allowed to create more new roles? (y/n) n
Create a new database. I named this database to be lendcafe.
postgres@django:~$ createdb lendcafe
To grant new user access to this database, first access the PostgreSQL interactive terminal and then grant all privileges. Type \q to quit.
postgres@django:~$ psql psql (9.1.11) Type "help" for help. postgres=# GRANT ALL PRIVILEGES ON DATABASE lendcafe TO anil; GRANT postgres=# \q postgres@django:~$
Test the new user log in to database. If you are not already logged in the system as new user, you will get error as shown below. Login as new user or use su command.
postgres@django:~$ psql -d lendcafe -U anil psql: FATAL: Peer authentication failed for user "anil" postgres@django:~$ su - anil Password: anil@django:~$ psql -d lendcafe -U anil psql (9.1.11) Type "help" for help. lendcafe=> lendcafe=> \q anil@django:~$
Django
To create a Django project, first switch to the virtualenv directory created during installation and activate the virtualenv.
anil@django:~$ cd /opt/lendcafe anil@django:/opt/lendcafe$ source /opt/lendcafe/bin/activate (lendcafe)anil@django:/opt/lendcafe$
Start a new Django project. If you receive permission denied error as shown below, change the ownership of your environment directory. Then start a new Django project again. Check to make sure the project directory was created in virtualenv directory.
(lendcafe)anil@django:/opt/lendcafe$ django-admin.py startproject lcproject CommandError: [Errno 13] Permission denied: '/opt/lendcafe/lcproject' (lendcafe)anil@django:/opt/lendcafe$ sudo chown -R anil:anil /opt/lendcafe [sudo] password for anil: (lendcafe)anil@django:/opt/lendcafe$ django-admin.py startproject lcproject (lendcafe)anil@django:/opt/lendcafe$ ls -la total 28 drwxr-xr-x 7 anil anil 4096 Feb 21 11:52 . drwxr-xr-x 3 root root 4096 Feb 20 10:37 .. drwxr-xr-x 2 anil anil 4096 Feb 20 10:37 bin drwxr-xr-x 2 anil anil 4096 Feb 20 10:37 include drwxrwxr-x 3 anil anil 4096 Feb 21 11:52 lcproject drwxr-xr-x 3 anil anil 4096 Feb 20 10:37 lib drwxr-xr-x 2 anil anil 4096 Feb 20 10:37 local (lendcafe)anil@django:/opt/lendcafe$
For Django to be able to communicate with PosgreSQL database, we need to install PostgresSQL adapter for the Python Psycopg.
(lendcafe)anil@django:/opt/lendcafe$ pip install psycopg2 Downloading/unpacking psycopg2 Downloading psycopg2-2.5.2.tar.gz (685Kb): 685Kb downloaded Running setup.py egg_info for package psycopg2 Installing collected packages: psycopg2 ... Successfully installed psycopg2 Cleaning up... (lendcafe)anil@django:/opt/lendcafe$
Edit the setting.py file in directory and subdirectory named same as your project name.
(lendcafe)anil@django:/opt/lendcafe$ nano lcproject/lcproject/settings.py DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'lendcafe', 'USER': 'anil', 'PASSWORD': 'myPassword', 'HOST': 'localhost', 'PORT': '', } }
Run the following command to add Django specific database tables and configuration to database. I received the following error message when I tried to run the command.
(lendcafe)anil@django:/opt/lendcafe$ python lcproject/manage.py syncdb Traceback (most recent call last): File "lcproject/manage.py", line 8, in <module> from django.core.management import execute_from_command_line ImportError: No module named django.core.management
The StackOverflow discussion django import error - No module named core.management provides potential solutions to this error.
My issue turned out that I originally installed Django as root user while I was trying to run the above command as different user. When I ran the above command with sudo, I encountered a different error.
(lendcafe)anil@django:/opt/lendcafe/lcproject$ sudo python manage.py syncdb Traceback (most recent call last): File "manage.py", line 11, in <module> ... File "/usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 25, in <module> raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e) django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2
At this point, I decided to reinstall Django, this time without sudo.
(lendcafe)anil@django:/$ pip install django Downloading/unpacking django Downloading Django-1.6.2.tar.gz (6.6Mb): 6.6Mb downloaded Running setup.py egg_info for package django warning: no previously-included files matching '__pycache__' found under directory '*' warning: no previously-included files matching '*.py[co]' found under directory '*' Installing collected packages: django Running setup.py install for django changing mode of build/scripts-2.7/django-admin.py from 664 to 775 warning: no previously-included files matching '__pycache__' found under directory '*' warning: no previously-included files matching '*.py[co]' found under directory '*' changing mode of /opt/lendcafe/bin/django-admin.py to 775 Successfully installed django Cleaning up...
Once I reinstalled Django, I was able to successfully add Django specific database tables.
(lendcafe)anil@django:/opt/lendcafe/lcproject$ python manage.py syncdb Creating tables ... Creating table django_admin_log Creating table auth_permission Creating table auth_group_permissions Creating table auth_group Creating table auth_user_groups Creating table auth_user_user_permissions Creating table auth_user Creating table django_content_type Creating table django_session You just installed Django's auth system, which means you don't have any superusers defined. Would you like to create one now? (yes/no): yes Username (leave blank to use 'anil'): Email address: xyz@example.com Password: Password (again): Superuser created successfully. Installing custom SQL ... Installing indexes ... Installed 0 object(s) from 0 fixture(s) (lendcafe)anil@django:/opt/lendcafe/lcproject$
Gunicorn
Create a gunicorn_config.py file in /opt/lendcafe directory and add following entries in the file.
(lendcafe)anil@django:/opt/lendcafe$ nano gunicorn_config.py command = '/opt/lendcafe/bin/gunicorn' pythonpath = '/opt/lendcafe/lcproject' bind = '127.0.0.1:8001' workers = 5 user = 'anil'
Use the following command to run the server.
(lendcafe)anil@django:/opt/lendcafe$ /opt/lendcafe/bin/gunicorn -c /opt/lendcafe/gunicorn_config.py lcproject.wsgi 2014-02-21 15:41:13 [2892] [INFO] Starting gunicorn 18.0 2014-02-21 15:41:13 [2892] [INFO] Listening at: http://127.0.0.1:8001 (2892) 2014-02-21 15:41:13 [2892] [INFO] Using worker: sync 2014-02-21 15:41:13 [2897] [INFO] Booting worker with pid: 2897 2014-02-21 15:41:13 [2898] [INFO] Booting worker with pid: 2898 2014-02-21 15:41:13 [2899] [INFO] Booting worker with pid: 2899 2014-02-21 15:41:13 [2900] [INFO] Booting worker with pid: 2900 2014-02-21 15:41:13 [2901] [INFO] Booting worker with pid: 2901
Background the process by using ctrl + z and then typing bg followed by Enter. supervisord and screen can be used to manage Gunicorn start/restart. Refer to How to Install and Manage Supervisor on Ubuntu and Debian VPS for more information.
Enter the IP address and port specified in gunicorn_config.py file in a browser's address bar. If you see the message It worked! Congratulations on your first Django-powered page., you are good to go.
NGINX
Django settings for the project are stored in settings.py file. As NGINX will be used for static files, modify the location of STATIC_URL in /opt/lendcafe/lcproject/lcproject/settings.py file.
(lendcafe)anil@django:/opt/lendcafe/lcproject/lcproject$ nano settings.py # Static files (CSS, JavaScript, Images) # https://docs.djangoproject.com/en/1.6/howto/static-files/ STATIC_URL = '/opt/lendcafe/static/'
Create a subdirectory static in /opt/lendcafe/ directory.
(lendcafe)anil@django:/opt/lendcafe$ mkdir static (lendcafe)anil@django:/opt/lendcafe$ ls -la total 36 drwxr-xr-x 8 anil anil 4096 Feb 22 15:26 . drwxr-xr-x 3 root root 4096 Feb 20 10:37 .. drwxr-xr-x 2 anil anil 4096 Feb 21 15:28 bin -rw-rw-r-- 1 anil anil 133 Feb 21 15:41 gunicorn_config.py drwxr-xr-x 2 anil anil 4096 Feb 20 10:37 include drwxrwxr-x 3 anil anil 4096 Feb 21 11:52 lcproject drwxr-xr-x 3 anil anil 4096 Feb 20 10:37 lib drwxr-xr-x 2 anil anil 4096 Feb 20 10:37 local drwxrwxr-x 2 anil anil 4096 Feb 22 15:26 static (lendcafe)anil@django:/opt/lendcafe$
NGINX is a reverse proxy and HTTP server. There are two blocks, Server-block and Location-blocks in configuration file that we will be primarily working with. The server-block is very similar to virtual host and location-block to URI. There is much more information available in NGINX Beginner's Guide.
Create a new NGINX config file lendcafe in /etc/nginx/sites-available/ directory and enter the following configuration information.
(lendcafe)anil@django:/opt/lendcafe$ sudo nano /etc/nginx/sites-available/lendcafe server { server_name localhost; location /static/ { alias /opt/lendcafe/static/; } location / { proxy_pass http://127.0.0.1:8001; proxy_set_header X-Forwarded-Host $server_name; proxy_set_header X-Real-IP $remote_addr; add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"'; } }
Create a symbolic link in /etc/nginx/sites-enabled directory to the NGINX configuration file lendcafe.
(lendcafe)anil@django:/etc/nginx/sites-enabled$ sudo ln -s /etc/nginx/sites-available/lendcafe
Delete the symbolic link default.
(lendcafe)anil@django:/etc/nginx/sites-enabled$ sudo rm default
Restart NGINX service to changes to take effect.
(lendcafe)anil@django:/etc/nginx/sites-enabled$ sudo service nginx restart Restarting nginx: nginx. (lendcafe)anil@django:/etc/nginx/sites-enabled$
VM is setup for Python/Django web development.
