Tuesday 22 July 2014

Running Crypt Server on a Mac via Server.app

(Revised for new Django 1.5 version of Crypt Server)
Crypt is software written by Graham Gilbert of pebble.it to provide a FileVault2 escrow solution. That is to provide a secure centralised store for FileVault2 recovery passwords. With this (or similar FileVault2 Escrow solution) the user or authorised administrator can generate the recovery key to get back into a FileVault2 protected machine should the user forget their original code or leave the company. See http://grahamgilbert.com/blog/2013/01/18/crypt-a-filevault-2-escrow-solution/

Crypt consists of two parts, a client part which firstly enforces the use of FileVault2 encryption on the computer and secondly stores the details for the recovery key in the matching Crypt Server. There are other similar solutions available but Crypt has the advantages of being free and not using any external hosted systems.

Whilst (obviously) the client part of Crypt is native Mac software designed to run on Macs, currently the only official documentation for the Crypt Server is aimed at running Crypt Server on a Ubuntu Linux Server. It is certainly possible to run the Crypt Server on an Ubuntu Server with Mac clients by following those instructions and you can even host the Ubuntu Server in a Virtual Machine running on a Mac, e.g. in VirtualBox. However some people might prefer to run Crypt Server natively on an existing OS X Server and this article therefore describes how to achieve this.

Firstly we need to look at the requirements for running Crypt Server -
Apache
Python
VirtualEnv
Django
mod_wsgi
Of these Apache is included on all Macs as is Python, Django can be easily installed but mod_wsgi is only included if you have Server.app installed on top of Lion or Mountain Lion. This article is only aimed at how to get Crypt Server working with Server.app. As Server.app is very cheap if your unwilling to pay even that modest sum then your on your own and perhaps should stick to using the free Ubuntu Server approach. So in terms of this article the full requirements will be -

OS X Lion or OS X Mountain Lion
Server.app
Apache
Python
VirtualEnv
Django
mod_wsgi

I will be differing from the normal Crypt Server install instructions so as to be able to integrate with Apple's Server.app approach. I will be including download links for the configuration files I had to create to achieve this.

Step 1.
Ensure you have Server.app installed and have run it at least once so it can configure itself. As is best practice your server should have a static IP address. The only service in Server.app we will require is the webserver service. We will configure this later. You can if you wish run other services and even other websites on the server you will be using. In order to be able to run multiple websites on the standard port 80, you need to have at least one extra DNS name pointing to this server. So if its main DNS name (A record) is server.example.com you would add an alias (CNAME) and pick a new name for that for example cryptserver.example.com this will allow using a website name of cryptserver.example.com for the website we will be using.

Note: I see no benefit to running Crypt Server on a (web) server accessible from the Internet. In fact I would suggest from a security point of view you should only run it on an internal private server. This does mean client machines will need to be setup either on the internal private network or via a VPN connection to the private internal network.

Step 2.
We will be installing various Django and other python modules. The standard Ubuntu instructions describe using apt-get and pip to install these modules. Neither of these commands is as standard part of OS X however we can install pip very easily by using the built-in 'easy_install' command. So the first command will be

sudo easy_install pip

We now have the pip command installed so we can now use it to install the other modules as follows. First we will install VirtualEnv which allocates the modules to an environment private for the use of Crypt to prevent conflicts with any other python based software which might use different versions of these modules.
(You can check to see if virtualenv is already installed by typing the command virtualenv -–version if it is already installed then you can skip this step.)
sudo pip install virtualenv
We will then create the environment for Crypt.
cd /usr/local
bash
(virtualenv prefers using the bash shell rather than the standard sh shell)
sudo virtualenv crypt_env
cd crypt_env
sudo source bin/activate
sudo pip install django==1.5.3
(The current version of Crypt Server is now written specifically for Django 1.5 and has not been tested with Django 1.6, the above command ensures that Django 1.5.3 is used.)
sudo pip install south
sudo pip install django-bootstrap_toolkit
Step 3.
We will now download the Crypt Server software. This is available here https://github.com/grahamgilbert/Crypt-Server this can in theory be downloaded using a GIT client however I chose to download the ZIP archive listed on the right-hand side and save having to install GIT on my server.

In order to have file paths matching (as much as possible) the original Ubuntu instructions and also to match the settings files I am providing you then need to expand the zip file and move/copy the resulting folder of files as follows. The zip file contains at the top a single folder called crypt-server-master and in that various subfolders. The folder crypt-server-master needs to be renamed crypt and moved into /usr/local/crypt_env/ therefore the path to crypt will become /usr/local/crypt_env/crypt/ I did this in Terminal.app.
Step 4.
We now need to create a wsgi file, the author provides an example one but strangely does not include it in the zip file, so here is one I prepared earlier crypt.wsgi :)

Note: Oops! Just discovered on 22nd July 2014 that the link above to the example crypt.wsgi was pointing to an old incorrect version, I have updated the link to point to a corrected version.

You need to download that file and copy it to /usr/local/crypt_env/crypt/crypt.wsgi
Note: Apple's Server.app will not process python webapps unless they use the file extension .wsgi if you try using the file extension .py they will not work.
Step 5.
We now need to create and configure the settings file for Crypt Server, first we copy the example file.
sudo cd /usr/local/crypt_env/crypt/fvserver/
sudo cp example_settings.py settings.py

We now need to edit settings.py pick your favourite commandline editor, e.g. nano, pico or vi.

You need to set the Administrator email details and the TimeZone for your server. This step is the same as the original Ubuntu instructions. You can therefore look at section 27 here http://derflounder.wordpress.com/2012/12/31/first-look-at-crypt/
Step 6.
We now need to generate the database that Crypt Server will use to store the accounts and FileVault2 recovery keys. To do this we use the following commands.
sudo cd /usr/local/crypt_env/crypt/
sudo python manage.py syncdb

These steps are again the same as the original Ubuntu instructions so you can look at section 28 here http://derflounder.wordpress.com/2012/12/31/first-look-at-crypt/

Note: The user account being created here is only used internally in the database it is not linked in anyway to Open Directory or any other OS X user account. It is used when you login to Crypt Server via a web-browser.

Then we do

sudo python manage.py migrate
sudo python manage.py collectstatic

Again this is the same as the standard Ubuntu instructions so see sections 29 and 30 here http://derflounder.wordpress.com/2012/12/31/first-look-at-crypt/

We have now in theory finished installing and setting up Crypt Server, the remaining steps will be integrating it into Apple's Server.app

Step 7.
Launch Server.app and go to the webserver service. Create a new website using the hostname you chose in step 1. Leave it using the standard port 80 and all IP addresses settings. Click on the Edit button next to Aliases and add a rule to map from a path of /static/ to a folder (any folder) we will be manually editing this later because Server.app does not let you browse to /usr/local where we need it to point to.

You should now quit Server.app for now, do not start the webservice yet. Next we want to manually edit the apache conf file corresponding to the website you have just created. This will be located in /Library/Server/Web/Config/apache2/sites/ it will have a name something like 0000_any_80_cryptserver.example.com.conf the exact name will depend on the host name you are using. You need to edit this in Terminal.app using your favourite editor. You want to set the line beginning with DocumentRoot (the fourth line typically) to the following

DocumentRoot "/usr/local/crypt_env/crypt"

You also want to set the line beginning with <Directory similarly as follows

<Directory "/usr/local/crypt_env/crypt">
Finally we want to edit the line beginning Alias /static/ it will be need the bottom of this file, change it to the following
Alias /static/ "/usr/local/crypt_env/crypt/static/"
We have to do this manually because the files for Crypt Server are not in the normal websites folder location, and because in Server.app you cannot browse and set the location to somewhere in /usr/local/ as this is 'hidden' from view.

Note: Fortunately my experience is that once this change is made manually, Server.app respects it and does not later overwrite it.

The standard Ubuntu instructions tell you to run the website with an additional user account setup specifically for it and that you need to add an additional command to the apache conf file you have just edited above. I could not get those instructions to work with Server.app but fortunately it is not necessary to do so. If you follow the instructions here the website will run successfully with the standard _www account. You do need however to set the ownership of the Crypt Server files to _www so that the standard account can access and modify the Crypt Server database. To do this issue the following command

sudo chown -R _www /usr/local/crypt_env

Step 8.
We now need to setup the extra config files to make the Crypt Server django webapp available as a webapp that will be listed in Server.app and this will allow us to have this webapp run when someone accesses this website. In this article I will merely tell you what to put where and how to then turn it on, but for more details on how you setup webapps in general with Server.app see my other article about this topic available here http://jelockwood.blogspot.co.uk/2013/06/running-django-webapps-with-os-x.html

You need to first place a file called com.crypt.webapp.wsgi.plist in /Library/Server/Web/Config/apache2/webapps/ here is a copy of com.crypt.webapp.wsgi.plist I have made for you. You also need to place a file called httpd_crypt.conf in /Library/Server/Web/Config/apache2/ here is a copy of httpd_crypt.conf for you to use.

You can now open Server.app again. Go to the webserver service and select the website you previously added in step 7 above. Edit the website by clicking on the pencil button, scroll down and click on the 'Edit Advanced Settings...' button. You should now see a list of available webapps, the one you want to enable (tick) is the 'FileVault Escrow Server'. This corresponds to the webapp that the two files you have just installed has defined and this will run the Crypt Server webapp when you access this website. Then click OK and then click Done. You can now start the websites service.


All being well you should now be able to access the Crypt Server in a web-browser at a URL like http://cryptserver.example.com/ depending on what hostname you are using.

16 comments:

  1. John followed this and I get a Bad Request 400 when trying to go to the website, only difference I have is Django 1.6

    ReplyDelete
    Replies
    1. Crypt is officially still only written for use with Django 1.4, I did as discussed above work out how to get it to work correctly with Django 1.5 by changing just two lines, I have not tried it with Django 1.6.

      I recommend you force the installation of an older version of Django by using a slightly different form of the command so try the following.

      sudo pip install django==1.4.3

      or

      sudo pip install django==1.5.1

      Delete
  2. John thanks I tried that but no luck. I have tried 4 times now and I can not get this working. Not sure if the instruction need revising or what. It looks like Graham has made some modifications to the program recently for 1.5 and now your process does not work. I have a feeling /usr/local/crypt_env/crypt/templates/base.html and /usr/local/crypt_env/crypt/server/templates/server/index.html need some additional modification with Django 1.5.3 as his install now suggests as the version needed.

    I really appreciate the help.

    ReplyDelete
    Replies
    1. I am going to download and install the new version you have pointed out. I will then post revised instructions. This will probably be in the form of a new page so this page may be deleted at that point (as being out of date). So keep an eye open for a possible new page. This may take a couple of days or so.

      Delete
    2. I have the latest Crypt Server working in the development Django web server with Django 1.5.3 but not yet in Apache. If you can send me a note via the contact form at the right of this page we can discuss this further privately via email.

      Delete
  3. Any update with this? I'm trying to configure crypt-server on my mac mini server with no luck

    ReplyDelete
    Replies
    1. I am currently running Crypt in a Linux virtual machine with Apache but (obviously) as per this article have previously got it working in OS X in Apache. I did modify this article in November to address some issues regarding installing for OS X.

      I would certainly expect you to have problems with Django if you have installed 1.6, the original version of Crypt was only written to work with Django 1.4 although I got round that, and as far as I am aware it now officially works with Django 1.5 but has not been officially tested with 1.6.

      I just checked it currently wants Django 1.5.3

      I therefore recommend you start again and make sure you install Django 1.5.3 and the versions of any other components it asks for rather than the latest versions. This is where using VirtualEnv helps.

      Crypt (server) comes with a file called requirements.txt which lists the versions it wants which are as follows

      Django==1.5.3
      South==0.8.2
      django-bootstrap-toolkit==2.15.0
      docutils==0.11
      wsgiref==0.1.2

      Either you can install the whole bunch using the command

      pip install -r setup/requirements.txt

      or you can do individual installs like

      pip install django==1.5.3

      Do this at the stage where I listed installing them to make sure you are doing it in the VirtualEnv (if you use that).

      If your still stuck after this send me a contact via the contact form with details of what is happening.

      Delete
  4. Finally got this to work on OS X 10.10.3. Thought I might report back for other readers to benefit from. What threw me off big time as a first time user turned out to be the logging configuration for Django. After attempting setup on a Linux VM as a last resort I finally found out by reading the Apache log that the web app crashes when the mail is not configured. As there's no smtp server available at this site, I set up logging to file instead. Maybe I'll setup an smtp service and configure it in Django to get mail alerts later. As this is a small scale setup, I did't want the overhead of supporting multiple servers for Mac deployment, hence the setup on the Apple server.

    Here's what I had to change at the end of the LOGGING section in /usr/local/crypt_env/crypt/fvserver/settings.py to:
    'handlers': {
    'mail_admins': {
    'level': 'ERROR',
    'filters': ['require_debug_false'],
    'class': 'django.utils.log.AdminEmailHandler'
    },
    'file': {
    'level': 'ERROR',
    'filters': ['require_debug_false'],
    'class': 'logging.FileHandler',
    'filename': '/var/log/django/crypt-server-error.log',
    }
    },
    'loggers': {
    'django.request': {
    'handlers': ['file'],
    'level': 'ERROR',
    'propagate': True,
    },
    }
    }


    To get the permissions for the log file access to work I ran a
    chmod -R 765 /var/log/django
    I also had to insert a couple of extra lines in the Apache config file for the virtual host. At the start I put in WSGIDaemonProcess _www user=_www group=wheel and then at the start of the section I put in a line
    WSGIProcessGroup _www

    Otherwise in step 2 I had to make some adjustments. Firstly the sudo source bin/activate throws an error. I got around it by turning the line for bash into sudo bash, and skipping the sudo part for the other commands in step 2. Remember to run the chown -R _www /usr/local/crypt_env as root afterwards. As there is a proxy at this site which is dealt with through environment variables, I added the -E switch to sudo to invoke bash as well at the start of step 2.

    Hopes this helps someone that comes looking here.

    ReplyDelete
    Replies
    1. Thanks for your information. I have not personally tried Django with Yosemite as I now use Linux VMs myself, however back when I was using Django webapps like Crypt with Mavericks I did not have any problems with not having mail setup. Nor do I currently have problems with the same issue in my Linux VM.

      Delete
    2. Both the installation on the OS X 10.10 server I did following your instructions and the and on the CentOS 7 minimal install I did following Grahams own recipe had the issue at first with the website throwing an Internal server error, and apache error logs showing smtp connection related errors triggered by the crypt.wsgi.
      (the wgsi script does get installed automatically if you clone using git by the way).
      I got the issue resolved on both installs by changing the logging section of settings.py to log to file instead of email, but reverting those sections to email logging like in the example file doesn't produce the issue again so far, but I have yet to test with client Macs.

      Delete
  5. PS - Used the combined pip command in step 2 in order to install all the requirements at once
    pip install -r setup/requirements.txt

    ReplyDelete
  6. Hi,

    Not sure if this is still being monitored.

    Trying to setup Crypt on a Sierra machine and getting this error when running sudo python manage.py migrate from step 6


    UnknownMigration: Migration 'server:0001_initial' probably doesn't exist.
    Traceback (most recent call last):
    File "/usr/local/crypt_env/lib/python2.7/site-packages/south/migration/base.py", line 302, in migration
    migration = __import__(full_name, {}, {}, ['Migration'])
    File "/usr/local/crypt_env/crypt/server/migrations/0001_initial.py", line 4, in
    from django.db import models, migrations
    ImportError: cannot import name migrations

    Your help would be greatly appreciated

    ReplyDelete
    Replies
    1. As per Mart's comment I also long ago gave up on using a Mac and switched to running Crypt in a Linux VM. Also this article was written with Crypt v1 and Crypt v2 is considerably different.

      Delete
  7. Hi Chris,
    Just a hunch: could be the crypt.db file doesn’t exist, you could try creating an empty one first by running
    touch crypt.db
    Strongly recommend moving to a Linux setup and docker though. I got it to run on on Yosemite before, but had to abandon it after upgrading to Sierra a while ago and recently attempting to update to latest Crypt-server release. Just couldn’t get it to work anymore, let alone run stable on macOS server’ Apache.

    ReplyDelete
    Replies
    1. Thanks for this.

      Just ran through the setup again using an Ubuntu 17 server and got the same error.

      I am new to python so completely lost.

      Delete