Tuesday 5 November 2013

mod_xsendfile and OS X Server.app

mod_xsendfile is a small Apache2 module that processes X-SENDFILE headers.

It makes it much quicker to download large files from a webserver and allows them to be streamed directly from disk without having to be first read in to memory.

This module is not included in OS X as standard but is included in OS X Server.app. You might want to use this module with websites you have written yourself or with web systems you have downloaded and installed. I recently wanted to install and test the free open-source MunkiServer project which uses this module, I however encountered an issue doing so which I am documenting here so anyone else using OS X Server.app and mod_xsendfile can understand and solve if they also encounter it.

I initially found that the Apache web-server in Server.app was crashing repeatedly when I tried loading the MunkiServer web application. I eventually tracked this down to when MunkiServer tried loading mod_xsendfile and I determined it was when MunkiServer tried using the command

XSendFilePath /path/to/files

This command was added in mod_xsendfile version 0.10 it turns out that the version of mod_xsendfile included in Server.app does not include this command because it is an older version and therefore only supports the previous command of XSendFileAllowAbove instead. This command was no help for MunkiServer. The solution is to download the current version, compile it in Terminal using the command

apxs –cia mod_xsendfile.c

which requires you to have XCode installed and then configure Apache to load it instead of the version included in Server.app.

It seems the majority of MunkiServer users either don’t run it on a Mac or don’t run it using Server.app and either option means they typically always download the latest version and compile it.

You can download the latest mod_xsendfile from https://tn123.org/mod_xsendfile/

Sunday 28 July 2013

FileVault 2 Escrow Servers

What is FileVault?
FileVault is Apple’s solution for securing a users files by encrypting them. The original version of FileVault (FileVault 1) was introduced with Mac OS X Panther (10.3) and continued through to Mac OS X Snow Leopard (10.6). FileVault 1 worked by storing the users home directory inside an encrypted disk image file, the rest of the contents of the hard disk where not encrypted. Later on security was further improved with the introduction of ‘Secure Virtual Memory’ whereby the contents of virtual memory stored on the hard disk was also encrypted, it was still the case that the rest of the hard disk was not encrypted.
sp_accounts_security[1] 

FileVault 1 however had two major problems, firstly it had a reputation for reliability problems potentially losing all your personal files (unless you had a backup), and secondly because the entire hard disk was not encrypted it was possible for either the user to mistakenly store files outside their encrypted home directory, or for misbehaving applications to do so. As a result FileVault 1 was never accepted as being adequate for use by Governments or Enterprise customers especially in regulated industries like finance, law, and medicine. As a result Government and Enterprise customers would instead use products meeting the FIPS 140-2 security standard such as CheckPoint Full Disk Encryption, PGP Whole Disk Encryption, Sophos SafeGuard, or WinMagic SecureDoc Disk Encryption (all of which are available for both Mac and Windows computers).

Apple therefore with OS X Lion (10.7) introduced FileVault 2, this encrypts the entire hard disk like its competitors and can also encrypt external drives as well (for storing your backups). FileVault 2 in OS X Lion eventually gained FIPS 140-2 certification itself, and OS X Mountain  Lion also gained FIPS 140-2 certification in July 2013. FileVault 2 is regarded as being far more reliable than FileVault 1 and as it now encrypts the entire hard disk there is no danger of files accidently leaking outside the protected area.

What is Escrow?
With all encryption products you need to ensure you can still access the contents by knowing the correct security key. If you lose the key you lose the ability to access the files. Therefore most if not all such encryption products provide a means to generate a ‘recovery’ key if you lose your passcode either by a user being forgetful or a user leaving and you then wanting to gain access. FileVault 2 is no exception to this and Apple have provided such a mechanism. This is where the term Escrow comes in, a third-party stores (securely) the information needed to generate a recovery key. The rest of this article discusses the alternatives available to do this in-conjunction with Apple’s FileVault 2 software.

1. Using your Apple ID to store the recovery key
Many people may forget that Apple provide a means when you enable FileVault 2 to at the same time store your recovery key on Apple’s servers in your Apple ID account and this service is completely free of charge. This does count as an Escrow service with Apple acting as the third-party.
See http://support.apple.com/kb/ht4790
HT4790_StoreKey----en[1]
However some users may be unhappy with the fact another company is storing this information. It is also not designed to make it easy for an IT administrator to manage multiple computers.

2. Cauliflower Vest
This is free open source software written by Google. It allows setting up a central store of recovery keys with secure access making it much more suitable for an IT administrator to manage. It can also make the use of FileVault 2 compulsory ensuring the laptop is secure.
See http://google-opensource.blogspot.co.uk/2012/02/cauliflower-vest-end-to-end-os-x.html
See http://code.google.com/p/cauliflowervest/
1330528122_mgpic_final[1]
However it uses Google’s App Engine servers to store the information so again some users may not be happy with the thought someone else is storing their security keys.

3. Casper Suite
JAMF Software produce an extensive suite of management software for managing both Macs and iOS devices. This includes the ability to manage FileVault 2 both to enforce its use (like Cauliflower Vest) and to store the recover keys.
See http://www.jamfsw.com/solutions/filevault/
Unlike the previous two solutions as Casper Suite runs on your own servers you don’t have to worry about the possibility of a third-party having access to your security keys. This is however a commercial solution so you do have to buy the Casper Suite software and licenses.

4. Crypt Server
This is another free open source solution written this time by Graham Gilbert of pebble.it. It allows you to run your own server internally and securely store the recovery keys. It includes a matching client component so that like Casper Suite and Cauliflower Vest you can enforce the use of FileVault 2 encryption and automate the storing of the recovery keys.
See http://grahamgilbert.com/blog/2013/01/18/crypt-a-filevault-2-escrow-solution/
See https://github.com/grahamgilbert/Crypt-Server 
home[1]
Above is a page from the server web administration interface, below is what the client sees when they setup a computer.
Crypt-Screenshot[1]
Crypt Server was however originally written to run on a Linux Ubuntu Server. I have however worked out how to run it on an OS X Server using Apple’s Server.app software and instructions on how to do this are available here -
http://jelockwood.blogspot.co.uk/2013/07/running-crypt-server-on-mac-via.html

Monday 24 June 2013

Mountain Lion gets FIPS 140-2 approval at last

Mountain Lion was released in July 2012, its encryption code was effectively identical to that in Lion. As is usual Apple still had to apply for a new certification for Mountain Lion and did so. However the wheels obviously grind very slowly and it was only on June 14th 2013 that Mountain Lion finally received FIPS 140-2 certification.

This should now make it possible to use Apple's built-in FileVault2 encryption in organisations that require a FIPS 140-2 certified product. I myself have been using PGP instead due to this issue.

Anyone interested in doing this should visit the following two links

http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/1401val2013.htm (search for Apple)
http://support.apple.com/kb/ht5396

Sunday 23 June 2013

Using Apple Lossless (aka. ALAC) in Windows

Apple has without a doubt won the music wars, their iPods and now iOS devices are by far the most popular personal music players, and the iTunes Store is by far the most popular music download store. Even Microsoft has conceded defeat and discontinued their Zune Player and store. Furthermore when Windows 7 was launched Microsoft added built-in support for AAC format music files as popularised by Apple.

However many people want to use a lossless music format rather than a lossey format like MP3 or AAC and this is still an area that requires a bit of extra work. First some background.

Many years ago I decided to setup a Microsoft Media Center because back then and still now, it was far superior to any equivalent solution on the Mac. Yes this maybe a shock but it’s true :)

Elgato’s EyeTV does a great job for recording and playing back TV shows but that is all it does. FrontRow (when it existed) could playback files including your iTunes Music library but could not do TV recording. At the time PlexApp did not exist but even XBMC which PlexApp is based on also does not do live TV or record TV. While there are some equivalents to Microsoft Media Center like Myth, or MediaPortal, or SageTV they either did not work on the Mac at all, or had poor support for Mac compatible TV tuners, or were not as attractive as Microsoft Media Center. Gasp! Another shock, a piece of attractive Microsoft Software!

So I wanted to use Microsoft Media Center (running on a Mac of course via Boot Camp). However I wanted to have a single copy of my music and still be able to sync it to my iPod or iPhone which meant the music needed to be in a format compatible with those devices. On the Apple side, supported music formats are MP3, AAC, AIFF, WAV, and Apple Lossless. On the Windows side supported formats were MP3, WMA, WAV, and AIFF. However Apple software does not support tags in WAV files, and Windows Media Player does not support tags in AIFF. Also as you can see from that list Apple Lossless was not supported at all in WMP. I therefore began by looking for additional codecs for WMP to let it play Apple Lossless files. I rapidly found several codecs that supported AAC for WMP but after an extensive search found there was none at all for Apple Lossless. So initially I had to settle for using AAC which could be used on both Apple and Microsoft systems with full support for meta-tags.

I did not give up, I then decided to look for any Windows solutions that supported Apple Lossless in the hope one might be adapted to my needs, I then found that a plugin was available for dbPowerAmp and Foobar2000 but that it could not be used with Windows Media Player. I then found a developer library called BASS which is available for Windows and Mac. On their website I also found an addon which supports Apple Lossless in Windows. These by themselves were not a solution, but at the same time I found a developer called Milenko Mitrovic had used the BASS library and a BASS MP3 addon and made a directshow filter called DC-BassSource out of them. This showed that it would be possible in theory to do the same with the Apple Lossless addon. I managed to get in touch with the developer of DC-BassSource and persuaded him to make a new version which added support for Apple Lossless (and AAC). When he sent it to me for testing, I then became the first person in the world to successfully play an Apple Lossless music track in Windows Media Player. As Microsoft Media Center uses WMP to manage and play the music it then meant I could also successfully play the music in Media Center as well.

There was only one final step that needed addressing which was allowing WMP to read the tags in AAC and Apple Lossless files. (Both use the same file MPEG4 file format, file extension and tag format.) As there was already several codecs available for WMP to let it play AAC files there was also already two different plugins available for WMP to let it read tags from AAC files and since Apple Lossless uses the same file extension, file format, and tag format these worked equally well for Apple Lossless files. These two plugins are WMP Tag Support Extender and WMP Tag Plus. The combination of the modified DC-BassSource codec and one of these two WMP plugins meant that you could easily add Apple Lossless tracks to WMP and it would read the embedded tags to show the track name, album name, artist etc. You could even set WMP to monitor your iTunes library folder and when you added a new track/album to iTunes WMP would automatically spot this and add them to its own library using the same single copies of each track.

This was an excellent result and worked fine from Windows XP through Windows Vista (not that I used Vista myself). However Microsoft did initially throw a spanner in the works when they released Windows 7. As you may remember above I mentioned that Microsoft added built-in support for AAC with Windows 7, they even added built-in support for reading MPEG4 tags as used in AAC files and even supported reading the embedded album artwork from AAC files. In theory this should not have been a problem, it was still possible to add an additional codec to allow playing Apple Lossless in WMP with Windows 7 and in theory as Apple Lossless files use the same file extension, file format, and tag format it should have happily read tags from Apple Lossless files as well. Unfortunately Microsoft went out of their way to specifically detect these files were not AAC files and even though (with the additional codec) it could play them Microsoft chose deliberately to move them to the ‘other’ section and not treat them as music files. This was incredibly frustrating as the pre-release version of Windows 7 had not done this. Fortunately Tim De Baets the developer of WMP Tag Plus was eventually able to come up with a way of tricking WMP in to thinking Apple Lossless files were AAC files. We could now once again have them play in WMP and have WMP accept them as music files and read the tags and artwork from them.

There was one more added complexity with Windows 7 which had already been solved. The preferred type of codec in Windows 7 was no longer directshow filters but a new type called Media Foundation. If a suitable Media foundation codec was present it took precedence over a directshow filter one. Therefore the built-in Media Foundation codec for AAC took precedence over the directshow AAC/Apple Lossless codec meaning that initially it would not play Apple Lossless even if you installed the appropriate directshow filter. Fortunately a new multi-codec pack was released for Windows7 which was known as Win7Codecs from ‘Shark007’. This included the same modified DC-BassSource directshow filter but had a button specifically for disabling the built-in Media Foundation AAC codec thereby allowing the DC-BassSource codec to take over.

If you would also like to use Apple Lossless with windows then download the appropriate choice from the list below.

Windows XP or Windows Vista
DC-BassSource - http://www.dsp-worx.de/?n=15
Wmp Tag Plus - http://bmproductions.fixnum.org/wmptagplus/index.htm

Windows 7 or Windows 8
Win7Codecs - http://shark007.net/win7codecs.html
WMP Tag Plus - http://bmproductions.fixnum.org/wmptagplus/index.htm
As a bonus iTunes itself can now automatically convert from Apple Lossless to AAC when syncing to an iPod or iOS device. This allows you to keep your music on your computer in its full lossless original quality, and to copy a slightly lower quality version to your music device that takes up less space – which as iPods or iOS devices have far less storage space is an important consideration. With this automatic conversion you do not have keep two copies of each track.

Apple Lossless is now an open-source standard with free source-code available here http://alac.macosforge.org/

Saturday 22 June 2013

Running Django webapps with OS X Server.app

Django is a framework for writing python webapps. Typical instructions for installing and running Django webapps are targeted at Linux environments but as OS X is a full Unix operating system and supports the same open-source software as Linux including Python, Apache and Django it is possible to use (almost) the same Linux aimed instructions to install and run a Django webapp.

However if you want to run such a Django webapp via Apple’s Server.app software then you need to undertake some extra steps. One step that you will not need to do if you have Apple’s Server.app installed is to install mod_wsgi to allow the Apache webserver to run Django i.e. Python webapps. While the standard OS X does not include this module Server.app does.

This article will give an overview for installing a Django webapp for use with OS X Server.app but another later article will specifically show how to install the Django webapp ‘Crypt Server’. First we will look at how to install Django itself, the typical instruction for installing Django is -

sudo pip install django

However OS X as standard does not have the pip tool installed. OS X does have a similar tool called easy_install which could be used to install django but fortunately you can also use easy_install to install pip itself as follows

sudo easy_install pip

You can then use the command

sudo pip install django

You can then test it has been successfully installed and confirm what version it is using the following commands

sh-3.2# python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> print django.get_version()
1.5.1
>>> quit()
sh-3.2#

After installing Django you would then download and install your webapp. We then need to setup various files so the webapp can be managed via Server.app. Apple don’t really provide any documentation on how to do this (hence this article) but fortunately they do provide an example which is located at

/Library/Server/Web/Config/apache2/webapps/com.apple.webapp.wsgi.plist

So the first step would be to make a copy of that with a new name. The following is what that files contains.

<?xml version="1.0" encoding="UTF-7"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>name</key>
 <string>com.apple.webapp.wsgi</string>
 <key>displayName</key>
 <string>Python "Hello World" app at /wsgi</string>
 <key>launchKeys</key>
 <array/>
 <key>proxies</key>
 <dict/>
 <key>installationIndicatorFilePath</key>
 <string>/Library/Server/Web/Data/WebApps/hello.wsgi</string>
 <key>includeFiles</key>
 <array>
  <string>/Library/Server/Web/Config/apache2/httpd_wsgi.conf</string>
 </array>
 <key>requiredModuleNames</key>
 <array>
  <string>wsgi_module</string>
 </array>
</dict>
</plist>

You then need to make the following changes, the name key needs to match the name of the copy of the above file you made, the displayName can be anything you want and is the description that will show up in Server.app, the installationIndicatorFilePath is a file it will look for to confirm your webapp is actually installed and therefore allow running it, and includeFiles is another configuration file we will look at next. You do not have to alter it but the requiredModuleNames ensures the mod_wsgi module is loaded so it can run the wsgi script i.e. the python code that makes up your webapp.

Now looking at the above mentioned includeFiles value. If you look at the file the example points to which is /Library/Server/Web/Config/apache2/httpd_wsgi.conf you will need to make your own copy of this (and updated includeFiles to match) and this is what that example contains

WSGIScriptAlias /wsgi /Library/Server/Web/Data/WebApps/hello.wsgi

This works just like a standard apache alias command and lets you point Apache to where your wsgi script is located which can be anywhere on computer and does not have to be in the standard websites folder. So you would put the full file path to your wsgi script here. This is typically somewhere in the set of folders making up your webapp. The wsgi file would be provided as part of your webapp.

Tip: If your 'wsgi' file actually came as wsgi.py then you need to rename it to have a wsgi file extension e.g. something.wsgi this is because Apache at least in Apple's configuration only accepts that file extension.

Presuming you have now installed Django, your webapp, and setup the above files for Server.app the final step is to create a website in Server.app which will host your webapp. This is pretty much the same process as creating a normal website but with the additional step that you click on the ‘Edit Advanced Settings…’ button in the new website and enable the entry for your webapp that should hopefully now be listed.

Note: You can also test your webapp without Apache using the command

python manage.py runserver 80

(You have to do this from the webapp directory.) This uses the lightweight webserver included with Django instead of Apache and means it also is not managed via Server.app

Saturday 15 June 2013

DHCP Server on OS X Server

No Apple code was harmed during the production of this utility.
Ever since Mac OS X Server was introduced, Apple have included a DHCP Server. This was and still is based on the open-source bootpd server.
This article is not intended to discuss the purpose of a DHCP server, for that you might want to read http://en.wikipedia.org/wiki/Dhcp.
While the original open-source bootpd server has the capability to define additional fields of information (via DHCP option fields) that can be provided via DHCP to clients, the version included with Mac OS X 10.4.11 Server and earlier did not support this. After several years of my and presumably others submitting requests for this feature, this was finally added to Mac OS X 10.5 Server. The overwhelming majority of router based DHCP servers do not have the ability to define DHCP option fields.
Note: This is a facility Microsoft have provided since at least Windows 2000 Server.
Probably the most common scenario that you would come across that needs DHCP option fields, is the use of a VoIP (Voice over IP) phone system. It was my experience that small to medium enterprises were earlier to adopt this type of phone system than big enterprises, and these smaller organisations are also more likely to use Macs and Mac servers - just like the company I run the IT for in fact.
So, Apple finally introducing the ability to define DHCP option fields in Mac OS X 10.5 Server was good news. Unfortunately, Apple did not and still does not even in OS X 10.8 Mountain Lion Server, provide a user friendly method for defining these values which are stored as Base64 encoded data values in the /etc/bootpd.plist configuration file. Thanks to the efforts of myself and others, the method of generating and encoding values has now been sufficiently deciphered that I am now able to create and provide a simple utility with a graphical user interface that mere mortals can use to generate any required DHCP option field. Here is what it looks like…
Screencapture
It can be downloaded free of charge from -
DHCP Option Code Utility
If you don’t feel comfortable editing Unix configuration files like /etc/bootpd.plist then I advise you don’t try doing this. If you have not already, I advise reading the Unix man page for “bootpd” on your Mac OS X 10.5, 10.6, 10.7, or 10.8 server. This is done in the Terminal.app by typing “man bootpd”.
Update: DHCP Option Code Utility 1.1 has the following improvements.
  • It now works under Mountain Lion properly (1.0 worked from Tiger to Lion), this was due to a change Apple made
  • It now can generate null-terminated strings as well as normal strings, null-terminated strings are used for example to define PXE boot servers in DHCP Option Code 67
  • It is now more forgiving on the format of text entered for hexadecimal values and will happily ignore spaces, colons, and dashes making it simpler to just paste a value in