Wednesday, 6 June 2018

Me and Apple's DHCP Server - a long history…

I have had a long and involved relationship with Apple with regards to their DHCP support.

I have in the past successfully persuaded Apple to :-


  • Add support to macOS X aka OS X aka macOS to be able to use WPAD i.e. Web Proxy Auto Discovery which Apple calls 'Auto Proxy Discovery'
  • and add support to their DHCP server for DHCP option codes as used by VoIP handsets and other network equipment


I also wrote myself a GUI tool to make it far easier for people to generate the encoded values Apple required for DHCP option codes in their bootpd.plist config file. See http://jelockwood.blogspot.com/2013/06/dhcp-server-on-os-x-server.html

This tool still works by the way.

However the one thing I did not succeed in persuading Apple to do was adding an IPv6 capable DHCP server. Apple's DHCP server is based on an extremely modified bootpd package which never was able to support IPv6.

With the recent(ish) announcement from Apple of yet more services going to be removed from their Server.app including the DHCP server it is clear that IPv6 is never going to be added. It also means my tool will no longer have a purpose.

:'(

Using Cat << EOF in a shell script to restore binary files

Unix/Linux/Mac shell scripts support what is commonly called 'Cat << EOF' whereby an entire file can be included in a shell script and 'restored' to a separate file stored on the drive.

As should be obvious this derives from the fact that at the command line you can do a command such as -

cat /etc/hosts > newfile.txt

In this scenario we want to be able to use a shell script to create the desired file. You might think that this could be simply achieved as follows -

#!/bin/sh
echo "This is the content of a file" > newfile.txt
exit

and yes this to some extent is possible, however if your file is going to contain multiple lines - some of which might be blank lines and some might contain commands or special characters this soon becomes effectively impossible with a simple echo command. Therefore shell scripts can use the Cat << EOF feature instead. Here is a simple example -

#!/bin/sh
cat <<- 'EOF' > newfile.txt
This is the content of a file
This is more content

Yet more content still
EOF
exit

You should now be able to see where the reference to Cat << EOF comes from, however the official term is a 'heredoc', see https://en.wikipedia.org/wiki/Here_document

This approach is commonly used to include and generate a single text file as shown above. However what if you want to do something more complicated? What if you want to either do an entire directory, or nested set of directories/files or binary files? Is this even possible?

The answer fortunately is yes. To do this the easiest way I have found is as follows -

tar -cv nameofdirectory | openssl base64 -e

This uses the standard tar command to convert the specified directory (or binary files) and then pipes the result to openssl, openssl is then told to encode the input in to base64 format, base64 is an ASCII i.e. text encoded version of the binary data. In this case the result is then displayed to standard output in your terminal and can be copy/pasted in to your shell script in the Cat << EOF section.

Here therefore is an example shell script which would 'restore' a file

#!/bin/sh
cd /where/you/want/to/restore
/usr/bin/openssl base64 -d << EOF | tar xf -

dGVzdC50eHQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAADAwMDY0NCAAMDAwNzY3IAAwMDAwMjQgADAwMDAwMDAwMDA1IDEzMzA1NzY0
MzUzIDAxNDUyNQAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAB1c3RhcgAwMGpvaG4ubG9ja3dvb2QAAAAAAAAAAAAA
AAAAAAAAAAAAc3RhZmYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwMDAwMDAg
ADAwMDAwMCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB0ZXN0CgAAAAAAAAAAAAAA
many lines deleted for the sake of readability
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAA==
EOF

exit

Whilst the command format is quite different it should be clear it is using the same openssl command to decode the base64 text and then sending it to the tar command to be 'restored'

So this approach allows storing and restoring either a single or multiple binaries files or an entire hierarchy of directories and text or binary files. This approach can even cope with restoring an Apple disk image or an ISO image and after you restore it could then use another command to mount the image and copy or run something from that image.

Thursday, 30 November 2017

Creating Certificates - Google Chrome issue

Here's some tips for you if you are creating your own server certificates.

Google Chrome requires your certificate have a SAN - Subject Alternative Name entry even if the certificate is only going to have a single name. You therefore need to add the main i.e. sole name as a SAN entry as well.

Note: If your certificate genuinely needs two or more names i.e. the main plus additional ones you should always add the main as a SAN entry along with the additional names.

For example lets say your certificate is for myserver.example.com your certificate would have a main name of myserver.example.com and a DNS type SAN entry also of myserver.example.com

Safari does not complain if you don't have a SAN entry but Google Chrome does.

I personally find using the free XCA tool far easier to work with especially with regards to adding SAN entries than trying to do this via the command line with openssl. XCA is a Java app that acts as a front-end to openssl. See http://xca.sourceforge.net/


You can either use XCA to just create the CSR - Certificate Signing Request and include the SAN entries in the CSR or you can use it to create self-signed certificates completely also with SAN entries.

Note: Apple's Keychain Access tool does not let you define SAN entries.

Friday, 4 March 2016

IKEv2 with iOS - issues and workarounds


Apple added support for IKEv2 VPN connections in iOS8 but only via mobileconfig profiles and added further support in iOS9 so you could define an IKEv2 profile in the GUI on the iOS device itself. (Apple also added IKEv2 support to OS X in El Capitan.)

Note: IKEv2 is considered much more modern and secure than previous older VPN standards such as IPSec, L2TP, and PPTP. Hence the fact Apple added support for IKEv2 and my using it.

While I have now successfully got an iPhone running iOS 9.2.1 to connect via IKEv2 to a matching IKEv2 VPN server I did come across a bug along the way which I have now reported to Apple. Obviously in getting it working I managed to get round these problems.

A common method for generating mobileconfig profiles for use with iOS devices is Apple Configurator. Apple Configurator 1.7.2 for Yosemite supports defining an IKEv2 profile but only for iOS clients, Apple Configurator 2.1 for El Capitan supports creating an IKEv2 profile for both iOS and Macs.

The issue I hit with Apple Configurator is that both the Yosemite version and the El Capitan version add an entry in the mobileconfig as standard which caused a conflict with my IKEv2 VPN server and prevented the iOS device from successfully connecting. The entry is in the IPv4 section and is a flag called OverridePrimary and AppleConfigurator sets this to be 'true' i.e. 1. This flag apparently tells the VPN client it must send all network traffic via the VPN connection including 'normal' traffic that needs to go to Internet connected sites, e.g. web browsing traffic. There is nothing wrong with wanting this to happen and in fact most corporates using IKEv2 would want that, however at least in my case this setting conflicts with settings in my IKEv2 VPN server which itself is already set to force all VPN clients to send all traffic via the VPN, this conflict causes the connection attempt to fail.

Note: I am using StrongSwan 5.1.2 on a Linux server as the VPN server.

To workaround this problem after identifying it I had to manually edit the mobileconfig file produced by Apple Configurator and delete the following section.

  1.      <key>IPv4</key>  
  2.      <dict>  
  3.           <key>OverridePrimary</key>  
  4.           <integer>1</integer>  
  5.      </dict>  

As my IKEv2 server is set to force all traffic via the VPN connection that still happens but this time with the above deleted from the mobileconfig the connection succeeds.

Unfortunately the Apple Configurator user interface does not list this option and hence does not itself allow disabling it if as in my case this turns out to be needed. Hence the need to manually edit the mobileconfig file. While investigating this issue I discovered that the old iPhone Configuration Utility which I had previously been using for Cisco IPSec configurations (being old it does not support IKEv2 configurations) does add the same OverridePrimary setting but sets it to 0 i.e. off and hence this is why I have not seen this problem before.

Note: Even though iPhone Configuration Utility set the setting to be 0 i.e. off because my StrongSwan5 setup has its own rule to force all traffic to go via the VPN connection this did indeed still as desired successfully force all traffic to go via the VPN connection. It should also be noted that the majority of example StrongSwan5 configurations include the same rule to force all traffic to go via the VPN connection which is to include the following in /etc/ipsec.conf

leftsubnet=0.0.0.0/0

This as should be obvious to anyone who has had to deal with configuring routers for a while means include every single IP address as the destination.


Now that I have got IKEv2 'working' on iOS I will move on to trying this in El Capitan and see how many bugs Apple have managed to include there. 

Wednesday, 17 February 2016

Automating the distribution of Apple Mail Stationery in a corporate environment

I recently had to find a solution to distribute Apple Mail Stationery files to all the Macs in a company. Stationery for use with Apple Mail is normally distributed either as an attachment to an email or as a file on a disk and either way would normally be installed in each users individual home directory. Not only would this result in potentially multiple copies of the same stationery file on a single Mac but the location in a users home directory is a rather complex one looking for example like this -

~/Library/Containers/com.apple.mail/Data/Library/Application Support/Mail/Stationery/Apple/Contents/Resources/Custom/

While hypothetically it might be possible to come up with a solution to both automate initial distribution of these files to each user and to distribute updates to these files to those users it seemed to me that the easiest solution was going to be to automate distributing them to each computer rather than each user especially as I was wanting to do this via Munki.

Note: This approach should equally work using similar tools like ARD, CasperSuite, etc.

The first step was therefore to locate where Apple's own included example mail stationery files were located which is…

/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/

…with multiple sub-folders to organise the different categories of stationery Apple provide. An added complexity is that Apple also use the following file as an index defining the list of categories/sub-folders

/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/TableOfContents.plist

I could have simply placed my files in one of the existing Apple folders but apart from this likely being confusing to users it may also be at risk of being wiped by Apple updates. I therefore wanted to create a sub-folder for my own files. I therefore needed to find a way to -

  1. Create the sub-folder
  2. Update Apple's TableOfContents if needed to include my sub-folder
  3. Copy my files in to it

I accomplished this by creating a standard Apple Installer package file with a payload of my mail stationery files, and a pre-install script in the Installer package to create the sub-folder and to update the TableOfContents index. While I have chosen to use a single Installer package to install multiple mail stationery files to the same folder this approach could easily be adapted to have individual installer package files one for each stationery file.

While the TableOfContents file is a plist file it is not of a format that makes it possible to use the standard 'defaults' command to modify it - defaults is particularly weak at handling arrays, it might have been possible to use PlistBuddy but I settled on using a perl script as part of a shell script to do this.

#!/bin/sh
/bin/mkdir -p "/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/MyCompanyName/Contents/Resources/" && /bin/cp -R "/private/tmp/Mail Stationery/" "$_";
searchtableofcontents=`grep '<string>MyCompanyName</string>' "/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/TableOfContents.plist"`
if [ "$searchtableofcontents" != " <string>MyCompanyName</string>" ]; then
perl -i -pe 'BEGIN{undef $/;} s/<\/array>.*<\/plist>/\t<dict>\n\t\t<key>Folder Name<\/key>\n\t\t<string>MyCompanyName<\/string>\n\t<\/dict>\n<\/array>\n<\/plist>/smg' "/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/TableOfContents.plist";
fi
exit 0

I then added this installer package to my Munki repo to be automatically distributed to all the Macs. I can and have issued updated versions simply by creating a new updated installer package with a new version number and Munki picks this up and distributes it as an update.

Users can then see the mail stationery in Apple Mail as a separate folder in the standard list of stationery. Users do not have to be bothered by or confused by getting an email attachment and working out how to install it themselves. Furthermore this works even if a user uses more than one different Mac and even if the user leaves and someone else gets their Mac.

I can also automate uninstalling this and tidying things up by using the following matching Uninstall script in Munki.

#!/bin/sh
searchtableofcontents=`grep '<string>MyCompanyName</string>' "/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/TableOfContents.plist"`
if [ "$searchtableofcontents" == " <string>MyCompanyName</string>" ]; then
perl -i -pe 'BEGIN{undef $/;} s/\n\t<dict>\n\t\t<key>Folder Name<\/key>\n\t\t<string>MyCompanyName<\/string>\n\t<\/dict>//smg' "/Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/TableOfContents.plist";
fi
exit 0

Note: The above script's If statement has exactly the number of spaces needed in it so be careful when copying it.

Note: Munki uses the package receipts list to delete i.e. uninstall the actual mail stationery files, the above script simply removes the sub-folder entry from the Apple TableOfContents file.

Saturday, 13 February 2016

Munki and Microsoft Office licenses

Many Mac using organisations will use a free open-source tool called Munki to deploy applications and updates for those applications including of course Microsoft Office for Mac. If you have a Microsoft volume license this is extremely easy as all you have to do is download the ISO and add the contents to your Munki repo. If you however still have individual aka. ‘boxed’ license numbers for Office then matters are a bit more complicated.

Firstly a bit of a rant about Microsoft and their ‘Volume’ licenses. To most people volume implies a volume discount as anyone who has to do the weekly supermarket shopping run will be familiar with. This also applies to most software companies but not Microsoft. Instead volume when associated with Microsoft probably refers to how loud their customers will scream when they see how much they are being ripped off by Microsoft. You will be staggered to learn that a Microsoft volume license for Office 2016 costs almost exactly double the price of the same number of ‘boxed’ Office 2016 copies. Yes you read that correctly double!

Now this is on top of the long standing practice by not only Microsoft but most software companies of not allowing you to convert individual boxed licenses to be part of a volume license and therefore requiring you to pay full price for a new volume license. This is as I said a longstanding practice and while arguably unfair and arguably counter-productive in that it can discourage customers moving to a volume license which even ignoring costs would be in the long-term interest of Microsoft as well.

Historically small organisations would start off by buying boxed licenses and then grow to a size that the simpler administration provided by a volume license becomes important, most companies at some point would hit this problem. (These days companies might start off with Office 365 and then move to a volume license.)

Note: The fact that the volume license for Office costs double the price of the same number of boxed copies applies to both Mac and Windows versions. At least one can say unusually that Microsoft are treating both Mac and Windows customers equally bad.

The main purpose of this post is therefore to describe how to continue to use boxed Microsoft licenses on Macs via Munki and to still keep much of the benefit of simplicity provided otherwise by a volume license.

The steps to do this are simple to do and only require a modest amount of initial work.
  1. Add as normal the (boxed) Office installer to your Munki repo
  2. Use Munki to deploy Office to a new Mac
  3. On the new Mac launch Office and manually activate its license as normal
  4. This will create the license file on that Mac, this now needs to be copied to your Munki server, I create an additional sub-folder in the Munki repo called licenses for this purpose, and then in the licenses folder create another folder called office2008 or office2011 or office2016 as appropriate, then create in that a folder named with the serial number of the Mac e.g. CK123456789Z and finally put the office license file in that
For Office2008 the license file is located at
/Applications/Microsoft Office 2008/Office/OfficePID.plist
For Office2011 the license file is located at
/Library/Preferences/com.microsoft.office.licensing.plist
For Office2016 the license file is located at
/Library/Preferences/com.microsoft.office.licensingv2.plist

So on your Munki repo you will have something like this
./licenses/office2008/CK123456789Z/OfficePID.plist or
./licenses/office2011/CK123456789Z/com.microsoft.office.licensing.plist or
./licenses/office2016/CK123456789Z/com.microsoft.office.licensingv2.plist

This now makes it possible to have Munki automatically install the license file the next time the same Mac needs Office reinstalling, perhaps after replacing a fault hard disk or fitting a bigger hard disk or wiping and reinstalling the entire Mac. As the license file is linked to the same Mac all that is required is to copy the license file back to the same location. To do this I use the following post-install script in Munki for the Office install.

Note: This example script is for Office 2011 it should be very obvious what changes are needed for other versions of Office based on the above information.

#!/bin/sh
officeversion=”office2011”
location=”/Library/Preferences”
filename=”com.microsoft.office.licensing.plist”
serialnumber=$(/usr/sbin/system_profiler SPHardwareDataType | /usr/bin/grep 'Serial Number (system)' | /usr/bin/awk '{print $NF}')
/usr/bin/curl https://munki.domain.com/repo/licenses/$officeversion/$serialnumber/$filename –o $location/$filename
exit

Note: If you run the Munki install of Office before a license file has been 
created and added to the relevant sub-folder of the licenses folder this is not 
a problem as Office will still be installed successfully and you simply have to 
license it manually, thereafter assuming you remember to copy the license file 
to the relevant folder in your Munki repo this will be copied automatically 
after the install of Office.

Friday, 20 November 2015

Apple HomeKit, the 'Internet of Things' and security

Apparently some manufacturers are criticising Apple for the 'excessive' security required in order to gain HomeKit certification. Some of them and some pundits may be thinking this is another fiendish scheme by Apple to achieve lock-in to yet another Apple only eco-system.

They are wrong.

Apple have shown themselves to be very much concerned about security and gone to great lengths for example to provide excellent security for iOS devices including end-to-end encryption in iMessage - so much so that the FBI is pretty unhappy.

Funnily enough on the topic of the FBI, some of you may have seen a new TV series called CSI: Cyber which about a cybercrime fighting division in the FBI, the first episode in the series was about a baby monitor being hacked over the Internet. This is actually based on true events, baby monitors have and are being hacked over the Internet. See http://www.bbc.co.uk/news/technology-34138480

So Apple is being well ahead of the field in building in extremely robust security in to HomeKit ready for when the 'Internet of Things' really takes off, it is after all far easier to do this at the beginning rather than trying to figure out how to fix things afterwards.