When It Comes To Facebook Scale, You Can Throw Out The Rulebook | TechCrunch

I read this techcrunch article this week about hardware engineering at Facebook. The nitty gritty details are only mildly interesting to me, but listen to what Facebook are saying about the culture they’ve created.

“We do it this way because this is the way we’ve always done it?”

Not at Facebook you don’t.

What’s the BPR equivalence? Can I achieve similar results through being a continual advocate for Kaizen as a process improvement methodology, or is this a cultural thing at FB that I can’t replicate on my own?

Get in touch or leave a comment to let me know!

When It Comes To Facebook Scale, You Can Throw Out The Rulebook | TechCrunch


Late Night Links – Sunday September 28th, 2014

It’s that time of the week again, and you should make the most of it too because in all likelihood late night links will be taking a few weeks off after today, seeing as I too am taking a few weeks off.

And that’s it! I’ll be back with more in a few weeks.


Single Sign-On (SSO) in PHP

There’s a project underway in work called single sign-on and identity and access management. I’m not involved in it directly, although by its nature it touches on several things that I am working on at the moment. The goal, as the name implies, is to rid ourselves entirely of multiple sets of credentials: anything we use should have the same login ID and password, whether it’s one of our hosted systems (which, to be fair, already behave this way for the most part) or a third-party system like the webapp that we use to deliver training.

Since I’m not directly working on it this project is not really anything more than a blip on my radar, but it’s interesting to me because I’m attempting to do a similar thing at home, albeit on an entirely different scale to the large enterprise-wide project that’s I hear about in my professional life.

After the recent upgrade to my home server that I’ve blogged about before I now have several virtual servers included in our home network setup. One of these runs Windows Server 2008 R2, and I’ve made that one a domain controller that all the other computers (and servers) connect to. There are several benefits to this approach, but chief amongst them is a single set of credentials – I use the same username and password regardless of which of our home computers I’m logging on to, and when I change my password I change it once for it to be effective everywhere.

There are few web services running on our home network which require signing into, such as a web interface for centralized torrent downloads, a file browser, and a simple content management system that pulls everything together into an intranet of sorts. Most of these are PHP-based, and I’m on a mission to add SSO capability to these too.

I’ve discovered two main methods of enabling SSO in PHP that I’ll write about after the break, and my eventual plan is to tie the two methods together into a single cohesive sign-on module that I can reuse. Read on to find out what I’m up to!

LDAP (Lightweight Directory Access Protocol) Authentication

Wikipedia defines LDAP as an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network.

That’s a lot of fancy words for saying that LDAP provides an address book (think of the global address listing you see in Outlook, and you’re thinking of an LDAP database). PHP has a set of LDAP extensions that can be used to query the address book and retrieve user information, but in the context of authentication, we don’t even need to worry about any of that. An LDAP server can (depending on the implementation) be queried anonymously, or we can pass in some credentials with the query to get more detailed information back (again, depending on the implementation).

It’s this last part that’s important. Active Directory on a Windows domain controller is an LDAP server. In PHP, all we have to do is attempt to log on to the LDAP server. If we’re successful, it’s because the username and password that we input is valid on the domain. Even better, “valid on the domain” in this case means it’s an active account, the password is not locked, and all other account-level restrictions (such as a restricted set of logon hours) are considered.

All of this makes using LDAP to test the authenticity of a set of supplied credentials pretty trivial:

   $username = "testuser"
   $password = "pa55w0rd";

   $domain = "testdomain.local";
   $domaincontroller = "dc1.testdomain.local";

   $ldap = ldap_connect($domaincontroller);
   if ($bind = ldap_bind($ldap, $username."@".$domain, $password)) {
      // user login successful
   } else {
      // user login failed

That’s all there is to it!

Depending on what you had in mind when you read “SSO” in the title of this post though, we may not have met your requirements here. If we meant that the user has a single set of credentials then, fantastic – they do! But if our intention was to only require that a user enters their single set of credentials once (when they log on to Windows) then we’ve fallen short here. The code above requires the username and plaintext password, so we’d have to present some kind of web-based login form to the user to request that information and get all this to work.

Enter NT LAN Manager (NTLM) Authentication

If a website (or intranet site) is part of the intranet or trusted zones (found in the Internet Settings control panel applet) then that site is allowed to pass a header requesting NTLM authentication. When it does, windows passes a header back containing some information about the currently logged-in user without the user being prompted for their credentials in any way.

I obtained some simple example code from someone called loune at and modified so that it doesn’t require apache as the webserver. Here’s the PHP:

   if (!isset($_SERVER['HTTP_AUTHORIZATION'])){
      header('HTTP/1.1 401 Unauthorized');
      header('WWW-Authenticate: NTLM');


   if (substr($auth,0,5) == 'NTLM ') {
      $msg = base64_decode(substr($auth, 5));
      if (substr($msg, 0, 8) != "NTLMSSPx00")
         die('error header not recognised');

      if ($msg[8] == "x01") {
         $msg2 = "NTLMSSPx00x02x00x00x00".
            "x00x00x00x00". // target name len/alloc
            "x00x00x00x00". // target name offset
            "x01x02x81x00". // flags
            "x00x00x00x00x00x00x00x00". // challenge
            "x00x00x00x00x00x00x00x00". // context
            "x00x00x00x00x00x00x00x00"; // target info len/alloc/offset
         header('HTTP/1.1 401 Unauthorized')
         header('WWW-Authenticate: NTLM '.trim(base64_encode($msg2)));

      else if ($msg[8] == "x03") {

         function get_msg_str($msg, $start, $unicode = true) {
            $len = (ord($msg[$start+1]) * 256) + ord($msg[$start]);
            $off = (ord($msg[$start+5]) * 256) + ord($msg[$start+4]);
            if ($unicode
               return str_replace("", '', substr($msg, $off, $len));
               return substr($msg, $off, $len);

         $ntlm_user = get_msg_str($msg, 36);
         $ntlm_domain = get_msg_str($msg, 28);
         $ntlm_workstation = get_msg_str($msg, 44);

   echo "You are $ntlm_user from $ntlm_domain/$ntlm_workstation";

There’s a big problem with this code, and the problem is that it’s just decoding the user information from the HTTP header, and assuming that all is good – there’s no work done to confirm that the header is genuine, and there is a possibility that it could have been faked. We could do some tricks like confirming that the page request is coming from within our local network, but that doesn’t really solve the problem – HTTP headers can be manually defined by an attacker that knows what they’re doing, and what we’re doing here is a bit like asking for a username and then just trusting that the user is who they say they are without doing any further authentication.

Combining the Two Approaches

Included in the NTLM authorization header that gets sent to the webserver during the passwordless authentication interaction described above is an MD4 hash of the user’s password. A newer version of loune’s code retrieves this and confirms its validity using samba. Unfortunately that setup won’t work for me – my intranet webserver is running a customized version of samba that comes with the software I use to manage the linux computers that are attached to my domain, and this trick just flat-out fails.

However, if I have a plaintext version of the user’s password then I can use PHP to generate an MD4 hash of it for the purposes of comparison. So here’s my plan:

Scenario A: The first time a user comes to my webapp we’ll get their credentials using NTLM, including the MD4 hash of their password. Since we won’t know if this hash is valid, we’ll present the user with a screen asking them to confirm their password (but not their username). When they input it, we’ll confirm that their username and password combo is good using LDAP, and also generate an MD4 hash of the plaintext password that they entered to compare with what NTLM gave us. If nothing weird is going on everything should match. At this point we’ll store the MD4 password hash for future.

Scenario B: When a user returns to our webapp we’ll get their credentials using NTLM as before, and compare the hash NTLM gave us to our stored hash from their previous visit. If they match, we’re good, and there’s no need to ask the user to enter their password.

Scenario C: If the NTLM hash and the stored hash don’t match then the most likely scenario is that the user has changed their Windows password since their previous visit to our webapp. In that case we’ll throw out the stored hash and start again at Scenario A.

If anyone knows of a better approach (is there a centrify Kerberos tool that I could use to get an MD4 hash of the user’s password for the purposes of my comparison, for example?) then please let me know! I’d love to be able to achieve true passwordless SSO, but so far I can’t a method for doing so unless I switch my webserver from linux to Windows, and I don’t want to do that.



This is interesting. Do you guys think this is the Friedrich Nietzsche?

The fact that he’s spelled his own name wrong is making me suspicious.


Creating a Self-Signed SSL Certificate for Lighttpd

You’ve probably heard of SSL, or at least know it from either the https:// prefix that you see when browsing certain websites or the padlock icon in your browser’s address bar that goes along with it.


You probably also know that this icon’s presence is an absolute must when you’re doing sensitive things on the internet, like online banking. Really though you should consider it a must on any site where you’re entering information that you wouldn’t want falling into the wrong hands – including your username and password for the site itself and anything you do on the site once you have logged in.

SSL does two important things: It encrypts the connection between your browser and the site’s webserver, meaning that if somebody had the ability to listen in to your internet traffic (which is actually frighteningly easy, especially if you’re using a public WiFi hotspot) then they won’t actually see any of your important personal details. SSL also provides identity verification for websites in order to thwart a more complex attack where your web traffic is somehow redirected to a fake version of the site. Today we’re going to tackle only the first part – encrypting the connection between the browser and my webserver, which is running lighttpd.

Recently I’ve created a web interface that allows me access to my documents from anywhere on the web. To log in I have to enter my user ID and password for my home network, and once I’m logged in I may want to open a file that includes some sensitive information. This whole interaction is something that should be protected end to end by SSL, so that’s precisely what I’m going to do.

Creating an SSL Certificate

Of the two things SSL can do for us (securing a connection and confirming the identity of the webserver we’re connected to), the first part is actually much easier than you might think. The problem (as we’ll discover), is that doing only that first part has some problems that make it unsuitable for a typical public website. More on that later, but in my scenario where I have a website that’s intended only for my use this will be an acceptable solution, and that’s what we’re going to do.

On the webserver, navigate to a directory where you’re going to store the SSL certificate file. This directory should not be web-accessible. We’re going to use OpenSSL to create our SSL certificate. OpenSSL is unfortunately best known for introducing the heartbleed bug that caused a panic in the not too distant past, so before you proceed make sure the version you have is not affected. The step we’re about to complete actually won’t be impacted even if your server is vulnerable to heartbleed, but the day to day use of any certificate on a vulnerable server is not safe.

Ready? Good. Type the following command:

openssl req -new -x509 -keyout server.pem -out server.pem -days 365 –nodes

OpenSSL will ask a few questions, the answers of which will form a part of the certificate we’re generating (and be visible to site visitors, if they choose to go looking for it). Everything is fairly self-explanatory with the possible exception of the Common Name field. Since we’re going to be using this certificate for web-based SSL, the Common Name must be the hostname (the full domain name, including the www prefix if you use it) of your website.

Country Name (2 letter code) [AU]:CA
State or Province Name (full name) [Some-State]:Alberta
Locality Name (eg, city) []:Calgary
Organization Name (eg, company) [Internet Widgits Pty Ltd]
Organizational Unit Name (eg, section) []:Hosting and web services
Common Name (eg, YOUR name) []
Email Address []:[email protected]

You’ll find that you now have a file called server.pem in your working folder, and that’s it! This is your SSL certificate that will be used to secure the connection.

Enabling SSL in Lighttpd

Now we need to configure our webserver to use SSL with the certificate we’ve just generated. As I noted, my webserver is lighttpd. If you’re using Apache, IIS, Nginx or something else then the steps you need to follow will be different.

For lighttpd, open up your lighttpd.conf file (typically found in /etc/lighttpd) and adjust your configuration similar to the following:

$SERVER["socket"] == ":80" {
   url.redirect = ("^/(.*)" => "$1") 

$SERVER["socket"] == ":443" 
   ssl.engine = "enable"
   ssl.pemfile = "/path/to/server.pem"
   server.document-root = "/path/to/web-root"

The first section identifies any traffic that reaches port 80 on our webserver (http), and redirects the user to the https version of the site. The second section applies to traffic reaching port 443 (https), enables lighttpd’s SSL engine and provides the paths to the server.pem file that we generated, and the appropriate content.

Restart lighttpd for the changes to take effect:

sudo /etc/init.d/lighttpd restart

And that’s it! Traffic to our site is now encrypted using SSL.

Identity Verification

As I alluded to earlier in the article though, there’s a problem. When you navigate to the site in your browser you see (depending on your browser of choice) something similar to the following on your screen.


It’s not particularly specific, but clearly Chrome has an issue with our setup.

The problem here is the one I alluded to earlier, and if you click the padlock icon in the address bar then Chrome will give you some additional details that show you what I mean.


Our connection is indeed secured by SSL as we’d hoped, but Chrome has been unable to verify the identity of the website we’re connecting to. This is not a surprise – since we created the SSL certificate ourselves, our browser has no means of knowing if the certificate should be trusted or not. This is why self-signed certificates are not suitable for public, production websites.

Since this is site is going to me for my use only, I can live with it. The important thing is that my connection is encrypted, and if I hit the Advanced link then I have an option to ignore the warnings and proceed to the site. I don’t want to do that every time if I can avoid it though, and the solution is to add the site’s SSL certificate to your computer’s Trusted Root Certificate Authorities store. Chrome (on Windows) and Internet Explorer both use this same location when checking the validity of SSL certificates, so the easiest way to go about doing this is actually to open the site in Internet Explorer and then complete the following steps which I took from a helpful user on stackoverflow:

  1. Browse to the site whose certificate you want to trust.
  2. When told There is a problem with this website’s security certificate, choose Continue to this website (not recommended).
  3. Click on Certificate Error at the right of the address bar and select View certificates.
  4. Click on Install Certificate… then in the wizard, click Next.
  5. On the next page select Place all certificates in the following store.
  6. Click Browse, select Trusted Root Certification Authorities, and click OK.
  7. Back in the wizard, click Next, then Finish.
  8. If you get a security warning message box, click Yes.
  9. Dismiss the message box with OK.
  10. Restart your computer.
  11. When you return to the site in either internet explorer or Chrome, you should find that the certificate is now trusted.

All done!