Itās been a while since Iāve posted code of any description, but Iāve been working on a couple of things recently that Iām going to make publicly available on my GitLab page (and my mirror repository at code.jnf.me)
Backblaze B2 Version Cleaner
I wrote last week about transitioning my cloud backup to Backblazeās B2 service, and I also mentioned a feature of it thatās nice but also slightly problematic to me: it keeps an unlimited version history of all files.
Thatās good, because it gives me the ability to go back in time should I ever need to, but over time the size of this version history will add up – and Iām paying for that storage.
So, Iāve written a script that will remove old versions once a newer version of the same file has reached a certain (configurable)Ā āsafe age.ā
For my purposes I use 30 days, so a month after Iāve overwritten or deleted a file the old version is discarded. If I havenāt seen fit to roll back the clock before then my chance is gone.
This one I created for work. Getting data from a SharePoint list into Excel is easy, but I needed to write Excel data to a list. I assumed thereād be a VBA function that did this for me, but as it turns out I was mistaken – so I wrote one!
At the time of writing this is inĀ āproof of conceptā stage. It works, but itās too limited for primetime (it can only create new list items, not update existing ones, and each new item can only have a single field).
Out of necessity Iāll be developing this one pretty quickly though, so check back regularly! Once itās more complete Iāll be opening it up to community contributions.
I have no plans to add functions that read from SharePoint to this library, but once I have the basic framework down that wouldnāt be too hard to add if youāre so inclined. Just make sure you contribute back!
My home network is domain-based, and Iām running a Windows Server 2008 VM as the domain controller. Iāve written
in the past about how to use PHP to do authentication using domain
credentials, and that works great for some scenarios. As a case in point, I use Pydio to host a web-based file manager that
allows me access to my files when Iām out and about. Pydio runs on a linux
server VM on my home
server, and it actually includes a built-in mechanism to authenticate
against an LDAP server (the Windows domain controller) so I didnāt have to
modify it with my PHP code. The principle is the same, though.
This is all good stuff for anything hosted on my home
server, but what if that isnāt what I want? What if I want to host something on
my external, public webserver, and still use my active directory credentials to sign in to
it? And, while weāre at it, what if I want to be even more restrictive and
limit access to a particular organizational unit within active directory?
As luck would have it, these are all problems that I solved
this week. Read on!
Creating a Reverse SSH Tunnel
The first thing we need is to establish a secure connection
between the external webserver (the VPS) and the internal webserver (the local
linux VM). Weāre going to use a reverse SSH tunnel to do this,
and, specifically, weāre going to use a tool called autossh that will keep an eye on
the tunnel and restart it if something goes wrong.
Iāll skip most of the technical detail here, but essentially
a reverse tunnel is going to forward a particular port on the external server
to a particular port on the internal server. Itās called a reverse tunnel
because itās the internal server that triggers the connection. Thatās important:
the internal server can reach the external one just fine, but not the other way
around (thanks to things like me having a dynamic IP address for my home
internet connection, my home routerās firewall, DHCP, etc).
I installed autossh on my Ubuntu server VM:
sudo apt-get install autossh
And then wrote a one-line script that I placed in /etc/network/if-up.d:
Teasing this apart just a little, it uses the local account ālocaladminā
to run the command enclosed in the quotation marks. That command forwards port
8080 on āremote-server.comā to port 80 on the local machine.
For this to work itās essential that the user ālocaladminā
is able to log on to āremove-server.comā without
needing to enter a password.
The Local Server
The local webserver is where most of the heavy-lifting is
going to take place. The first thing I did was create a new virtual host in the
webserver configuration on that machine. Iām using lighttpd, so my configuration looks like
this:
Essentially it creates a new host with the hostname āauth.gatewayā
thatās only accessible to the local machine (127.0.0.1). Any request that comes
in regardless of the URI is rewritten to a file called auth.php in the document
root.
The hostname here isnāt real (i.e. thereās no public DNS
entry for it), and thatās probably a good thing for the sake of locking down
security as tightly as possible. Also on that line of thinking is the fact that
access is limited to the local machine. The webserver doesnāt know that requests
coming through our SSH tunnel are coming from another machine thanks to the
virtues of port forwarding ā it thinks theyāre coming from other processes
running locally (i.e. coming from 127.0.0.1).
Next is auth.php itself, the engine that runs all this
stuff:
As you can see, this is quite a bit more sophisticated than
the previous example in my SSO post, but letās tease this one apart at a high
level too.
When the script is loaded, it first checks to see if the thereās
a username, password and āAUTH_KEYā contained within the HTTP headers of the
request. If so, it verifies that the āAUTH_KEYā is what it was expecting, and it
tries to use the username and password to establish an LDAP connection to the
Windows server VM. If thatās successful, it retrieves some information about
the user and checks if theyāre a member of the required organizational unit.
If all that works it sends back an empty page, but,
crucially a HTTP 200 status (meaning everything is OK). If any of those checks fail
then it sends back a HTTP 401 status (unauthorized) header instead.
In addition to this, the script creates a PHP session with a
custom name. The name is custom to avoid collisions with any session that
the external server may be creating, but essentially the session lasts 10
minutes, and if subsequent requests come in for a user thatās already been
authorized then it skips all the checks and just sends back the HTTP 200
header. It does that because without it, every HTTP request made to the remote
server (not just every page served, but every image on every page, every CSS
file, JavaScript file, etc, etc) would involve a query to the domain controller
to validate the credentials, and thatās a potential bottleneck with performance
implications.
The Remote Server
My remote server is running nginx as its webserver (Iām new to it,
but I think I like it better than lighttpd. Thatās kind of beside the point
though). The configuration looks like this:
When we tease this one apart, there are two key details. One
is that āauth_requestā declaration in the fourth line. As per nginxās
documentation, auth request āimplements client authorization based on the
result of a subrequest.ā In other words, instead of nginx doing the
authentication itself it forwards the request on. The configuration above
defers the authentication of http://example.com to http://example.com/__auth.
The second crucial chunk of the configuration file is
everything within the ālocation = /__authā declaration. The first thing this does
is turn off the āauth_requestā functionality (otherwise, weāre stuck in an
endless loop), and it then creates a proxy to redirect any requests to http://example.com/__auth to http://localhost:8080. Port 8080, if you recall,
is in turn forwarded through our SSH tunnel to port 80 on the local server.
Additionally, it sets the hostname involved in the request
to āauth.gateway,ā forwards a couple of other pieces of information as headers,
and, for some added security, sends an āX-Auth-Keyā header that our PHP script
checks for.
Back outside of this block the āauth_request_setā
declaration takes the session cookie from our PHP script and saves it to a
variable, then the following line (āadd_headerā) sends that cookie back to the
clientās browser as part of the response they receive.
Done!
Real World Problems with All This
I mentioned earlier that we set a session cookie as part of
the whole interaction to avoid the need to query the LDAP server for every HTTP
request the remote server receives, and I said this was to avoid a performance
bottleneck. Thatās true, but we still have a performance bottleneck here: for
every HTTP request the remote server receives itās still querying the local
server through our SSH tunnel, even if all the local server is doing is
responding that the credentials are good based on the existence of the session
cookie.
This communication has to take place over my home internet
connection, which is absolutely not what my ISP intended it to be used for. I don’t think it’s against their terms and conditions or anything like that, itās just that it isnāt
really fast enough.
If the site deployed on the remote server was one of my own
making then Iād modify this approach to create an authentication API of sorts
on the local server, and Iād do the session setting and credential caching
entirely on the remote server, drastically reducing the number of queries to
the local server (all the way down to a single query, in fact, when the session
is first created and the user logs on).
The other problem is one of securing the whole interaction.
Weāre using ābasicā HTTP authentication methods here, which means that the
username and password are passed around in the clear (theyāre not encrypted or hashed as
part of the process). Thatās necessary: the auth.php script has to receive the
password in cleartext because it has to pass it to the Windows server to check
its validity. Itās also not an issue with the communication between the remote
and local webservers, because that happens through an SSH tunnel thatās using
public/private keypairs to provide encryption. It is a problem for the
communication between the user and the remote webserver though, and it leaves
our user vulnerable in several ways (especially if theyāre using a public WiFI
hotspot). Essentially what Iām getting it is that you must use SSL between the user and the remote server.
Conclusion
This is not the most optimal way of doing things,
particularly in regards to the bottleneck it creates by deferring authentication
of every HTTP request to my local server which is connected to the internet
using a typical consumer-grade ADSL connection, but as a quick and dirty way of
securing resources on my public webserver without needing to modify the
resource itself in any way, it works great!
I want my public web server, which runs nginx, to authenticate against my active directory server using LDAP. Iāve written in the past about how to use PHP to authenticate against active directory in this way, but there are a couple of problems: my active directory server isnāt accessible to the internet, and I want to use standard HTTP authentication instead of username and password boxes included on a webpage.
The answer, I think, is to put an authentication PHP script on my home server, make that available to the public web server through an SSH tunnel, and then use nginxās Auth Request module to authenticate against it using the public server as a proxy.
This is – I hope – less complicated than it sounds. Weāll see, and Iāll post more if and when Iām successful, but the problem Iāve initially run into is that nginx in Ubuntuās repositories doesnāt include the Auth Request module. I have remove nginx and re-install it from source, compiling it with the additional module included.
Itās a bit of a daunting process, but the page Iāve linked seems like it will take me through it step by step.
Hopefully you didnāt even notice I was gone, but two days ago Tumblr terminated my account, removing this blog and Shrapnel from the internet. I immediately contacted support as directed and heard back from them yesterday evening: my account had been closed for contravening Tumblrās community guidelines in relation to spamming and affiliate marketing.
I replied to make the point that at no point have I engaged in spamming or affiliate marketing, and apparently someone there agreed because I am now back online. The issue, as it turns out, was that my two Tumblr blogs were sending visitors back to jason.jnf.me (where I had a script that presented the blog content in a subfolder, integrating it into the site to a much greater degree than a separate domain would).
In the short-term Iāve removed the redirect by simply resetting my blogās theme to the default, and Iāll take some time on the weekend to restore the look and feel I had previously, and probably give each of them a custom subdomain.
In the longer term, I think itās time to start looking for an alternative blogging platform. When it seemed as though all the content I had on this blog had disappeared I was extremely disappointed. I run my own server, so I probably shouldnāt be relying on third-party services anyway.
The obvious suggestion would be to install WordPress, and while that would work great for my blog content I think Iād have a hard time implementing some of the other site pages on that platform. What I want is a CMS (to give me the ability to quickly and easily manage and edit content) that lets me build custom bits and pieces (like my feed page) on top of it. Iāve chosen PyroCMS. Itās built on the CodeIgniter framework that Iāve previously used which should make for relatively easy extensibility. Itās going to take me some time, but Iāve installed it on my development server to start getting my hands dirty. Iām just happy Iām back online and I donāt have to spend this weekend trying to rebuild.
Thereās a project underway in work called single sign-on and identity and access management. Iām not involved in it directly, although by its nature it touches on several things that I am working on at the moment. The goal, as the name implies, is to rid ourselves entirely of multiple sets of credentials: anything we use should have the same login ID and password, whether itās one of our hosted systems (which, to be fair, already behave this way for the most part) or a third-party system like the webapp that we use to deliver training.
Since Iām not directly working on it this project is not really anything more than a blip on my radar, but itās interesting to me because Iām attempting to do a similar thing at home, albeit on an entirely different scale to the large enterprise-wide project thatās I hear about in my professional life.
After the recent upgrade to my home server that Iāve blogged about before I now have several virtual servers included in our home network setup. One of these runs Windows Server 2008 R2, and Iāve made that one a domain controller that all the other computers (and servers) connect to. There are several benefits to this approach, but chief amongst them is a single set of credentials ā I use the same username and password regardless of which of our home computers Iām logging on to, and when I change my password I change it once for it to be effective everywhere.
Iāve discovered two main methods of enabling SSO in PHP that Iāll write about after the break, and my eventual plan is to tie the two methods together into a single cohesive sign-on module that I can reuse. Read on to find out what Iām up to!
Wikipedia defines LDAP as an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network.
Thatās a lot of fancy words for saying that LDAP provides an address book (think of the global address listing you see in Outlook, and youāre thinking of an LDAP database). PHP has a set of LDAP extensions that can be used to query the address book and retrieve user information, but in the context of authentication, we donāt even need to worry about any of that. An LDAP server can (depending on the implementation) be queried anonymously, or we can pass in some credentials with the query to get more detailed information back (again, depending on the implementation).
Itās this last part thatās important. Active Directory on a Windows domain controller is an LDAP server. In PHP, all we have to do is attempt to log on to the LDAP server. If weāre successful, itās because the username and password that we input is valid on the domain. Even better, āvalid on the domainā in this case means itās an active account, the password is not locked, and all other account-level restrictions (such as a restricted set of logon hours) are considered.
All of this makes using LDAP to test the authenticity of a set of supplied credentials pretty trivial:
Depending on what you had in mind when you read āSSOā in the title of this post though, we may not have met your requirements here. If we meant that the user has a single set of credentials then, fantastic ā they do! But if our intention was to only require that a user enters their single set of credentials once (when they log on to Windows) then weāve fallen short here. The code above requires the username and plaintext password, so weād have to present some kind of web-based login form to the user to request that information and get all this to work.
Enter NT LAN Manager (NTLM) Authentication
If a website (or intranet site) is part of the intranet or trusted zones (found in the Internet Settings control panel applet) then that site is allowed to pass a header requesting NTLM authentication. When it does, windows passes a header back containing some information about the currently logged-in user without the user being prompted for their credentials in any way.
I obtained some simple example code from someone called loune at Siphon9.net and modified so that it doesnāt require apache as the webserver. Hereās the PHP:
Thereās a big problem with this code, and the problem is that itās just decoding the user information from the HTTP header, and assuming that all is good ā thereās no work done to confirm that the header is genuine, and there is a possibility that it could have been faked. We could do some tricks like confirming that the page request is coming from within our local network, but that doesnāt really solve the problem ā HTTP headers can be manually defined by an attacker that knows what theyāre doing, and what weāre doing here is a bit like asking for a username and then just trusting that the user is who they say they are without doing any further authentication.
Combining the Two Approaches
Included in the NTLM authorization header that gets sent to the webserver during the passwordless authentication interaction described above is an MD4 hash of the userās password. A newer version of louneās code retrieves this and confirms its validity using samba. Unfortunately that setup wonāt work for me ā my intranet webserver is running a customized version of samba that comes with the software I use to manage the linux computers that are attached to my domain, and this trick just flat-out fails.
However, if I have a plaintext version of the userās password then I can use PHP to generate an MD4 hash of it for the purposes of comparison. So hereās my plan:
Scenario A: The first time a user comes to my webapp weāll get their credentials using NTLM, including the MD4 hash of their password. Since we wonāt know if this hash is valid, weāll present the user with a screen asking them to confirm their password (but not their username). When they input it, weāll confirm that their username and password combo is good using LDAP, and also generate an MD4 hash of the plaintext password that they entered to compare with what NTLM gave us. If nothing weird is going on everything should match. At this point weāll store the MD4 password hash for future.
Scenario B: When a user returns to our webapp weāll get their credentials using NTLM as before, and compare the hash NTLM gave us to our stored hash from their previous visit. If they match, weāre good, and thereās no need to ask the user to enter their password.
Scenario C: If the NTLM hash and the stored hash donāt match then the most likely scenario is that the user has changed their Windows password since their previous visit to our webapp. In that case weāll throw out the stored hash and start again at Scenario A.
If anyone knows of a better approach (is there a centrify Kerberos tool that I could use to get an MD4 hash of the userās password for the purposes of my comparison, for example?) then please let me know! Iād love to be able to achieve true passwordless SSO, but so far I canāt a method for doing so unless I switch my webserver from linux to Windows, and I donāt want to do that.