Wednesday, April 11, 2012

Why Not Use Port Knocking?

The robots currently at work knocking around for your guessable password could easily be repurposed to guess your Unicode password currently known as your port knocking sequence, and quite likely have been already. Plus, we already have authpf(8) for network-level restrictions on access.

Whenever you write about security in general and SSH security in particular (and for good measure also get slashdotted for your efforts), the comments inevitably turn up a variety of useful and less useful suggestions.

The April 6th, 2012 article about the possible preparations for a new round of slow bruteforcers was no different. Some of these recurring ideas are worthy of some more attention, and a more or less complete list is as follows -

  • Set up iptables with the recent module - a clear indication that the commenter did not, in fact read the article. That module implements a subset of the state tracking techniques from my PF rule set referenced in the article, but unlike the PF rules it is specific to the SSH protocol rather than a general technique. And inevitably, the actual configuration the poster presents would in fact not trigger on the kind of traffic we see from the Hail Mary Cloud with its signature slow brute force attempts. Misunderstandings of this kind keep cropping up, for no good reason.

  • Disable root logins over the network. Yes, this is obviously useful, and if I remember correctly, PermitRootLogin no is the default in recent OpenSSH versions. Do remember to check what the setting is on your kit, obviously.
     
  • Use packet filtering or other means to restrict where users can log in from. Once again, a very useful suggestion, but some sites may require shell access from semi-random locations, so practical issues (aka real life) can interfere with this one.

  • Disable password logins entirely, and allow only key-based logins. Another a very useful suggestion, and one that is useful for several reasons. You do get the downside of managing your keys securely, but key-only logins are generally recommended practice and should be encouraged.

  • Set up your sshd to listen on a non-standard port. Several sites report less log file noise after moving their sshd to a non-standard port. Easy to implement and possibly useful, but if a prospective attacker takes the time to do a real port scan for active services, they will find your sshd running on port 22222 fairly easily. I'd say moving your sshd to listen on a non-standard port postpones rather than solves the problem.

  • Use denyhosts to automatically add hosts with too many failed logins to a blacklist. Yes, an excellent idea in principle, but it comes with one caveat: Creating a blacklist automatically leaves you in charge of maintaining its content. Blacklist maintenance -- as in keeping your lists up to date and useful, is a complex enough issue that I'm fairly confident it is worth a separate column, and I most likely will return to the issue later.

  • Why not set up port knocking? Well, as you may have guessed I'm about to tell you about a few whys not. I've been circling port knocking as an article subject for a long time, and now it seems that the time has come to outline why I think that port knocking emphatically not a good idea. Read below for my take on the subject.
Port Knocking: Threat Or Menace?
I must admit that the port knocking concept fascinated me at first, but all implementations had the downside of adding yet another piece of clearly experimental software to my system along with somewhat convoluted procedures for setting the thing up, so I left it for later.

At each of the later dates when i decided to look into the topic again, I discovered new reasons not to like the method (read this thread on OpenBSD-misc for a particularly disenchanting example -- how's single-packet unconditional remote reboot or remote kill your sshd for a feature?). Finally, it was the appearance of the slow bruteforcers (also known as the Hail Mary Cloud) that made me finally decide that port knocking is neither particularly elegant or actually useful.

One of the reasons the port knocking concept seems so deceptively attractive is probably that it is fairly easy to explain (see the Port Knocking article on Wikipedia for a fairly thorough treatment). The target system runs with no services accessible to the outside, and in order to gain access a remote system must send a specific, pre-determined sequence of packets aimed at specified ports. Unless you know (or are able to guess) the correct sequence of ports, you will not be able to gain any access to the target system.

And after this explanation, the naive listener goes
"Oh, how clever, we send packets to random ports!
Nobody will ever be able to guess our combination!".

But first impressions are not always correct. First, please take one step back and explain to me what problem this is supposed to solve. If the answer is the old canard "all the insecurities of that horrible SSH protocol", I will ask you to please point out to me just what those are, and if you will, describe which parts of what I am about to describe actually adds value (that's "security" to you) in a real life scenario.

So let's go straight to the the core of the matter, and consider what actual information content an attacker would need to get right in order to gain access to the system. The hopeful would need to know or guess a sequence of TCP or UDP ports. In both cases, the range of possible ports is a 16 bit number, with a total of 65536 possible values.

 Each value is a 16-bit number, with a size of two bytes, or equal to two ASCII characters or one Unicode character. Port knocking examples generally do not run to more than three packets, which means that the minimum amount of information a prospective attacker would need to get right in order to gain access is six bytes, equal to six ASCII characters or three Unicode characters.

Seen from this angle, all port knocking gets you is a somewhat cumbersome method for encoding your unicode password. In most implementations, that password would even be common to all users, with no easy way to change it.

And of course, in almost all other contexts where passwords are used, most sites dictate that you choose a personal password that is longer than six bytes. So that's at least two failures rolled into one: a password that's common to several or all users, and one that is hard or impossible to change, possibly even short enough to fail even basic guidelines.

The amount of information an attacker would need to get right, measured in number of bits or bytes is a fairly useful measure. I admit it's a fairly crude measure, but at least real data are easy to obtain, and since we are talking here about designing and maintaining systems, not writing them, how much cryptographic fun goes into generating the necessary data is not really relevant to the context. Proper cryptography is essential to maintain confidentiality and ensure the integrity of the communications, but we'll leave the details of the crypto bits for a later time.

One other factor that speaks against most implementations of port knocking is that the system runs with no ports open, and a daemon that parses firewall logs for anticipated sequences of port numbers contacted as the sole authority to determine whether access will be granted. We all know that all non-trivial software will contain bugs, so what are the chances that even a fairly simple deamon will at some point in the future be confronted with a situation that makes it terminate, making your system inaccessible (or as we say around here, "bricking your system")?

And one other thing, how do you spot an attacker? If you're looking for access attempts to closed ports, as you should be if you run a port knocking setup, how do you spot an attacker in the primal soup of noise that will anyway be arriving at your public interfaces? Do you, for example, record all sequences that could possibly be seen as unsuccessful attempts and put the source addresses in a "definitely banned" list?

I have not seen any useful answer to that one either. Doing a sudo tcpdump -nettti pflog0 action drop (that is, using tcpdump(8) to extract information on blocked traffic from my packet filtering setup, directly from the log device) on my gateway here certainly shows me access attempts to ports I didn't know much about -- what legitimate purpose, for example, is UDP port 287 actually used for? Once again it all boils down to the fact that if you rely on port knocking, you're really only implementing a cumbersome variant encoding of relatively short passwords.

On a side note (thanks to Adrian Close for reminding me), anybody who sits in the signal path between your knock target and somebody executing the correct knock sequence will be able to record the connections (using, for example, a netflow based tool -- expect to see Netflow-themed columns here in the future), making your closely guarded secret roughly equivalent to a plaintext password much like we used to see in old-style ftp setups.

Next, let us let us instead look at the thing port knocking is trying to protect, namely the sshd service. Typical SSH connection setup involves the transfer of at least the public host keys (typically in the some hundred bytes to several kilobytes range), followed by the user authentication which can involve passwords, keys or a number of cryptography based toys^H^Hols, in any case producing further transfer and validation of anything up to several more kilobytes of data (and in some setups, use of out-of-band methods like short lived keys generated by special-purpose devices) before any meaningful access is possible. The exchange and validation of host keys in itself involves more information that the attacker would have to get right in order to gain access than is involved in any port knocking setup I've seen demonstrated. And remember, host keys exchange is only one early steps of several on the way to authentication.

For the actual math on how data sizes and entropy is significant (yes, in some contexts, size does matter), see the Wikipedia entry and a somewhat shorter summary by Phil Ratcliffe.

All port knocking implementations share the problems I've outlined, and their proponents have tended to ignore or gloss over the issues rather than address them. It is an unfortunate and perhaps embarrasing fact that port knocking in practice comes down to implementing and maintaining a separate set of passwords, and in all likelihood you will be using tools that are less appropriate for the task than the ones that already come with the base system on any reasonable Unix-like system.

In an environment where we know there are large, distributed efforts underway to crack easily guessable passwords, it is naive not to assume that the same password guessing tools could be adapted to try sequences of TCP or UDP ports instead of character sequences. If there is in fact any significant number of systems that use port knocking today, it would surprise me if this isn't already happening. The way I see it, the belief that port knocking in fact offers an effective protection against attackers is a dangerous misunderstanding and probably only serves to divert your attention away from the real problems involved in keeping your systems running in a secure fashion.

If you're still not convinced, I'll show you a better way

Now, if you really, really want to implement port knocking anyway, I'll let you in on a dirty little secret: you don't need a write a new daemon or install anything besides what's already in the OpenBSD base system (or for that matter, on other PF-equipped BSD variants).

You can implement port knocking in a relatively straightforward manner via minor contortions in your PF rule set, and I know for a fact that people have done just that. The examples I was thinking of before I started writing this piece appear to have been removed from the public eye, but with a little effort using the obvious keywords, some well intended but actually quite bad advice on how to put together a seemingly elegant implementation will be on its way to you.

Be aware that you will be doing things the tools were not designed for, and more likely than not you will soon find yourself in a mess of rinky-dink workarounds that keep fighting back.

If, however, what you really want to do is create a separate line of defence with its own round of authentication required for access, your OpenBSD base system already contains a suitable tool in authpf(8). It's fairly easy to come up with a setup that lets your users log in to the authenticating gateway, using any authentication method sshd(8) supports.

You can even run your inital sshd(8) on a separate, non-standard port if you like, and a successful login at that point will only result in a set of packet filtering rules (possibly tailored to that specific user) being loaded. When the ssh session terminates, the rules are unloaded, and the system reverts to the state it had before the user authenticated.

The tools are as flexible and robust as you have come to expect from the OpenBSD system tools (even if you're not running OpenBSD at the moment, you are by a large margin more likely than not using OpenSSH as your ssh client and server, which is supplied to the world by the OpenBSD project), and you can tailor your access profiles and services to your heart's content so they fit your needs.

If you've read this far, you've probably also come to the conclusion that port knocking is really not the way to go. Other, more appropriate tools exist. But how to go about improving the situation?

My suggestions follow below, with some specific suggestions that may have a personal slant.

A Path To Self Improvement

As with any problem that is really rooted in a lack of information or knowledge, educating yourself is the key. Read up on relevant subjects, go back and determine what problem you are actually trying to solve, and then perhaps shop around for tools that will help you solve that problem.

My suggestion is to go to first to www.openbsd.org and browse the resources available there, including the online manual pages and the extensive FAQ document that actually serves quite well as a user manual.

The next logical step is to go to the Orders page and order at least one copy of the latest release (and if you're reading this not too long after publication, do pre-order the upcoming release as well). If you're too impatient to wait for the postal services' mules to load up and deliver, you could download the release (or a -current snapshot if you're feeling adventurous) from a suitable mirror site and start installing right away.

If you don't order any merchandise, you really should head over to the donations page and donate at least an amount equal to the cost of the CDs and other stuff you would otherwise have ordered. There are several fit and frightening ghosts primed and ready to come haunt you if you don't.

If you want some literature to back up the official OpenBSD documentation, you could do worse than order The Book of PF, 2nd edition and Michael W. Lucas' 2012 book SSH Mastery. Using those links here will ensure the OpenBSD project gets a larger-than-otherwise cut of the money as net profit. There are also electronic versions available for both titles (see The Book of PF home page or the SSH Mastery home page for details). If you're really short of funds, even the free, online PF tutorial that the Book of PF evolved from will get you started.

And finally, there are few places better to collect useful information about OpenBSD and its sister operating systems than the various local and regional user groups and conferences such as the upcoming BSDCan in Ottawa, Canada May 9th through 12th, 2012 or EuroBSDCon, with this year's version to be held in Warsaw, Poland October 18th through 21st.

These events attract both regular users (some of them quite proficient) as well as a useful subset of the people who write and maintain the tools.
Whatever you end up doing, buying or downloading, please keep in mind that it's your responsibility to keep your systems in the best possible shape, and make sure you monitor them properly, via log files or other means. Also see the Pledge of the Network Admin for my (slightly humorous) take on that responsibility. Along with the better parts of the tutorial, it made its way into the book, too.

Thanks
to Michael Dexter, Kristaps Dzonsons, Michael W. Lucas, Thordur Bjornsson and Henning Brauer for insightful comments and valuable input.

Copyright © 2012 Peter N. M. Hansteen


If you found this article inspiring, irritating, useful or otherwise moving, please let me know via the comments field just below here, or if you like, drop me a line at peter at bsdly dot se.

Note: A Better Data Source Is Available
Update 2013-06-09: For a faster and more convenient way to download the data referenced here, please see my BSDCan 2013 presentation The Hail Mary Cloud And The Lessons Learned which summarizes this series of articles and provides links to all the data. The links in the presentation point to a copy stored at NUUG's server, which connects to the world through a significantly fatter pipe than BSDly.net has.

15 comments:

  1. If the bulk of them actually worked that way, yeah it would be nearly useless. Reality is there are numerous implementations that have some kind of crypto integration that's safe from replaying. And there are few that are as simple and dumb as just listening for a sequence of ports.

    I'm not a fan of it in general, but I have seen situations where it could be useful for reasons other than security. For instance, my cable ISP will periodically port scan you from varying source IPs and will shut you down for TOS violation if you have a port open ("that's a server!!1!"). In that scenario if I needed a TCP service open on that connection, port knocking would be a good solution. And if it is one of the many solutions that's protected from replay attacks and not insecurely implemented, it can have security benefits. I'd argue you have far worse issues that should be addressed by something other than obscurity if your system can actually be compromised from crud like SSH brute force attacks though.

    ReplyDelete
  2. I've been using the excellent Blockhosts for a number of years now. It's not a port knocker per se but more in the vein of denyhosts. It has a few advantages like not being protocol specific ( it will handle those pesky pop and imap trawlers amongst others ), and has configurable timing values. No management overhead as blocked hosts are automatically expunged after a config'd period of time.

    ReplyDelete
    Replies
    1. The maintenance of tracking keys is mentioned as a valuable tradeoff but a single automated file is too much to maintain? This makes no sense. Tarpitting is much more common than key-based logins because it serves multiple purposes BEFORE even getting to identification.

      Delete
  3. Good Lord.

    You gloss over my recommendation of xtables-addons and use of the 'tarpit' module as if it has no bearing?

    And then you have the temerity to say it is an ssh only solution? Incorrect.

    tarpit will operate on any listening port.

    Honestly, I think you are the one who didn't read:
    http://dev.medozas.de/files/xtables/xtables-addons.8.html

    There's a bonanza of target features one can employ in iptables, not just tarpit. It includes port knocking, delude, chaos, port scan detection, geoip (block ip ranges by country). Here's a table of modules:
    http://xtables-addons.sourceforge.net/modules.php


    I think its irresponsible of you to not mention the importance of this tool.

    Here's the details of tarpit usage:

    /**
    TARPIT

    Captures and holds incoming TCP connections using no local per-connection resources. Connections are accepted, but immediately switched to the persist state (0 byte window), in which the remote side stops sending data and asks to continue every 60-240 seconds. Attempts to close the connection are ignored, forcing the remote side to time out the connection in 12-24 minutes.

    This offers similar functionality to LaBrea but does not require dedicated hardware or IPs. Any TCP port that you would normally DROP or REJECT can instead become a tarpit.

    To tarpit connections to TCP port 80 destined for the current machine:

    -A INPUT -p tcp -m tcp --dport 80 -j TARPIT

    To significantly slow down Code Red/Nimda-style scans of unused address space, forward unused ip addresses to a Linux box not acting as a router (e.g. "ip route 10.0.0.0 255.0.0.0 ip.of.linux.box" on a Cisco), enable IP forwarding on the Linux box, and add:

    -A FORWARD -p tcp -j TARPIT

    -A FORWARD -j DROP

    NOTE: If you use the conntrack module while you are using TARPIT, you should also use the NOTRACK target, or the kernel will unnecessarily allocate resources for each TARPITted connection. To TARPIT incoming connections to the standard IRC port while using conntrack, you could:

    -t raw -A PREROUTING -p tcp --dport 6667 -j NOTRACK

    -A INPUT -p tcp --dport 6667 -j TARPIT
    **/

    Dietrich T. Schmitz
    Linux Advocate, Human Being
    https://dtschmitz.com

    ReplyDelete
  4. I had to modify knockd (port knocking) to ignore repeated retries so it wouldn't invalidate the entire knock on a network induced repeated port hit.
    I also use longer sequences.
    The flavor I like will open the target port (i.e. high sshd port) for 10 minutes.... (for new originating ssh clients that is)... before changing the iptables back. It's just one extra layer of precaution.
    It's nice since it will knock open from anywhere as long as I give the entire sequence.

    ReplyDelete
  5. I get the impression that the author has never actually implemented port knocking.
    I found it incredibly simple.
    To prevent any issues, knockd can be auto-restarted periodically using cron (if one worries about lock ups that do not allow access).
    I use a port knock program for the Android that, after knocking, calls the ssh client....
    When I have to reset a VM in my rack I just port knock from my Android and go straight to my own workstation and then tunnel to the virtual host and then manage the client.

    ReplyDelete
  6. There is an open source project called Taferno on sourceforge.net. It implements simple Two Factor Authentication along with dynamic firewall rule modification that effectively prevents/reduces brute force attacks.
    taferno.sourceforge.net

    ReplyDelete
  7. http://www.mnxsolutions.com/security/two-factor-ssh-with-google-authenticator.html

    ReplyDelete
  8. Security heterogeneity.

    Port knocking per-se is not a bad thing - it has drawbacks and it has a place.

    I personally use rate-limited port knocking - if you knock, port 22 opens. If you try port 22 without knocking then you're locked out for a few mins (ie even a correct knock fails).

    Main benefit: cleaner sshd logs; which means that when I *do* see entries in sshd logs, they mean something.

    Since port knocking has too many drawbacks to be widely implemented (and therefore isn't worth much effort on the part of the bad guys), and since I'm not a special target, it probably adds value for me.

    Of course I otherwise configure sshd as if port knocking were not present

    ReplyDelete
  9. Port knocking is about lowering your profile. Bots scan IP ranges looking for zero-day vulnerable hosts. Port knocking elides the host from the list of hosts to attack. This sort of attacker cannot afford to attempt to brute force any significant number of hosts; they're looking for vulnerable listeners.

    Just a few days ago, non-NLA terminal servers had a zero-day remote hole appear out of the blue. This is what port knocking can, in limited cases, help defend against.

    ReplyDelete
  10. Hi Peter,

    Your article makes a few good points, but as some commenters above have already stated, you forget to mention that security as a whole should be looked at in terms of 'defence in depth'. Port Knocking shouldn't be looked at in isolation, but rather as an added layer of security (where it performs moderately well, despite many implementations being deeply flawed).

    But my bigger gripe is that your article's focus on port knocking means that you're looking solely at an outdated and largely deprecated mechanism. Modern 'port knocking', that is more widely used today, is Single Packet Authorization (SPA). SPA doesn't suffer from many (if not all) of the drawbacks experienced in PK. No more weak sequences, out-of-order delivery, no more crypto issues or replay attacks.

    In fact, SPA came out precisely because many of the points you made were being discussed and resolved. The ability to hide all services from external access is a huge benefit, meaning that, for the most part, you no longer have to worry about the security of individual services, or how strong your authentication credentials are, and instead ensure that the SPA daemon itself is secure enough to withstand exploitation.

    If you and your readers are interested in a more in-depth comparison of PK/SPA, I'll direct you to a paper I wrote on the subject back in 2006: http://www.securitygeneration.com/single-packet-authorization/

    Note that I don't develop any implementations of SPA myself, but if anyone wants to play around with one, I highly recommend the fwknop project (http://www.cipherdyne.com/fwknop), to which I've contributed, and which I think is the more advanced implementation available.

    ReplyDelete
  11. I'd just like to point out that denyhosts works like Blockhosts (as pointed out by Robby): the blacklist is maintained by the system with automatic expiry of blocked hosts, i.e. (usually) no management :)

    ReplyDelete
  12. authpf has its uses, but it also has many of the same problems as running your "main" sshd on an open port, e.g. opening yourself to a bunch of writes to the logging disk, and high CPU use from people trying to connect (which also applies if you disable passwords and only allow public key authentication). The CPU use is especially "fun" when the processes are too short-lived to appear in top(1)'s display, you just see the high CPU% figures with no apparent cause.

    This can be mitigated by blocking people who connect too often (either via rate-limiting firewall rules or denyhosts/sshguard/etc) but that's equally acceptable for an authpf sshd as the main sshd, the same reasons you might not want it for main sshd apply here too.

    ReplyDelete
  13. I'm considering using port knocking as an extra layer of protection. Remote users need to knock before regular authentication (i.e knock, login, two factor...).

    Another thing I am considering is using port knocking to whitelist IP addresses on a staging version of a website. Currently, if an IP has not been whitelisted to view staging.site.com they are served a blank page and an email is sent to an admin, who then follows a link which adds their IP. With knocking, users can simply knock and then their IP will be added. This can be automatically time limited.

    An as *extra* tool, I can't see anything wrong with port knocking.

    ReplyDelete
  14. You can just use xtables (http://sysadm.pp.ua/linux/xtables-addons.html) or this instruction - http://sysadm.pp.ua/linux/iptables-antiscan.html

    ReplyDelete

Note: Comments are moderated. On-topic messages will be liberated from the holding queue at semi-random (hopefully short) intervals.

I invite comment on all aspects of the material I publish and I read all submitted comments. I occasionally respond in comments, but please do not assume that your comment will compel me to produce a public or immediate response.

If your suggestions are useful enough to make me write on a specific topic, I will do my best to give credit where credit is due.