Category Archives: Security

Setting up TOTP for OpenVPN with OATH toolkit

This one took some effort. Actually, my first setup had Authy’s OpenVPN plugin and their 7 digit TOTP flavour, but it always felt wrong to require a service in the middle for something which needs a shared secret and the clock to be within the same window. There’s no actual requirement for a service here. Twillio also decided to stop maintaining the Authy plugin, so there’s not a lot of great news in the middleware department.

There’s no TOTP support in OpenVPN. OATH toolkit came to the rescue. To make the two work together, there’s a PAM module for OATH, pam_oath. OpenVPN has a PAM plugin. Now, this seems as easy as RTFM, but, it isn’t. Couple of years ago when I made this setup first, there was no comprehensive end-to-end guide on how to achieve this. PAM isn’t the most friendliest environment to debug. When you add the slowdown of having to input random 6 digit codes for every try to see what’s going on, the whole process comes to a grinding halt.

The OpenVPN authentication strategy:

  1. Static key tls-auth. This is something which OpenVPN does and it is recommended in most cases. The second benefit of having this on is that the server doesn’t identify itself as OpenVPN, like it does without tls-auth. There’s no banner to grab to please those doing enumeration. If the right TLS key with the right direction isn’t presented within a fairly short window of time, the connection is simply closed.
  2. Mutual TLS authentication. This is pretty standard in the OpenVPN world where you have a CA, the server gets a cert + key, the clients get certs + keys. This is an excellent guide on how to create a CA. I’m pointing this one out as most guides forget to mention the X509v3 extensions. OpenVPN is honouring the server_cert and the usr_cert extensions which I have accidentally discovered trying to do mutual auth with a cert issued by the same CA with the server_cert extension. Pro Tip: the CRL must not be expired as it drops the mutual authentication despite the server and client having valid certificates. Other people and I have learned this the hard way. This isn’t an OpenVPN specific problem as, for example, I have had the same problem with Haproxy-based mutual authentication and very unhelpful errors about “expired certificates” when the expired bit is the revocation list itself.
  3. TOTP via the PAM plugin. Because not all OpenVPN clients can handle the OTP field, this is implemented on top of the username + password fields. This isn’t an issue for the CLI client, but most GUI options aren’t smart enough to prompt for username + password + OTP. Given that this is the 3rd authentication factor besides the static key tls-auth and mutual TLS auth, the lack of password isn’t a problem.

Plugging pam_oath into OpenVPN is as easy as:

reneg-sec 0
plugin /usr/lib/openvpn/ "openvpn login USERNAME one-time PASSWORD"

Bear in mind that the actual plugin path may be different on your distribution. This is an Ubuntu example. The reneg-sec 0 option disables the re-keying which otherwise will drop the VPN connection unannounced. By default this is set to 3600 seconds and I had a fun time determining why my connection was dropping until I realised it happens periodically. Sometimes it wouldn’t even reconnect after such drop. The problem is caused by the fact that the re-keying can not happen with the credentials provided upon the initial connection since by design the TOTP has only a limited amount of time during which the OTP is valid.

The first bit after the .so, “openvpn”, is the name of the PAM module. The “login” argument gets the USERNAME value from the OpenVPN authentication dialogue, and the “one-time” argument gets the PASSWORD value from the OpenVPN authentication dialogue. The client configuration needs auth-user-pass to prompt for the username and OTP, besides the mutual TLS auth configuration options, ca, cert, and key.

The PAM module is configured in /etc/pam.d/openvpn and reads as follows:

auth requisite usersfile=/etc/openvpn/users.oath window=5 digits=6
account required

The first line of that module is where pam_oath is actually referenced. The “usersfile” path is where the credentials are stored i.e the “login” – checked against USERNAME and “one-time” seed – checked against PASSWORD. 6 digits is the typical TOTP used by most authenticator apps, although FreeOTP supports 8 digit TOTP as well. The window sets the search depth rather than being a reference for a time window.

The second line is just waiving by anybody who’s passing the OTP challenge. That line took the most effort to get there after a lot of groaning, swearing, and generally ranting about PAM and non-sensical error messages. Turns out, an actual account is required in the PAM flow after auth, but there isn’t one as there’s no account anywhere, whether the system itself or another authentication system, to match the OTP username. pam_permit must not be used without having a proper use case. This is one of those use cases. Otherwise, it may be a catastrophic security issue if used as a solution for every PAM problem. You have been warned!

The users.oath file itself needs to be properly protected as all the pre-shared OTP secrets live there. Basically root rw and nothing else. Even though my openvpn worker process, i.e the one taking in client connections, runs as nobody, that file is still readable/writable as the master process runs as root. Every time a successful challenge is passed, pam_oath updates that file.

The structure of users.oath is: Option User Prefix Seed. The manual is not brilliant, therefore I can’t tell why the prefix is just a dash, but for all intents and purposes, this is unused. The Arch wiki explains this better.


HOTP/T30/6 foo - 16a142bc8c34f2682f219dd9e75f76c3a7b7d62aad85047411bef4beb97514b8

TOTP is a particular case of HOTP i.e the counter is substituted for a non-decreasing time value, hence the Option reads HOTP/T30/6 which makes it the most common TOTP scheme – 30 seconds time step size with 6 digits OTP. Authy, Google Authenticator, FreeOTP, etc. support this. 6 digits is a commonly used number, not the mandated number of digits. The number of digits must match the digits value passed as argument to pam_oath.

“foo” is the username value. I have only tried alphanumeric values in there, therefore I can’t really tell what OATH tookit truly supports i.e whether dashes, dots, and underscores are supported. I know there’s groaning in some tooling when UNIX usernames contains characters like dot, hence mentioning this.

The “-” dash is the prefix.

The hex code is the pre-shared secret. RFC 4226 says:

The algorithm MUST use a strong shared secret. The length of the shared secret MUST be at least 128 bits. This document RECOMMENDs a shared secret length of 160 bits.

RFC 4226 – Section 4, R6

That secret is hex encoded, which means it uses 2 characters for each byte. That makes the minimum length 32 hex chars to encode 128 bits.

For example, one can use this to generate secrets:

oathtool --verbose $(head -10 /dev/urandom | sha256sum | cut -b 1-64)

That line generates 256 bit secrets which is above the recommended value. While sha256sum itself generates 256 bit values, putting that through oathtool has more benefits. The hex secret value is simply reflecting the input hex secret.

Sample output:

Hex secret: 16a142bc8c34f2682f219dd9e75f76c3a7b7d62aad85047411bef4beb97514b8
Digits: 6
Window size: 0
Start counter: 0x0 (0)


The interesting bits (pun not intended): the Hex secret and the Base32 secret. The hex encoded secret of the TOTP goes into users.oath. The Base32 encoded secret may be used to generate QR codes which may be easily read with an authenticator app on your phone, like Authy, Google Authenticator, FreeOTP, etc.


qrencode -o foo.png 'otpauth://totp/[email protected]?secret=C2QUFPEMGTZGQLZBTXM6OX3WYOT3PVRKVWCQI5ARX32L5OLVCS4A===='

The output of qrencode looks like:

You can scan that with an authenticator app to check that it works and check against:

oathtool --totp --digits 6 16a142bc8c34f2682f219dd9e75f76c3a7b7d62aad85047411bef4beb97514b8

It should read the same value provided the time is in sync on both your devices. You can even get future TOTP’s with the window argument:

oathtool --totp --digits 6 --window 2 16a142bc8c34f2682f219dd9e75f76c3a7b7d62aad85047411bef4beb97514b8

That prints the current TOTP plus the next 2.

Two things to keep in mind:

  1. Reusing the example secret which I have used here would be catastrophically stupid.
  2. Don’t scan QR codes when strangers on the Internet tell you to. While the one from above is legit, that may not always be the case.

WiFi adapters card-to-card packet injection test

Update: I made something which I call aircrack-db. It contains the actual results of my hardware testing for the Wi-Fi attacks and it is kept up to date. Consider this article to be obsolete.

As I started working on the Offensive Security Wireless Attacks (WiFu) course, I needed to know which of the wireless interfaces that I own is capable of doing the job. Turns out that the information is hard to find, even coming from fellow pentesters.

I’ve to admit, my wireless attacking skills were next to null before starting this course. I had to read a lot of stuff before getting some hardware. I had the surprise to see that the most capable wireless interface, so far, is the one that came with my really-really-really-really old notebook which I use as lab environment.

I bought an Unex DNUA-93F because it has the same chipset as ALFA AWUS036NHA (*), but unfortunately it doesn’t support (yet?) the fragmentation attacks. Anyway, the card is really discreet and it may be a good choice for some inconspicuous testing. I also have an ALFA AWUS051NH (**) mostly because it supports both 2.4 GHz and 5 GHz. I don’t really care about 802.11a, but 802.11n works on both frequency bands.

Here are the results for the card-to-card packet injection test:

ASUS A6M built in (miniPCI): Broadcom Corporation BCM4318 [AirForce One 54g] 802.11g Wireless LAN Controller (rev 02)

Driver: b43

Attack -0: OK
Attack -1 (open): OK
Attack -1 (psk): OK
Attack -2/-3/-4/-6: OK
Attack -5/-7: OK

Unex DNUA-93F: ID 0cf3:9271 Atheros Communications, Inc. AR9271 802.11n

Driver: ath9k_htc

Attack -0: OK
Attack -1 (open): OK
Attack -1 (psk): OK
Attack -2/-3/-4/-6: OK
Attack -5/-7: Failed

ALFA AWUS051NH ID 148f:2770 Ralink Technology, Corp. RT2770 Wireless Adapter

Driver: rt2800usb

Attack -0: OK
Attack -1 (open): OK
Attack -1 (psk): OK
Attack -2/-3/-4/-6: OK
Attack -5/-7: Failed

I borrowed some interfaces just to extend the test.

Canyon CNP-WF518N1 ID 148f:3070 Ralink Technology, Corp. RT2870/RT3070 Wireless Adapter

Driver: rt2800usb

Attack -0: OK
Attack -1 (open): OK
Attack -1 (psk): OK
Attack -2/-3/-4/-6: OK
Attack -5/-7: Failed

ID 0bda:8176 Realtek Semiconductor Corp. RTL8188CUS 802.11n WLAN Adapter

Driver: rtl8192cu

Attack -0: OK
Attack -1 (open): OK
Attack -1 (psk): OK
Attack -2/-3/-4/-6: Failed
Attack -5/-7: Failed

Don’t know the manufacturer of this Realtek card. It was sold naked (just the board with a soldered USB interface) for the reason that it worked OOTB with Raspbian. To state the obvious, RTL8188CUS is basically useless beyond deauth and fake auth.

The -1 to -9 are, of course, the aireplay-ng attack flags. My cards were tested with both the drivers that came with the latest Kali 1.0.8 / Linux 3.14.5-1 and with the compat-wireless compat-drivers backports package (guys, pick a name that sticks) 3.16.2-1. The borrowed cards were tested only with the backports drivers. I also used the mac80211.compat patch for the backport package provided by aircrack-ng with Mathy Vanhoef’s compatdrivers_chan_qos_frag patch, with some fixes from Devil_D. This patch can be merged with “patch -p1 -F3” as it fails to match the offsets.

I didn’t get the fixed channel issue with aireplay-ng until I started to do more stuff with it, with or without updated drivers. Even though I applied various patches, I found out that the issue is triggered by Gnome’s Network Manager if the service is restarted. It reconnects ALL of the installed wireless interfaces if it has a device profile for that particular interface. If it’s started again while a VAP is monitor mode is running, I get a fixed channel issue. Sometimes it sticks to -1, sometimes it sticks to 7. The channel of the interface can’t be changed after that.

iw and iwconfig silently fail to change the channel. A reboot solves the issue. Presumably, removing the modules with rmmod and loading them back in also fixes the channel issue. Adding the –ignore-negative-one flag to aireplay-ng doesn’t do anything useful and the attacks still fail. I am running the latest aircrack-ng suite, so the “it was patched against this” doesn’t apply here. I think the lesson here is: don’t start network-manager if a VAP created by airmon-ng is running. airmon-zc doesn’t solve the problem, as opposed to which is stated on aircrack’s issue tracker.

The conclusion is: I am waiting for an ALFA AWUS036H.

Update: the fixed channel issue appears when the network-manager daemon is restarted. This happens with or without the presence of the monitoring VAP. However, my hunch was correct. Unloading the driver with rmmod and loading it back with modprobe solves the fixed channel issue and the attacks may resume.

(*) AWUS036NHA – something that people had “no issues with it”, while Unex DNUA-93F is an adapter which is endorsed by FSF as it needs “completely free software for operating”
(**) also, something that people had “no issues with it”, but I guess they didn’t try the fragmentation attacks with it

portspoof trolling

Marius once told me about portspoof. A service to troll those who use various scanners by feeding the scanners with false results. Well, while the idea is good, I’m wary about a service like this as this is the kind of service where you wouldn’t want a buffer overflow.

Giving it a run inside a VM, I noticed something odd when using nmap’s service and version detection probes. This happened on the lower ports (1-50). Then I started to look at something that started to look like a pattern, therefore I increased the port range to include 1-50. portspoof is indeed a tool that trolls baddies and pen testers.

Ran it with:

nmap -sV --version-all -p 1-50

Really smooth guys, really smooth. Sometimes you have to see the big picture:

How to fix nginx and PHP/FastCGI PATH_INFO issue

You may be disappointed by this statement: you don’t. nginx has something in it that’s broken by design, while the author didn’t bother at least to reply my email, explaining the situation. I can demonstrate this by using some comparisons.

Apache (+mod_php5) knows the difference between a script and a PATH_INFO input request that ends in .php. It would be ridiculous not to do so since the PHP runtime is part of the webserver itself. Didn’t bother to try various Apache + mod_fcgid configurations since most of the time Apache simply wastes my own time. lighttpd binds the FastCGI proxying to the file extension. This is the part where nginx fails: it tries to use a one-size-fits-all configuration logic that actually doesn’t fits all the usage modes. The FastCGI pass is done into a location (not file / file extension!) directive that doesn’t tell anything about the nature of the input. Let’s take a look at this example:


Except nginx, both Apache and lighttpd won’t create a mess out of a path like this. A location directive is a little big vague and no amount of regex will ever fix this. Of course, you can fix it for every single damn virtual host, but then again, can you spell boilerplate? Having different configurations for every virtual host, when clearly for other web servers this isn’t a bundled “feature”, is not fun from the system administration point of view. Usually I generate the virtual host configuration from a bash script I wrote myself. I have configuration templates for all of the applications I administer, thus is all about flags and options, unlike manually writing configuration files. Working around nginx’s inabilities to tell which stuff is which could only mean that I have to write a whole bunch of configuration boilerplate for each type of application. Doesn’t sound like one-size-fits-all anymore. Is that fun? Let me rephrase that: that’s not fun. In case AWS disappears into a black hole, I can recreate everything from scratch in a matter of minutes onto a complete new hosting service while changing Cotendo origins is a child’s play.

Here are a bunch of proposed solutions for something that can turn out into a remote exploit. I’ve being using for quite a while the same solution as provided by one of the people commenting the article:

if (-f $request_filename)
    fastcgi_pass php_upstream;

Usually because this is more readable than try_files. I usually tend to understand code blocks better.

Of course, a proper PHP script won’t save any uploaded junk to a public accessible location, but what sysadmin trust his coders anyway? I usually don’t. That doesn’t mean that they don’t do a good job, but mistakes happen. I can’t made every living thing to be as paranoid about security as I am. This exploitable situation happens when people validate their upload via the $_FILES array. I have news flash for you: the MIME type defined by the $_FILES array is defined by the browser. The browser does a lousy job at providing a proper MIME. It matches a specific MIME based onto the file extension. PHP file with JPEG extension, anyone? fileinfo would be the proper alternative. PHP should deal with this junk by design, but that’s a whole other joke about the design of PHP.

Getting back to PATH_INFO. The juicy part is that basically you can extract the PATH_INFO from an input path by using fastcgi_split_path_info, but that directive uses … regex. Which brings us to the above statement: no amount of regex will ever fix this crap. Let’s take a look at $request_filename by throwing a custom debug logging configuration that places some stuff into the access_log. Guess what, the $request_filename for the above example is … /directory.php/file.php/pathinfo.php while it’s pretty clear that the actual request filename is /directory.php/file.php. Which is the other broken-by-design thing that nginx features. Q: What damn server side variable would ever lie to your face that a $request_filename is not exactly a file? A: duh!

This doesn’t mean that you should throw nginx and PHP-FPM away and go back crying to Apache. Just simply avoid the PATH_INFO junk. However, even by using my proposed configuration directive aka check if $request_filename is actually a file before doing the FastCGI pass, you can still use fastcgi_split_path_info for a limited amount of work. fastcgi_split_path_info can replace the need for doing URL rewrites by simply using:

if (!-e $request_filename)
    rewrite ^ /index.php last;

This works for a lot of stull like WordPress, Drupal, or Zend Framework. It works for certain stuff, except the stuff containing .php somewhere. I might want to use /%postname%.php as permalink structure in WordPress. Guess what … with a properly configured nginx (cough!) + the above rewrite rule replacement simply can’t. You have to go back to:

if (!-e $request_filename)
    rewrite ^/(.*)$ /index.php?q=$1 last;

Which is exactly what I did. For all my apps. Happens to be more deterministic by nature, while I tend to sleep better when I can predict the request pipeline, no matter which is the input junk.

I guess I could give Hiawatha a run which seems to be lightweight enough to support PHP with a threaded architecture. PHP is process based, doing blocking I/O by design, therefore the web server is rarely the actual bottleneck.

Password Hashing in PHP with bcrypt

I’ve being doing it wrong for years. Till 5.3, PHP didn’t help very much either with it’s broken bcrypt implementation. So here goes the vicious circle. However, there’s no excuse for getting smarter as the tools I’m using get smarter. Here’s a couple of reasons.

For quite a while I’ve being using my own implementation of phpass. There are a couple of reasons for doing my own homework instead of using the already existing solution:

  • The Suhosin patch doesn’t play nice with PHP extensions available as binary blob only as it breaks the ABI. Therefore a consistent bcrypt implementation lacked before PHP 5.3 since at most I could use just the Suhosin extension.
  • The random seed generator was quite broken across different platforms. Unices have /dev/random and /dev/urandom. Even more, these devices are not consistently implemented across them. If you aren’t inspired (/dev/random under Linux is a blocking device), then your random seed generator is screwed till the system gains enough entropy. Under Windows, the used PRNG was something really crappy, if you could use the system implementation, otherwise people used to hack something together. phpass is still a mess, while being backwards compatible with a lot of PHP implementations.

openssl_random_pseudo_bytes() is still slow under Windows, although since PHP 5.3.4 it fixes the blocking behavior. But it’s still better than the previous mess. My method for generating the bcrypt salts is still about 5.8X faster than the method used by phpass under Ubuntu.

I am not saying that my implementation is the implementation, but I got the feeling that for once I am doing it better that I used to do it before, or doing it better than phpass does.

Grab/fork it from GitHub if you’re curious.

Update: I also implemented Ulrich Drepper’s SHA2-based crypt() scheme, but the v0.2 is incompatible with v0.1 due to the fact that it returns the hashes as base64 string. This is a side effect of using 16 bytes of random seed for the SHA2 salts. The PHP manual simply states 16 chars, but that’s a little bit vague. Except chr(0) and chr(36), any other single byte char is valid for the random seed. bcrypt takes a more cleaner approach for the 16 bytes random seed by using a 22 chars base64-like string that encodes those 16 bytes.