portspoof trolling

Marius once told me about portspoof. A service to troll those who use various scanners by feeding the scanners with false results. Well, while the idea is good, I’m wary about a service like this as this is the kind of service where you wouldn’t want a buffer overflow.

Giving it a run inside a VM, I noticed something odd when using nmap’s service and version detection probes. This happened on the lower ports (1-50). Then I started to look at something that started to look like a pattern, therefore I increased the port range to include 1-50. portspoof is indeed a tool that trolls baddies and pen testers.

Ran it with:

nmap -sV --version-all -p 1-50
1/tcp  open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
2/tcp  open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
3/tcp  open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
4/tcp  open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
5/tcp  open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
6/tcp  open  smtp    Unrecognized SMTP service (12345 0ffffffffffffffffffffffffffffffffffffffffffffffffffff00)
7/tcp  open  smtp    Unrecognized SMTP service (12345 0fffffffffffff777778887777777777cffffffffffffffffffff00)
8/tcp  open  smtp    Unrecognized SMTP service (12345 0fffffffffff8000000000000000008888887cfcfffffffffffff00)
9/tcp  open  smtp    Unrecognized SMTP service (12345 0ffffffffff80000088808000000888800000008887ffffffffff00)
10/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffff70000088800888800088888800008800007ffffffff00)
11/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffff000088808880000000000000088800000008fffffff00)
12/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffff80008808880000000880000008880088800008ffffff00)
13/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffff000000888000000000800000080000008800007fffff00)
14/tcp open  smtp    Unrecognized SMTP service (12345 0fffffff8000000000008888000000000080000000000007fffff00)
15/tcp open  smtp    Unrecognized SMTP service (12345 0ffffff70000000008cffffffc0000000080000000000008fffff00)
16/tcp open  smtp    Unrecognized SMTP service (12345 0ffffff8000000008ffffff007f8000000007cf7c80000007ffff00)
17/tcp open  smtp    Unrecognized SMTP service (12345 0fffff7880000780f7cffff7800f8000008fffffff80808807fff00)
18/tcp open  smtp    Unrecognized SMTP service (12345 0fff78000878000077800887fc8f80007fffc7778800000880cff00)
19/tcp open  smtp    Unrecognized SMTP service (12345 0ff70008fc77f7000000f80008f8000007f0000000000000888ff00)
20/tcp open  smtp    Unrecognized SMTP service (12345 0ff0008f00008ffc787f70000000000008f000000087fff8088cf00)
21/tcp open  smtp    Unrecognized SMTP service (12345 0f7000f800770008777000000000000000f80008f7f70088000cf00)
22/tcp open  smtp    Unrecognized SMTP service (12345 0f8008c008fff8000000000000780000007f800087708000800ff00)
23/tcp open  smtp    Unrecognized SMTP service (12345 0f8008707ff07ff8000008088ff800000000f7000000f800808ff00)
24/tcp open  smtp    Unrecognized SMTP service (12345 0f7000f888f8007ff7800000770877800000cf780000ff00807ff00)
25/tcp open  smtp    Unrecognized SMTP service (12345 0ff0808800cf0000ffff70000f877f70000c70008008ff8088fff00)
26/tcp open  smtp    Unrecognized SMTP service (12345 0ff70800008ff800f007fff70880000087f70000007fcf7007fff00)
27/tcp open  smtp    Unrecognized SMTP service (12345 0fff70000007fffcf700008ffc778000078000087ff87f700ffff00)
28/tcp open  smtp    Unrecognized SMTP service (12345 0ffffc000000f80fff700007787cfffc7787fffff0788f708ffff00)
29/tcp open  smtp    Unrecognized SMTP service (12345 0fffff7000008f00fffff78f800008f887ff880770778f708ffff00)
30/tcp open  smtp    Unrecognized SMTP service (12345 0ffffff8000007f0780cffff700000c000870008f07fff707ffff00)
31/tcp open  smtp    Unrecognized SMTP service (12345 0ffffcf7000000cfc00008fffff777f7777f777fffffff707ffff00)
32/tcp open  smtp    Unrecognized SMTP service (12345 0cccccff0000000ff000008c8cffffffffffffffffffff807ffff00)
33/tcp open  smtp    Unrecognized SMTP service (12345 0fffffff70000000ff8000c700087fffffffffffffffcf808ffff00)
34/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffff800000007f708f000000c0888ff78f78f777c008ffff00)
35/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffff800000008fff7000008f0000f808f0870cf7008ffff00)
36/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffffff7088808008fff80008f0008c00770f78ff0008ffff00)
37/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffc8088888008cffffff7887f87ffffff800000ffff00)
38/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffff7088888800008777ccf77fc777800000000ffff00)
39/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffff800888880000000000000000000800800cfff00)
40/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffffff70008878800000000000008878008007fff00)
41/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffffffff700008888800000000088000080007fff00)
42/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffffffffffc800000000000000000088800007fff00)
43/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffffffffffff7800000000000008888000008ffff00)
44/tcp open  smtp    Unrecognized SMTP service (12345 0fffffffffffffffffffffffff7878000000000000000000cffff00)
45/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffffffffffffffffffffffffffc880000000000008ffffff00)
46/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffffffffffffffffffffffffffffff7788888887ffffffff00)
47/tcp open  smtp    Unrecognized SMTP service (12345 0ffffffffffffffffffffffffffffffffffffffffffffffffffff00)
48/tcp open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
49/tcp open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)
50/tcp open  smtp    Unrecognized SMTP service (12345 0000000000000000000000000000000000000000000000000000000)

Really smooth guys, really smooth. Sometimes you have to see the big picture:
big-picture

Converting a file to a JSON array

For some reason I need that. OK, not any reason. For integrating a CloudInit YAML file into an AWS CloudFormation template. By using this article as reference, I made a simple node.js script for doing just that.

#!/usr/bin/env node
 
var fs = require('fs');
 
fs.readFile(process.argv[2], function (err, file) {
	if (err) {
		console.error(err);
		process.exit(1);
	}
	file = file.toString().split('\n');
	var idx, aux = [];
	for (idx = 0; idx < file.length; idx++) {
		aux.push(file[idx]);
		aux.push('\n');
	}
	file = JSON.stringify(aux);
	console.log(file);
});

Save as something.js, make it an executable, then invoke it with ./something.js /path/to/file.

The end.

Converting a DMG to ISO under OS X

There’s a lot of wrong information floating on the internets. People that usually call no-OS X stuff “lesser operating systems” but with no clue about the different internals of a CDR image and an ISO image. CDR has a native OS X filesystem (HFS+), while ISO carries ISO9660. Just rename the CDR to ISO they say. It will be an ISO they say. However, that’s far from the truth.

The correct hdiutil command for converting a DMG to ISO is this one:

hdiutil makehybrid -iso -joliet -o output.iso input.dmg

file should return something like this:

file output.iso
output.iso: ISO 9660 CD-ROM filesystem data 'LABEL'

Performance breakdown for libxml-to-js

Background

libxml-to-js was born to solve a specific problem: to support my early efforts with aws2js. At the time, the options were fairly limited. xml2js was a carry-over from aws-lib which aws2js initially forked. I was never happy with xml2js for a couple of reasons: performance and error reporting. Therefore I looked for a solution to have a drop-in replacement. Borrowed some code from Brian White, made it fit to the xml2js (v1) formal specifications, then pushed it to GitHub. At some point the project had five watchers and five contributors. I guess it hit a sweet spot. That’s why it’s got support for XPath and CDATA, most of it from external contributions. And only then I started using it for other XML related stuff.

The name was chosen to make a distinction from libxmljs which sits at the core of this library which actually binds to Gnome’s libxml2.

Due to the fact that aws2js gained some popularity and I’m doing a complete rewrite with 0.9, the output of libxml-to-js most probably won’t change beyond the “specs” of xml2js v1.

Performance

The actual reason for why I’m writing this article is the fact that people keep asking about the reason for choosing libxml-to-js over xml2js, therefore next time when this question arrives, I am going to simply link this article.

Even now, two and a half years later, with some crappy benchmark that I pushed together, it is somewhere around 25-30% faster than xml2js under usual circumstances. In only specific cases that don’t apply to the XML returned by AWS, xml2js closes in. The part where it really shines is still the error reporting where besides the fact that’s accurate, it is also screaming fast compared to xml2js. In my tests it came out to be around 27X faster.

The code:

var Benchmark = require('benchmark');
 
var suite = new Benchmark.Suite;
 
var parser1 = require('libxml-to-js');
var parser2 = new require('xml2js').Parser({
    mergeAttrs: true,
    explicitRoot: false,
    explicitArray: false
}).parseString;
 
require('fs').readFile(process.argv[2], function(err, res) {
    if (err) {
        console.error(err);
        return;
    }
    var xml = res.toString();
    // add tests
    suite.add('XML#libxml-to-js', function() {
        parser1(xml, function(err, res) {});
    })
        .add('XML#xml2js', function() {
            parser2(xml, function(err, res) {});
        })
    // add listeners
    .on('cycle', function(event) {
        console.log(String(event.target));
    })
        .on('complete', function() {
            console.log('Fastest is ' + this.filter('fastest').pluck('name'));
        })
    // run async
    .run({
        'async': true
    });
 
});

The results, based onto the XML files from the libxml-to-js unit tests and the package.json for the error speed test:

# package.json
XML#libxml-to-js x 18,533 ops/sec ±3.46% (75 runs sampled)
XML#xml2js x 673 ops/sec ±1.35% (68 runs sampled)
Fastest is XML#libxml-to-js
 
# ec2-describeimages.xml
XML#libxml-to-js x 1,122 ops/sec ±4.59% (74 runs sampled)
XML#xml2js x 818 ops/sec ±7.02% (83 runs sampled)
Fastest is XML#libxml-to-js
 
# ec2-describevolumes-large.xml
XML#libxml-to-js x 65.41 ops/sec ±3.13% (65 runs sampled)
XML#xml2js x 50.88 ops/sec ±2.14% (65 runs sampled)
Fastest is XML#libxml-to-js
 
# element-cdata.xml
XML#libxml-to-js x 14,689 ops/sec ±5.41% (72 runs sampled)
XML#xml2js x 11,551 ops/sec ±2.36% (88 runs sampled)
Fastest is XML#libxml-to-js
 
# namespace.xml
XML#libxml-to-js x 9,702 ops/sec ±5.75% (72 runs sampled)
XML#xml2js x 5,802 ops/sec ±2.41% (81 runs sampled)
Fastest is XML#libxml-to-js
 
# root-cdata.xml
XML#libxml-to-js x 22,983 ops/sec ±7.11% (69 runs sampled)
XML#xml2js x 14,849 ops/sec ±6.01% (87 runs sampled)
Fastest is XML#libxml-to-js
 
# text.xml
XML#libxml-to-js x 2,669 ops/sec ±3.68% (78 runs sampled)
XML#xml2js x 2,617 ops/sec ±2.41% (88 runs sampled)
Fastest is XML#libxml-to-js
 
# wordpress-rss2.xml
XML#libxml-to-js x 2,056 ops/sec ±4.08% (75 runs sampled)
XML#xml2js x 1,226 ops/sec ±2.79% (84 runs sampled)
Fastest is XML#libxml-to-js

The tests ran under node.js v0.10.22 / OS X 10.9 / Intel Core i5-4250U CPU @ 1.30GHz with the latest module versions for both libxml-to-js and xml2js.

Fixing the AMD AHCI drivers for SB7xx on Windows 7

I heard a lot of urban legends about the Windows Update service that messes up your machine. Of course, I dismissed all of them with the classic “worksforme” as didn’t happen to me. Until Microsoft delivered a 3rd party driver update via an optional package. You know, like the stuff that comes from the vendor and it isn’t properly tested. I had the lack of inspiration to check that too instead of simply ignoring it, like I usually do with Bing Desktop and Silverlight. The next thing was a BSOD at boot.

Had to disable the AHCI in BIOS and revert to using IDE mode for the SATA ports. Which kinda sucks for some reasons. The most important: the SSD performance is hurt under IDE mode, the TRIM command won’t work under IDE mode without 3rd party software since only the MSAHCI driver implements TRIM from Windows 7, and the fact that my HDD array doesn’t support NCQ under IDE mode.

When it comes to drivers, AMD is still a shitty company. Even more, their engineers didn’t grasp the concept of backward compatibility. Uninstalling the driver that broke my installation and installing a driver that works proved to be a non-trivial task. Fortunately I found this post on pchelpforum.com.

For the sake of avoiding the link rot, I’m going to reproduce the essentials for posterity, with the same disclaimer as the original – you’re on your own if you mess up your machine and I’m not taking any responsibility if you follow these:

  • Delete any older version of the amd_ahci driver from here: C:\Windows\System32\DriverStore\FileRepository. The folders with older AMD AHCI drivers are named something like: amd_sata.inf_amd64_neutral_c85cc6046149a413 (i386 on 32-bit and most probably another hash). In order to remove the directory, you need to either elevate your explorer / shell to SYSTEM privileges, or take the ownership of the driver directory, add proper permissions, then delete it.
  • From HKLM/SYSTEM/CurrentControlSet/services delete amd_sata and amd_xsata. There’s no need to remove the entries without the underscore (amdsata and amdxsata).
  • Reboot the computer. Don’t change from IDE to AHCI. The driver that actually worked for my combination, which is AMD 780G / SB700 is this one. Execute the installer, wait till it finishes to copy the files to C:\ATI\Support, then cancel the setup when the Catalyst installer starts.
  • Open the Device Manager. Action » Add legacy hardware » Advanced mode » Show All Devices » Have Disk. Browse the extraction path for the above package: C:\ATI\Support\11-12_vista32-64_ahci\Packages\Drivers\SBDrv\SB7xx\AHCI. There’s a couple of directories: LH – for 32-bit and LH64A – for 64-bit. Select “AMD SATA Controller” then continue. Unlike the author of the original material, I didn’t get an error about the device not starting.
  • Reboot the computer. Don’t change from IDE to AHCI. Go to Device Manager. Under IDE ATA/ATAPI controllers should be at least an entry with a yellow exclamation mark, AMD SATA Controller. Uninstall “AMD SATA Controller” without checking “Delete the driver software for this device”. Reboot the machine.
  • Go to BIOS, enable AHCI. After boot, the OS installs the proper drivers, then prompts for another reboot. Reboot the machine. Done.

In my case, it simply fixed the driver installation from the failed Windows update as the driver that runs on my machine is from 2013 and the driver used in the above steps is from 2011. The drivers from the latest Catalyst, 13.4 failed to install via the “Add legacy hardware” method or via a standard Catalyst setup.

amd-sata-controller

Some benchmarks with a SSD drive under IDE mode:

benchmark1-idebenchmark2-ide

And some benchmarks under AHCI mode:

benchmark1-ahcibenchmark2-ahci

I guess the sharp drop was due to TRIM doing its job. Yes, it’s enabled:

trim