Brute forcing device passwords

When working with IoT and embedded systems, brute-force password guessing attacks are an effective tool to gain access. Over the years, I’ve learned some tips and tricks to make these attacks more effective.

What is brute forcing?

Very simply, it’s guessing passwords so that you can find a valid one and login to the device.

It’s often referred to as “password cracking”.

Online vs offline brute forcing

There are two forms of brute-force attack.

One is online. This means you are actively trying to login to the device using the web interface, telnet, SSH, or local console. This has disadvantages. It’s generally quite slow (less than 10 attempts per second, sometimes much slower) and account lockout is a challenge.

The other is offline. This is when you have the hash of a password and can guess passwords on an entirely separate machine. This has advantages. For many types of hash, hundreds of thousands of guesses can be performed each second. But, you need to have the hash in the first place.

(you can read a bit about hashes in an earlier post here)

If I have the device in front of me, and I can risk destroying it, I will generally obtain the hash and perform an offline attack. This isn’t always appropriate though – sometimes devices are too expensive or you cannot risk breaking it.

Sometimes the device has strong physical protections that make obtaining hashes difficult.

Often, an online attack can be setup and carried out in seconds – all you need is network access and a list of passwords. It’s often wise to try this first.

Do I need the password?

If your goal is to gain access to a device in front of you, make sure you have considered other paths.

Devices frequently have vulnerabilities allowing bypass of passwords. Command injection via web interfaces is still very common.

If you can modify the firmware, consider running an unauthenticated telnet shell. Or replace the password hash with one of your own, avoiding the existing password entirely.

What is the goal of getting the password?

With physical access to a device, time, and some skill, it is nearly always possible to login to a device without knowing any passwords.

So why do we want to find out the password?

Firstly, it’s very common to find hardcoded or default passwords on embedded systems. If you can find the password from one device, you can use it on many others. I call these BORE attacks – Break Once, Run Everywhere. The password has much higher value than just the device in front of you.

Secondly, you can perform a high-risk attack against a device on the bench, obtain a password, and then use that password in the real-world. This was what we did on an oil rig, where an attack was developed against a Siemens switch in the lab, and then the attack used on the rig itself.

Thirdly, it’s often possible to quickly and easily obtain a password hash from one device using something like the Cisco Password Recovery mode. You can quickly reset a Cisco switch on an non-critical, physically exposed switch, brute force the password, and then find that same password is in use on the core network.

How much effort are you willing to expend?

Obviously, the longer you spend guessing passwords, the more passwords you can guess.

Online brute force attacks are extremely rate-limited. The workload can be divided up by attacking multiple devices, but if you can obtain multiple devices, why not get the hashes from one of them and move to an offline brute force attack?

Offline brute force attacks cost in processor time. The more computing power and time spent, the more attempts you can make. You can literally throw processor time (and hence money) at the problem!

Depending on the value of the password, you may be willing to spend different amounts. A hardcoded root password for an IoT lightbulb behind NAT has a lot less value than the root password on a maritime satellite router with Internet exposed SSH!

Online attacks

There are three common network services which allow online brute force attacks to be carried out: Web interfaces, telnet and SSH.

A tool called hydra comes in very useful here. It handles these protocols and many more.

SSH is the easiest to deal with. Nearly all devices have a standard response, and the protocol feeds back if you have authenticated successfully or not.

Telnet is a bit awkward. The password prompt, and the feedback you get if you successfully login varies from device to device. Often you need to write scripts to do this properly. Always make sure you catch the success message properly – I once left a script running against a device for an entire weekend and it had found the password in less than 10 minutes.

Web interfaces can be easy sometimes, and incredibly challenging at others. The easiest are ones using HTTP Basic or Digest authentication – this is when you get the pop-up password Window in your browser. This can be easily attacked. Web forms take more effort – generally you need to intercept traffic and work out how the password is being sent. Some devices use JavaScript and other complex authentication flows that take significant effort to reverse engineer and replicate. An intercepting proxy such as Burp or Zap can be of use here.

If the device has account lockout, preventing you trying more than a few passwords, it is always worth checking that this persists through a device reset. It is very common for devices like DVRs to lockout for 15 minutes after 5 wrong guesses, but if you restart the device, the lockout has gone. A USB or WiFi-connected socket can be scripted to restart the device automatically.

Just be warned: a lot of ICS equipment responds badly to brute force attacks. Lockups and restarts are common, and I’ve even caused the relay output on one device to change state.

Cracking rigs

Calculating password hashes requires processor power. The first common password cracking tool, John the Ripper, made use of the main CPU in a machine.

It turns out that graphics cards are far more efficient at calculating most types of hash. Another password cracking tool, hashcat, became available, making use of multiple graphics cards. This hugely accelerated the rate at which passwords could be guessed, but required significant investment in hardware, and ongoing electricity and cooling costs.

Then cloud computing happened. It’s now possible to rent machines with graphics cards in them and perform password cracking in the cloud. You can spin up as many machines as required and use tools to split the workload amongst them. This can be very helpful when you do not know how often you need to perform password brute force attacks.

Picking your passwords

There are several common methods to generate lists of passwords to attack devices.

A dictionary attack takes a pre-generated list of words and tries them all. There are many readily available password dictionaries of varying quality, some contain millions of passwords. These dictionaries can be very effective against large sets of hashes, such as those from a website breach or Windows Active Directory.

Building your own dictionary based on the vendor and product can help. The tool CeWL can spider a website and return a list of words for use in password attacks.

A brute force or incremental attack tries all possible combinations. With these attacks, the character set used and the length of the password become important. The more characters tried and the longer the password, the larger the search space becomes and the longer an exhaustive search will take.

Most passwords consist of upper and lower case letters, numbers, and possibly symbols. The more of these that are used, the larger the search space.

  • 26 lower case
  • 26 upper case
  • 10 numbers
  • 10 common symbols
  • 20 relatively common symbols

So, if the password contains lower/upper/number/common symbols, there are 72 possible characters.

If the password is 8 characters long, this results in 72^8 possible passwords. This is 7.22e14 – a very large number.

It’s at this point that we need to consider how quickly we can guess passwords. This is highly dependent on the hash algorithm used. For a typical high-end hashcat rig, the following are approximate rates.

AlgorithmGuesses per second

There is a huge variation here. This can dictate what kind of attack is viable. With our 72^8 attack above, the length of time for an exhaustive search for the above algorithms is as follows:

MD51 hour
descrypt1 day
md5crypt105 days
bcrypt218 years

On average, the password will be found in half that time. 30 minutes vs over 100 years! It’s clear that brute-force is not a viable attack for bcrypt. For md5crypt, as long as you know the password is in that space, 105 days may be a reasonable expense depending on the value of that password.

This is also why it is key to understand the password complexity. descrypt only hashes the first 8 characters of the password. Combined with the speed at which we can guess, it makes exhaustive searches possible.

Let’s see what changing the character set does to the times if md5crypt is used.

Character setOptionsTime
lower/upper527 days
lower/upper/number6232 days
lower/upper/number/common symbol72105 days
the kitchen sink922 years

Again, there is huge variation. 7 days is likely viable, but it is a big risk assuming that no number of symbols were used. Equally, 2 years is a long time to spend to find a password and there is still a chance they used some quirky symbol and you never find the password.

Always keep in mind the language of the developers. It’s rare to find umlauts in US and UK passwords, but don’t assume that for a product developed in Germany.

And what about the password length changing? Again, with md5crypt and 72 characters.

Password lengthTime
8105 days
101500 years
128 million years

It becomes obvious that if the password is longer, plain brute force attacks quickly become useless.

Hashcat also has other modes of operation that are more refined.

Mask attack mode takes a pattern and then applies a brute force attack within those limits. It’s very common to find that certain vendors, companies and users follow certain patterns, such as using iwhd7262, baid8621. Mask attack mode can be very helpful here.

This is also a good mode for things like passwords based on serial numbers and MAC addresses.

One of the most powerful modes is the rules mode. This takes a series of different rules and applies them to a list of passwords. This can be things like changing the case, appending numbers, swapping letter for numbers (i->1 a->4). There are complex sets of rules published, all of which can be very effective.

Open Source Intelligence

It is always worthwhile researching the vendor and their other products. The one you are looking at may not have a published password, but other products could, and they may follow a pattern.

In one instance, Samsung had a hardcoded root password across many of their cameras. Nearly all of their cameras used bcrypt, making an exhaustive search unlikely to succeed as it is so slow. However, one model of camera was using descrypt. An 8 character exhaustive search was carried out, yielding a complex (but short) password. This was found to be the same password on the device using bcrypt. With bcrypt, we would never have found the password.

Find it in other forms

Never assume that the password is only stored as a single hash on the device. It’s very common to find it hashed using different algorithms or stored in plaintext.

One device we looked at used Webmin for the web interface. This stored the passwords as descrypt, so an exhaustive 8 character search was started. This quickly gave up the passwords for all accounts.

The main Linux login used bcrypt though. Exactly the same accounts existed in Webmin as for the OS. Taking a stab in the dark, we assumed that the passwords were the same for both hashes, just truncated for the descrypt one. We knew the format and character set, so launched a hybrid attack using the first part which we already knew and brute force against the bit that we didn’t. We would never have found the passwords without the first 8 characters being known.

Passwords often get stored in plaintext. Common locations for this are WiFi hotspot configurations in hostapd.conf (the WiFi password is often the same as the OS login) and setup/factory reset scripts.


Hopefully the above tips and tricks have helped you understand password brute force attacks so that you can effectively use them against devices.

Why is unauthenticated encryption insecure?

Cryptography is a complex subject. There are many subtle issues that can be introduced if you don’t know what you are doing.

There is a common mantra: “don’t roll your own crypto”. This is because both inexperienced and experienced developers frequently build cryptographic systems that are insecure.

However, there has to be a line – when does it start becoming “rolling your own”? Particularly in embedded systems, there are times when custom protocols need to be used, and developers stray into the dangerous area of cryptography.

One of the most common mistakes we have seen is the use of unauthenticated encryption.

What is encryption?

Encryption is encoding a plaintext into a ciphertext using a key, with the goal of keeping the plaintext confidential.

Only someone with the correct key should be able to decrypt the ciphertext and turn it back into plaintext.

Encryption provides confidentiality. It stops someone working out what the message is.

So what’s the issue?

An attacker can modify the ciphertext and cause the plaintext to change. There is no inherent means in encryption to detect this change.

Encryption does not provide authenticity. You cannot check that the message is genuine and has not been tampered with.

What can an attacker do with this?

I’m going to describe one attack against unauthenticated encryption.

Many encryption algorithms only operate on fixed-size blocks of data – they are called block ciphers. To encrypt longer lengths of data, a mode of operation is used to apply the block cipher repeatedly.

One mode of operation is called CBC (Cipher Block Chaining). When encrypting the data, the previous ciphertext block is mixed into the current plaintext block using an operation called “exclusive OR“. This is denoted with the + in a circle in diagrams.

There is also an input called the initialisation vector, or IV. This is a random input to the algorithm, and is intended to ensure that the ciphertext is different, even if the same plaintext is encrypted. This prevents leaking information about the content.

The initialisation vector is transmitted alongside the ciphertext.

Decryption is similar. The previous ciphertext block is exclusive ORed with the output of the block cipher to obtain the plaintext.

Exclusive OR is a deterministic operation. If we look at a single bit, then it operates as follows:


I always think of this as “if one input is high, invert the other input, otherwise leave it alone”.

The operation is carried out for each bit in a byte.

A: 0 1 0 1 1 0 0 1 (0x59)
B: 1 1 1 1 0 0 0 0 (0xF0)
O: 1 0 1 0 1 0 0 1 (0xA9)

What this means is that modifying one of the inputs to exclusive OR results in a predictable change to the output. And the operation can be easily reversed.

A: 0123456789ABCDEF
O: FEDC459879A4C21F

If we now exclusive OR the output with one of the inputs:

A: FEDC459879A4C21F
O: 0123456789ABCDEF

Hopefully that explains exclusive OR.

Let’s look back to how CBC uses this in decryption. In the first block, the IV is exclusive ORed with the output of the block cipher. The IV is transmitted alongside the ciphertext and an attacker can modify both at at will.

We can encrypt the string “A dog’s breakfast” using a key and the initialisation vector of all 0x00 (here on CyberChef).

Key: 0123456789ABCDEF0123456789ABCDEF
IV:  0000000000000000000000000000000
Plaintext: A dog's breakfast
Ciphertext: c7b1d96f0f520f33faaccfdc107f718aafe8892c3a29c76b0732a760a0f54f50

Of course, this can be decrypted (here on CyberChef).

If I change just one byte in the ciphertext, the entire message is corrupted (here on Cyberchef). There’s no way for me to predictably modify this plaintext by changing the ciphertext.

Key: 0123456789ABCDEF0123456789ABCDEF
IV:  0000000000000000000000000000000
Ciphertext: c7b2d96f0f520f33faaccfdc107f718aafe8892c3a29c76b0732a760a0f54f50
Plaintext: .L...Q½êU...ì7Ò.t

But the attacker also has control over the IV. Let’s set the first byte of the IV to 0xFF (here on CyberChef). Only the first byte of the plaintext has changed!

Key: 0123456789ABCDEF0123456789ABCDEF
IV:  FF00000000000000000000000000000
Ciphertext: c7b1d96f0f520f33faaccfdc107f718aafe8892c3a29c76b0732a760a0f54f50
Plaintext: ¾ dog's breakfast

And it has changed predictably. The capital A (ASCII 0x41) has been exclusive ORed with 0xFF to become 0xBE (which decodes as ¾ although it’s above the normal ASCII range).

A: 0 1 0 0 0 0 0 1 (0x41)
B: 1 1 1 1 1 1 1 1 (0xFF)
O: 1 0 1 1 1 1 1 0 (0xBE)

This is a very high level of control! The attacker can now modify the plaintext without detection. Let’s try and significantly change the meaning of it.

The original message contained “A dog’s breakfast”. Can we change this canine feast into a feline one?

We exclusive OR the original plaintext with the desired one (here on CyberChef). Notice how the output only has value for the characters we have changed.

Original: A. .d.o.g.'.s. .b.r.e.a.k.f.a.s.t.
Original: 4120646f67277320627265616b66617374
Desired:  A. .c.a.t.'.s. .b.r.e.a.k.f.a.s.t.
Desired:  4120636174277320627265616b66617374
Output:   0000070e13000000000000000000000000

Pop that output in as the IV to the decryption, and we’ve successfully changed the message (here on CyberChef). All of this without even knowing the key.

Key: 0123456789ABCDEF0123456789ABCDEF
IV:  0000070e130000000000000000000000
Ciphertext: c7b1d96f0f520f33faaccfdc107f718aafe8892c3a29c76b0732a760a0f54f50
Plaintext: A cat's breakfast

Of course, the attacker needs to have knowledge of the plaintext to make use of this attack. However, it’s extremely common for some or all of the message to be known. For example, when we visit most websites, the first part of the response will be “HTTP/1.1 200 OK”. If this was only protected by CBC encryption, we could change that to “HTTP/1.1 404 No”, changing the behaviour of the browser (here on CyberChef).

This doesn’t just impact the first block of data either. After the first block, instead of the IV, the previous ciphertext block is used in the exclusive OR operation. The attacker can modify the ciphertext and end up controlling the plaintext.

This comes at a cost though – the previous plaintext block will be totally corrupted as a result.

To illustrate this, we can encrypt a longer block of text (here on CyberChef).

Let’s change “baud” to “cats”. We need to locate the correct place in the ciphertext. AES (the encryption algorithm we are using) works in 16 byte blocks. The word “baud” is 85 characters in, so in the 6th block. We therefore want to modify the 5th block of ciphertext.

The exclusive OR is a bit more complex than last time – we now need to exclusive OR the ciphertext, the original text, and the desired text (here on CyberChef). But change those 4 bytes, and we change the word “baud” to “cats”.

The only issue is, as expected, the previous block has been entirely corrupted. Whilst in this case, it’s made part of the message nonsensical, it frequently has no impact when carrying out attacks.

But there are worse problems?

The above issue allows an attacker to modify the plaintext without detection. This would be an issue in certain situations, such as lock/unlock messages to a door.

But not authenticating your encryption can lead to worse issues. A type of attack called padding oracle attacks can let an attacker obtain the plaintext by sending a large number of specially crafted packets.

Block ciphers only operated on fixed blocks. If the data is shorter than a block, it must be padded. There are a number of ways of doing this, such as appending the number of padding bytes (e.g. 0x02 0x02 or 0x05 0x05 0x05 0x05 0x05). The process of decryption may check this padding is correct or not, and respond differently in each case.

An attacker can exploit these differential responses to leak the plaintext. This can break the confidentiality of messages.

What’s the solution to this?

Encryption should always be authenticated. There are two common solutions to this:

  • Add a Message Authentication Code (MAC). This is a keyed cryptographic checksum that provides authenticity and integrity.
  • Use an authenticated mode of operation such as GCM.

Even with this advice, there are many pitfalls. Applying the authentication and encryption in the wrong order can lead to weaknesses; this is so common that it has been deemed the Cryptographic Doom Principle.

Generally, developers shouldn’t be working with cryptography at this level unless they are suitably skilled. That’s easy to say, harder to put into action. There is a big movement to make use of secure-by-default cryptographic libraries and APIs that provide developers with useful functions without giving them so much rope they can hang themselves.

There are scant few reasons for not authenticating encryption.

Checksums, hashes, and security

If you have worked with embedded systems, you will have come across terms like checksums, hashing, password hashes, and CRCs. Whilst checksums and hashes are two distinct terms, they are often used for similar purposes. They are also often used incorrectly.

What is a checksum?

A checksum is a short piece of data derived from a longer block of data, generally for the purpose of detecting errors that have been introduced. If the checksum does not match the data, then something has gone wrong.

They can be used to ensure data integrity i.e. has this data been accidentally modified? Good checksums mean that it is highly unlikely that the data could be changed and the checksum could also be accidentally changed to be valid.

They cannot be used to ensure data authenticity i.e. has someone maliciously altered this data? If an attacker can modify the data, they can recalculate the checksum so that changes cannot been detected.

Message:  The quick brown fox jumped over the lazy dog.
Checksum: 82a34642

Message:  The quick brown cat jumped over the lazy rat.
Checksum: 0319c205

There are many common forms of checksum.

Check digits are added to numbers all around us. Credit card numbers, IMEI numbers for phones, VIN numbers on cars. These are generally designed to detect human errors, such as off-by-one (3->4) and transposition (54->45).

Parity bits are used in communications. The number of 1s in the data are counted. If it’s an even number of 1s, the parity bit is 0. If it’s an odd number of 1s, the parity bit is 1. The receiver simply needs to sum all the bits including parity together. If it’s not an even number of 1s, then an error has been introduced. Clearly this has issues – if two bits are flipped, the error may not be noticed!

Modulo checksums are used frequently in custom radio frequency protocols and wire protocols. You sum up the value of each byte sent, then perform a modulo 256 operation i.e. what’s the remainder when divided by 256.

If we want to send the following data:

45 67 AB 43 23 98 FD E1

We sum each byte to get 0x433 – too big to fit in a single byte! So we do “modulo 0x100”, leaving us with a remainder of 0x33. Generally, we take what is called the “two’s complement” (0x100 – 0x33 -> 0xCD) of that and append it. Why? Because now if we add all the bytes together and do modulo 0x100, we should get 0 if all is well.

Cyclic Redundancy Checks (CRCs) are more complex but found everywhere around us. There are hundreds of common variants, with different lengths – CRC-8, CRC-16, CRC-32 – and different “polynomials” – a set of coefficients used. Although they have all these variations and can be more challenging to reverse engineer, this does not mean they can prove data is authentic.

Keeping a Checksum the same

There are situations where an attacker cannot modify both the data and the checksum. For example, if the checksum of a firmware image is displayed on the manufacturer’s website, and the attacker wants to modify the firmware.

The issue with the checksums mentioned above is that it is nearly always possible to modify the data and keep the checksum the same.

With the modulo checksum, this is trivial. If one byte is increased by 1, another byte must be reduced by 1 to maintain the same checksum. This can be really easily shown with ASCII text. Changing a letter’s case adds or subtracts 0x20 from the value. A is 0x41, a is 0x61. 8 times 0x20 is 0x100, so change the case of 8 characters, and the modulo checksum doesn’t change!

Message:  I really like bananas very much.
Checksum: 74

Message:  I really LIKE bananas VERY much.
Checksum: 74

It’s not as easy with CRCs, but by modifying any 4 consecutive bytes in a file, the checksum can be kept at the same value. This does mean additional bytes need to be changed, but these can often be squeezed into padding or irrelevant text.

Message: I really like bananas. This bit is not important.
CRC-32:  8D00795D

Message: I really HATE bananas. This biN�`Z not important.
CRC-32:  8D00795D

What is a hash?

A hash function is a deterministic function that maps an arbitrary sized piece of data to a fixed-sized piece of data.

Technically, the checksums mentioned above are actually hash functions – you can provide any length of input to parity or a modulo checksum, and get a fixed length output.

However, in security, we are generally talking about something called a cryptographic hash function.

You will probably be familiar with some of the common ones: MD5 and SHA are the most well known.

First, I’d like to talk about one of the inherent qualities of a hash function called the pigeonhole principle. If you have 9 pigeonholes, and 10 pigeons, then at least one of the pigeonholes must contain more than 1 pigeon.

Because a hash function maps something of arbitrary length into something of fixed length, there will always be multiple inputs that result in the same output. There are infinite numbers of arbitrary length inputs, and only a finite number of outputs.

As a result, collisions exist – two different inputs that result in the same output.

An ideal cryptographic hash function should hold certain properties.

It must be deterministic – if you provide it with the same input twice, you should get the same output. This one seems fairly obvious – if the purpose of the hash function is to detect errors or similar, it must always behave the same.



It should be quick to calculate the hash. Remember how it works for arbitrary data size? You should be able to quickly hash a short sentence or the entire contents of Wikipedia.

It should be infeasible to determine a message that yields a given hash value. This is given the fancy name of “preimage resistance“.


It should be infeasible to determine two messages that yield the same hash value. This is given the fancy name of “collision resistance“. We know that collisions will exist, but they should not be easy to find.

A small change in the input should result in a large change in the output. This is called the “avalanche effect“. This makes sense later when we consider some of the use cases.




So what do you use these cryptographic hash functions for?

Error Detection

Similar to checksums, hashes can be used to detect errors or changes in data.

Exactly as with checksums, if an attacker can modify both the data and hash, there is no way to determine if the data has been changed. Hashes are not safe against an active attacker, only accidental modification.

Most hashes have an output that is longer than a checksum. The longest commonly used checksum is CRC-32, which has a 32 bit or 4 byte output. The shortest commonly used hashes are 128 bits or 16 bytes. There are almost 8e28 more potential values for a 128 bit value compared to 32 bits.

Checksums are therefore far more prone to collisions than hashes. It is perfectly possible to find two pieces of data with the same CRC-32, but incredibly difficult to find two pieces of data with the same hash.

Due to the avalanche effect, even a single bit changing in the input should result in a completely different hash. This is different to most checksums, where changes to the input result in predictable changes to the checksum.

As such, hashes tend to be more resistant to certain attacks.

File and Data Identification

A cryptographic hash of a file or data can be used to uniquely identify it. This is often used with software or firmware update files to ensure that the file is as expected. They are also used in computer forensics to ensure that data has not changed.

Digital Signatures

To ensure that data is authentic, we make use of digital signatures. The identity of websites you visit using HTTPS is checked because it is digitally signed by a trusted certificate authority. The software installed on your PC can be signed to ensure it is genuine.

A technical limitation of digital signatures is that they can only operate on short blocks of data. It’s not possible to directly sign an entire software installer.

To sign large amounts of data, the data is first hashed. The hash is then signed instead of the data itself.

If there is a weakness in the underlying hash function, the signature cannot be trusted.

Password hashing

When you log in to most systems, you send the server your password. The server then checks that your password matches the one you have previously provided.

That sounds simple. But user databases are frequently leaked or breached. If the passwords are in the database, they can be read and then used. Because of this, passwords are not stored in their raw form. We hash the passwords instead.

Because it is difficult to determine the password that results in a given hash, or find another password that results in the same hash, the attacker cannot trivially work out what the password is.

There is still a weakness here – if you have the hash, you are free to guess the password. You can guess as many passwords as you like. This is called a brute-force attack.

An ideal cryptographic hash function is fast. This means that calculating the hash for a password would be fast. This allows many guesses to be attempted in a short period of time.

An ideal cryptographic hash function is also deterministic. This means that the same password will result in the same hash. It would be possible to precalculate the hashes for many passwords, and then simply lookup the password that corresponds to a hash.

Because of these issues, specific password hash functions exist. Common ones are bcrypt and PBKDF2.

Firstly, they increase the time taken to perform a hash. They do this by using a normal hash function many times. This makes brute-force attacks slower. The number of times this is done is configurable, allowing a balance between the time taken and resistance to brute-force.

Secondly, a random value called a salt is also input into the hash function alongside the password. This is stored alongside the hash. When the password needs to be checked, the salt is read and used with the password to calculate the hash. This means that the same password will not result in the same hash, making the precalculation of password hashes useless.

Despite these dedicated password hash functions existing, it is still common to find conventional hash functions used. It is around 2 million times faster to calculate an MD5 hash compared to bcrypt. This can be the difference between an attacker working out the password or not.

Password hashing cannot be used when the password needs to be sent to another system. For example, if an IoT lightbulb connects to your WiFi, the WiFi password cannot be stored as a hash inside the device. It must be in a recoverable form to be used to connect.

Common Mistakes

We see a lot of common mistakes made with checksums and hashes. Here are some of them.

Hashes used for Authenticity

As mentioned several times above, if an attacker can modify both the data and hash, it is not possible to detect if the data has been modified. They are not useful for providing data authenticity. This is what digital signatures are for.

It is incredibly common for a hash to be prepended to a firmware file. This does not prevent someone modifying the firmware maliciously.

We’ve even seen a product call a MD5 hash a signature.

Encrypted Checksums used for Authenticity

Data is encrypted to provide confidentiality, not authenticity. It stops someone determining the content of the message. It does not stop them modifying the message.

With most cryptographic algorithms, if you change the encrypted data, it results in random decrypted data.

If we encrypt a simple phrase:

I like cats

We can then decrypt it:

I like cats

But change just one bit in the encrypted data, and we get a totally unrelated output.


This doesn’t seem useful to an attacker. But if a CRC-32 is contained inside that encrypted data, you can just provide random encrypted data until you randomly get the right CRC value in the decrypted data. This would, on average, take just over 2 billion attempts – well within the realms of possibility for an automated attack.

The root cause of this is assuming that encryption proves that data is authentic.

Weak Password Hash Functions

Despite the existence of dedicated password hash functions, it is still common to find conventional or weak hash functions used.

MD5 is millions of times faster to calculate than bcrypt. For every single password we can guess with bcrypt, we can try millions with MD5. This is a huge difference.

One of the password hash functions that can be used by Linux systems is a DES-based one, commonly known as descrypt. Whilst this was suitably slow to calculate when it was designed forty years ago, increases in processing power mean that it is not longer considered secure.

Worse still, descrypt only considers the first 8 characters of the password entered. “password” and “password&767%^%46” are handled exactly the same.

The combination of it being fast to brute-force and only supporting 8 character passwords mean that it is viable to exhaustively search all passwords in a reasonable time frame.

Only dedicated and modern password hash functions should be used.

Use of Broken Hash Functions

Due to their heavy use in digital signatures, significant effort is spent analysing the security of hash functions. Weaknesses are found in them, and then they are considered “broken”.

MD5 is very commonly used, but it is possible to find a collision in a few seconds on a laptop. A simple collision like this – two arbitrary inputs that produce the same hash – is not of much use to an attacker. But with some more effort, it is possible to find two related inputs with the same MD5 hash. This has been used in attacks the past – a certificate authority believed they were signing a conventional website certificate, but in reality, they were signing another certificate authority certificate that had the same hash value. More can be read about this here, but it is not particularly easy to follow.

A viable pre-image attack has not been found for MD5. This would allow an attacker to simply generate another message given the value of a hash. This is a much more serious attack.

Regardless, MD5 should be considered broken and no longer used. More to the point, there are more secure alternatives that can be used. No new systems should use insecure hash functions.

This site has a useful table showing how various hash functions have been broken after time. Once a collision is found, it should be considered game over. Cryptographic attacks can only improve over time, not get worse.


Hopefully you’ve learned about some of the uses of checksums and hashes, and the pitfalls surrounding them.

Understanding Binary and Data Representation with CyberChef

A significant part of reverse engineering and attacking devices relies on viewing and recognising data in various forms and working out how to decode it.

We typically use Linux tools and scripts to do this, but you can make the first few steps using a really neat online tool called CyberChef.

What is binary?

All data is stored as a series of 1s and 0s. A single 1 or 0 is called a bit. We call this binary because there are two values.

The next largest common unit is a byte, which is 8 bits.

Beyond this, SI prefixes are used. 1 kilobyte (kB) is technically 1000 bytes and 1 kibibyte (KiB) is 1024 bytes. However, kB is frequently used for both 1000 bytes and 1024 bytes, even in technical contexts. During most reverse engineering, kB means 1024 bytes.

kB (kilobyte)1024 bytes
MB (Megabyte)1048576 bytes or 1024kB
GB (Gigabyte)1073741824 or 1024MB

To determine how many possible values can be stored in a data of a given length, you do the following calculation:

Values = 2^bits

^ means “to the power of”

For example, a single byte (8 bits) can store 2^8 or 256 values. 2 bytes (16 bits) can store 2^16 or 65536 values. Increasing the bit length by 1 bit will double the number of possible values.

You can see that by the time you have reached 64 bits, there are a huge number of possible values.


The number of potential values can be important when calculating the search space for performing brute-force attacks.

Although there are 256 values in a byte, the values normally start at 0. Therefore, the range is 0-255, covering all 256 values.

Binary data can encode information in many different forms. The following are all representations of the same data

  • 01000001 (binary)
  • 65 (decimal)
  • 41 (hexadecimal)
  • A (ASCII or text)
  • QQ== (base 64, a means of encoding binary as text)

What is hexadecimal? Well, instead of each digit representing 10 values (0-9 as in decimal), each digit represent 16 values (0-15). Clearly we can’t put 15 into one digit, so we use letters above 9.


As a single hex digit represents 16 values, this is only 4 bits (2^4 = 16). To represent a byte, we need to use 2 hex digits such as D4 or 8E.

We will frequently use the prefix of 0x to represent hex i.e. 0x41. Context is everything though – never assume how data is encoded! It can be text, part of a floating point number, or code.

You can use the built-in calculator in Windows and OS X to convert from one to the other if you switch to programmer view:

Onto CyberChef

A useful tool for many of these understanding data is called CyberChef. This is an online tool that runs entirely in the browser. None of the data entered leaves your machine, and it can be saved and run locally.

Yes, it’s GCHQ. No, they aren’t stealing your secrets. At least not using this tool.

Multiple operations can be chained together to form a pipeline. This includes simple conversions, but also complex things such as encryption and decryption.

Let’s start my putting some text into the “Input” section. This will be copied verbatim to the “Output” section as no “Recipe” has been created.

On the list of “Operations” on the left hand side, drag “To Hex” into the “Recipe”. The output will now show a hexadecimal representation of the text.

You can search the operations using the box on the top left rather than hunt through all the subsections.

The text is encoded using a method called ASCII. Each character is represented by a single byte. In reality, only 7 of the 8 bits in the byte are used, giving 128 (2^7) possible characters.

Converting to and from ASCII is a very common task. Tables showing all the values are available online.

Ranges and values worth becoming familiar with are:

  • 0x20 – space
  • 0x30-0x39 – 0-9
  • 0x41-0x5A – A-Z
  • 0x61-0x7A – a-z

It’s very common for the value of 0x41 – capital A – to be used when performing tests for buffer overflows. You’ll start to recognise long strings of 41414141 when looking at memory!

ASCII is not the only way of encoding text. Unicode is a common format that allows many more possible characters, but there are tens of different encodings. You can use the “Text encoding” operation to see these. UTF-16LE encodes each letter as 2 bytes (the 16 means 16 bits). Now each second character looks like a “.”.

If we now add the “To hex” operation, we can see that every second byte is 0x00 – a null. Those “.” just mean “I’m not sure how to display this”.

We can flip this round and firstly take hex, convert it to binary using “From hex”, and then decode the text using “Text decode”.

You can click this link to see this directly.

If we use the correct encoding, (UTF-16LE), then the text looks as expected. Change it to UTF-16BE though, and suddenly we have nonsense.

This is a vital point – binary data can be interpreted in many different ways. Our operating systems and programs use file extensions and metadata to determine how to handle the content, but as reverse engineers we often need to guess.

So far we have just looked at text. However, executables on our machines are also just binary data. We can load these into CyberChef and analyse them.

I have chosen to look at write.exe from C:\Windows\system32\ – it’s small enough that CyberChef can handle it but provide some interest.

You’ll immediately see some recognisable strings. Nearly all executables will have these in some form. They can be incredibly useful in reverse engineering software, allowing us to determine function, endpoints, passwords and more.

Another operation called “Strings” will filter out lengths of text longer than a certain limit. As you can see, it matches some data that is not text – this is just where binary data happens to decode as ASCII correctly.

We know that this is a Windows executable as we just read it from our own system. The .exe on the end is just part of the filename – it would still be the same binary data if it was called cheese.txt.

But frequently, we don’t actually know what a file is actually meant to do – is it an executable? A zip file? An image?

CyberChef has the operation “Detect File Type”. This fingerprints the file and gives you a best guess as to what it is. It’s not infallible, but it is helpful.

Let’s analyse a slightly longer text file of words.

Add the operation “Entropy”. What this does is look at the “randomness” of a file. By default, this uses something called Shannon entropy calculated across the whole file. Data in the middle ground has structure – it’s either text, an executable, or some other form of information.

Another useful tool to determine file content is “Frequency Distribution”. This counts how often given byte values occur across the file. Frustratingly, the lower axis in in decimal not hex, but you can see some clear spikes. The very high one – at 32 – is 0x20 or space. The cluster around 107 are lowercase characters.

Now try the same with write.exe and a zip file (>10Kbytes or so).

You’ll see that write.exe actually has entropy similar to text – despite it not being text. It has structure however and does contain some text. The frequency distribution is quite different though!

I’ve added another operation – remove null bytes – so that the huge number of 0x00 in the file are removed from the graph to make it more clear.

You can see the same “bump” around 107 – this is the text embedded inside the executable. But there is a much wider spread of values than with the text file.

The zip is a different story though. The Shannon entropy is 7.99 out of 8. It’s as random as it could be. This is nearly always a sign of compression or encryption. Nearly all compression works by spotting patterns and condensing them down – hence the structure is removed. Encrypted data should be indistinguishable from random noise.

This is a zip file – so the compression results in high entropy.

When we look at the frequency distribution, we can see a much more uniform distribution across all values. Again, a sign of compression or encryption. These are handy tools to determine what is in a file, especially firmware and the like.

To demonstrate this, you can encrypt a file using AES in Cyberchef and check the entropy and frequency distribution.

You can click this link to see the operations required.


I hope this introductory post helped you to understand binary and some of the tools we can use to convert, decode, and understand the purpose of various data.

Investigating a tricky network problem with Python, Scapy and other network tools

We’ve had a fairly long-term issue at work with connectivity to one of our application servers. Every now and then you can’t login or connect and it has seemed fairly random. This finally annoyed myself and a customer enough that I had to look into it.

The connection is made to the server on port 1494 – Citrix ICA. Initially we suspected that the .ica file downloaded and opened by the Citrix Receiver software was incorrect or corrupt, but inspection and testing of this showed that it was OK. It really did look like the connection was just being randomly rejected.

It seemed that myself and a single customer were having far more frequent issues that other users. Of course it could just be my tolerance for whinging is lower than my colleagues.

Note that nearly all of the below was done on OS X – syntax of some of these commands differs under Linux and Windows. I have changed the host IP for security reasons.


Most applications that listen on a TCP port will respond to telnet, even if they don’t send anything back. Telnet is almost raw TCP – it has some control and escape sequences layered on top, but it is very simple at a protocol level.

ICA responds when connecting by sending back “ICA” every 5s:

andrew@andrews-mbp:/$ telnet 1494
Connected to
Escape character is '^]'.
telnet> quit
Connection closed.

But every now and then I was getting nothing back:

andrew@andrews-mbp:/$ telnet 1494
telnet: connect to address Operation timed out
telnet: Unable to connect to remote host

Oddly, whenever the Citrix Receiver failed to launch, I wasn’t always having problems with telnet, and vice versa. This is good – we’ve replicated the issue with a very simple utility using raw TCP rather than having to look into the intricate details of Citrix and whatever protocol it uses.


So let’s fire up tcpdump to see what happens when the connection is working. tcpdump is a command line packet analyser. It’s not as versatile or as easy to use as Wireshark, but it is quick and easy. You can use tcpdump to generate a .pcap file which can then be opened in Wireshark at a later date – this is good for when you are working on systems with no UI.

I filtered the tcpdump output to only show traffic where one of the two IPs was the server.

andrew@andrews-mbp:/$ sudo tcpdump -i en0 host
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:16:40.027018 IP andrews-mbp.49425 > Flags [S], seq 2598665487, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027031986 ecr 0,sackOK,eol], length 0
23:16:40.080030 IP > andrews-mbp.49425: Flags [S.], seq 3949093716, ack 2598665488, win 8192, options [mss 1400,nop,wscale 8,sackOK,TS val 188590479 ecr 1027031986], length 0
23:16:40.080173 IP andrews-mbp.49425 > Flags [.], ack 1, win 8241, options [nop,nop,TS val 1027032039 ecr 188590479], length 0
23:16:40.321664 IP > andrews-mbp.49425: Flags [P.], seq 1:7, ack 1, win 260, options [nop,nop,TS val 188590504 ecr 1027032039], length 6
23:16:40.321739 IP andrews-mbp.49425 > Flags [.], ack 7, win 8240, options [nop,nop,TS val 1027032280 ecr 188590504], length 0
23:16:42.389928 IP andrews-mbp.49425 > Flags [F.], seq 1, ack 7, win 8240, options [nop,nop,TS val 1027034342 ecr 188590504], length 0
23:16:42.794413 IP andrews-mbp.49425 > Flags [F.], seq 1, ack 7, win 8240, options [nop,nop,TS val 1027034746 ecr 188590504], length 0
23:16:42.796361 IP > andrews-mbp.49425: Flags [.], ack 2, win 260, options [nop,nop,TS val 188590739 ecr 1027034342], length 0
23:16:42.796430 IP andrews-mbp.49425 > Flags [.], ack 7, win 8240, options [nop,nop,TS val 1027034748 ecr 188590739], length 0
23:16:43.055123 IP > andrews-mbp.49425: Flags [.], ack 2, win 260, options [nop,nop,TS val 188590777 ecr 1027034342], length 0
23:16:45.591455 IP > andrews-mbp.49425: Flags [R.], seq 7, ack 2, win 0, length 0

This all looks fairly normal – my laptop is sending a SYN to the server, which responds with SYN-ACK, and then I respond with an ACK. You can see this in the “Flags” part of the capture. S, S., . (. means ACK in tcpdump). Everything then progresses normally until I close the connection.

However, when the connection fails:

23:23:31.617966 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027442033 ecr 0,sackOK,eol], length 0
23:23:32.812081 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027443220 ecr 0,sackOK,eol], length 0
23:23:33.822268 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027444226 ecr 0,sackOK,eol], length 0
23:23:34.830281 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027445232 ecr 0,sackOK,eol], length 0
23:23:35.837841 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027446234 ecr 0,sackOK,eol], length 0
23:23:36.846448 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027447234 ecr 0,sackOK,eol], length 0
23:23:38.863758 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1027449238 ecr 0,sackOK,eol], length 0
23:23:42.913202 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,sackOK,eol], length 0
23:23:50.934613 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,sackOK,eol], length 0
23:24:07.023617 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,sackOK,eol], length 0
23:24:39.154686 IP andrews-mbp.49569 > Flags [S], seq 644234313, win 65535, options [mss 1460,sackOK,eol], length 0

I get nothing back at all – it’s just telnet trying the connection again and again by sending SYNs. I was expecting part of the connection to succeed, but this looked like the host just wasn’t responding at all. This might indicate a firewall or network issue rather than a problem with Citrix.

I used Wireshark on the server side to confirm that no traffic was getting through. I could see the successful connections progressing fine, but I could see nothing of the failing connections. I wanted to check both sides because there were a number of potential scenarios where a client could send a SYN and not get a SYN-ACK back:

  1. Client sends SYN, server never sees SYN.
  2. Client sends SYN, server sees SYN, server sends SYN-ACK back which is lost.
  3. Client send SYN, server sees SYN, choses not to respond.

It seemed that 1 was happening here.

So what was causing this? Sometimes it worked, sometimes it didn’t. Did it depend on what time I did it? Was there another variable involved?


Let’s check for outright packet loss. ping and traceroute are useful for diagnosing packet loss on a link, but it can be hard work working out which step is causing problems. Step in mtr, or my trace route. This provides a tabular, updating output which combines ping and traceroute with a lot of useful information.

                                      My traceroute  [v0.85]
andrews-mbp (                                                    Thu Dec  5 00:52:49 2013
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                         Packets               Pings
 Host                                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
 1.                                     0.0%    36    1.8   4.6   1.2 104.0  17.0
 2.                                         0.0%    36   10.1  41.3   8.1 291.2  68.1
 3.        0.0%    36    9.0  22.4   8.3 185.6  33.7
 4.            0.0%    36   13.5  13.8   8.9  80.2  11.6
 5.         0.0%    35   14.4  17.1  12.9  29.7   3.5
 6.          0.0%    35   17.3  16.0  13.0  23.5   2.3
 7.          0.0%    35   14.5  15.3  12.5  21.1   2.0
 8.                   0.0%    35   15.1  20.7  14.7 113.9  16.4
 9.                                      0.0%    35   20.5  19.7  16.3  34.0   2.9

I let this run for a while and observed virtually no packet loss. It’s important to note that it is using ICMP pings – not TCP as Citrix uses. ICMP messages can be dealt with differently to TCP. mtr does support TCP pings but I can’t get it to work under OS X.

Python and telnetlib

So wrote a small Python program using the telnetlib module to periodically connect to the port using telnet and indicate when there were problems. The output was simple graphical representation so that I could spot any timing patterns.

import telnetlib
import time
import socket

RHOST = ''
RPORT = 1494

StatCounter = 0
FailCounter = 0

while True:

    if StatCounter != 0 and StatCounter % STATFREQ == 0:
        print str(FailCounter) + ' / ' + str(StatCounter)

    StatCounter += 1

    Fail = False
        ica = telnetlib.Telnet(host=RHOST, port=RPORT, timeout=1)
    except socket.timeout:
        Fail = True
        FailCounter += 1

    if not Fail:

    if Fail:
        print '*',
        print '.',

So this prints a . for a successful connection and * for unsuccessful. After every 16 packets, the number of failures/total is printed.

. . . * . . * . . . . * . . * . 4 / 16
. . . * . * . . . . * . . * . . 8 / 32
. . * . . * . . . . * . * . . . 12 / 48
. . . * * . . . . . . * * . . . 16 / 64
. . . . * . . . . . . * * . . . 19 / 80
. . . * * . . . . . . * * . . . 23 / 96
. . . * * . . . . . . * . * . . 27 / 112
. * . . * . . * . . . * . . . . 31 / 128
* . . * . . . * . . * . . . * . 36 / 144
* . . . . . . * * . . . . . . * 40 / 160
. . . . . . * * . . . . . . * . 43 / 176
. . . . . * * . . . . . . * . . 46 / 192
. . . . * * . . . . . . * . . . 49 / 208
* . . . . . . * * . . . . . . * 53 / 224
. . . . . . . * * . . . . . . * 56 / 240
* . . . . . . * * . . . . . . * 60 / 256

What can we note?

  • There is some vague pattern there, often repeating every 8 packets.
  • The rate of failed to successful connections is nearly always 25%.
  • Varying the WAITTIME (the time between connections) had some interesting effects. With short times, the patterns were regular. With longer times they seemed less regular.
  • Using the laptop for other things would disrupt the pattern but packet loss stayed at 25%. Even with very little other traffic the loss was 25%.

What varies over time, following a pattern, but would show behaviour like this?

The source port.

Every TCP connection not only has a destination port, but a source port – typically in the range of 1025 to 65535. The source port is incremented for each connection made. So the first time I telnet it would be 43523, the next time 45324, then 45325 and so on. Other applications share the same series of source ports and increment it as they make connections.

When I run the test program with a short wait time, there is very little chance for other applications to increment the source port. When I run it with a longer wait time (30s or so), many other applications will increment the source port, causing the pattern to be inconsistent.

It really looked like certain source ports were failing to get through to the server.


I had to test this theory. You can’t control the source port with telnet, but you can with the excellent netcat (or nc, as the command is named). “-p” controls the source port:

andrew@andrews-mbp:/$ nc -p 1025 1494
andrew@andrews-mbp:/$ nc -p 1026 1494
andrew@andrews-mbp:/$ nc -p 1027 1494
andrew@andrews-mbp:/$ nc -p 1025 1494
andrew@andrews-mbp:/$ nc -p 1026 1494
andrew@andrews-mbp:/$ nc -p 1027 1494

As you can see – connections from 1025 and 1027 always succeed and 1026 always fail. I tested many other ports as well. We have our culprit!

Python and Scapy

Now, can we spot a pastern with the ports that are failing and those that are succeeding? Maybe. I needed something to craft some TCP/IP packets to test this out. I could use netcat and a bash script, but I’ve recently learnt about Scapy, a packet manipulation module for Python. It’s incredibly flexible but also very quick and easy. I learnt about it after reading the book Violent Python, which I would recommend if you want to quickly get using Python for some real world security testing.

The script needs to connect to the same destination port from a range of source ports and record the results. With Scapy, half an hour later, I have this (note, I did have some issues with Scapy and OS X that I will need to go over in another post):

import time
from scapy.all import *

RHOST = ''
RPORT = 80
LPORT_LOW = 1025

for port in range(LPORT_LOW, LPORT_HIGH):
    ip = IP(dst=RHOST)
    # TCP sets SYN flag by default
    tcp = TCP(dport=RPORT, sport=port)

    # Build packet by layering TCP/IP
    send_packet = ip/tcp

    # Send packet and wait for a single response - aggressive timeout to speed up tests
    recv_packet = sr1(send_packet, timeout=0.1, verbose=0)

    # Red for failed tests, normal for success
    if recv_packet is None:
        colour_start = '�33[0;31m'
        colour_start = '�33[1;0m'

    # Show source port in decimal, hex and binary to spot patterns
    print colour_start + str(port) + ':' + format(port, '#04x') + ':' + format(port, '016b')


This produced really helpful output. The failed packets are highlighted in the excerpt below:


At this point in the port range it appears that packets ending in 001 or 110 are failing.


Move further down the port range and packets ending 000 and 111 are failing.

In fact, at any given point it seems that the packets failing are either 000/111, 001/110, 010/101, 011/100 – complements of one another. Higher order bits seem to determine which pair is going to fail.


What makes this even stranger is that changing the destination port (say, from 1494 to 80) gives you a different series of working/non-working source ports. 1025 works for 1494, but not 80. 1026 works for both. 1027 works only for 80.

All of my tests above have been done on my laptop over an internet connection. I wanted to test a local machine as well to narrow down the area the problem could be in – is it the perimeter firewall or the switch the server is connected to?


The local test machine is a Linux box which is missing Python but has hping3 on it. This is another useful tool that allows packets to be created with a great degree of flexibility. In many respects it feels like a combination of netcat and ping.

admin@NTPserver:~$ sudo hping3 -s 1025 -S -i u100000 -c 20 -p 1494

What does all this mean?

  • First parameter is the IP to connect to.
  • -s is the start of the source port range – hping3 will increment this by 1 each time unless -k is passed
  • -S means set the SYN flag (similar to the Scapy test above)
  • -i u100000 means wait 100000us between each ping
  • -c 20 means send 20 pings
  • -p 1494 is the offending port to connect to

And what do we get back?

--- hping statistic ---
20 packets transmitted, 16 packets received, 20% packet loss
round-trip min/avg/max = 0.3/0.4/0.8 ms

The same sort of packet loss we were seeing before. Oddly, the source ports that work differ from this Linux box to my laptop.

Here’s where it gets even stranger. I then decided to try padding the SYN out with some data (which I think is valid for TCP, though I’ve never seen a real app do it and mtr’s man page says it isn’t). You use -d 1024 to append 1024 bytes of data. I first tried 1024 bytes and had 20% packet loss as before. They I tried 2048 bytes:

--- hping statistic ---
20 packets transmitted, 20 packets received, 0% packet loss
round-trip min/avg/max = 0.5/0.6/0.9 ms

Wait? All the packets are now getting through?

Through a process of trial and error I found that anything with more than 1460 bytes of data was getting through fine. 1460 bytes of data + 20 bytes TCP header + 20 bytes IP header = 1500 bytes – this is the Ethernet MTU (Maximum Transmit Unit). Anything smaller than this can be sent in a single Ethernet frame, anthing bigger needs to be chopped up into multiple frames (although some Ethernet networks allow jumbo frames which are much bigger – this one doesn’t).

I then ran the hping3 test from my laptop and found that altering the data size had no impact on packet loss. I strongly suspect that this is because a router or firewall along the way is somehow modifying or reassembling the fragmented frames to inspect them, and then reassembling them in a different way.

At this point I installed the Broadcom Advanced Control Suite (BACS) on the server to see if I could see any further diagnostics or statistics to help. One thing quickly stood out – a counter labelled “Out of recv. buffer” was counting up almost in proportion to the number of SYN packets getting lost:


This doesn’t sound like a desirable thing. It turns out the driver is quite out of date – maybe I should have started here!


I’m still not sure what is going on here. The packets being rejected do seem to follow some kind of pattern, but it’s certainly not regular enough to blame it on the intentional behaviour of a firewall.

At this point we are going to try upgrading the drivers for the network card on the sever and see if this fixes the issue.

The point of all of the above is to show how quick and easy it can be to use easily available tools to investigate network issues.

Reverse engineering Megamos Crypto?

Some of you might have read the stories going around a few weeks ago – “Scientist banned from revealing codes used to start luxury cars“. The short of it is that a security researcher has had a injunction imposed on him, preventing him from publishing a paper. The paper reveals security problems in the Megamos Crypto system used in the immobiliser system of many cars. Volkswagen are not happy – it really seems they want this shut down.

(As an aside, I hate the way that mainstream media refers to “codes” – it can mean source code, executables, an algorithm, or even a secret key. Often used interchangeably in the same article)

Details were a little scant, but last night the EFF passed comment, based on the court’s decision.

I am not a lawyer – I’m not going to pass judgement on the legal side. But what is interesting is how the researchers got hold of the Megamos Crypto algorithm. It wasn’t by decapping the chips in the transponders, it wasn’t from observing them black-box, it wasn’t from looking at an embedded software implementation – they took a Windows program used to clone car key transponders and reverse engineered that.

In terms of working out how Megamos was implemented, someone else had already done the hard work. This left the researchers to perform detailed cryptanalysis of the algorithm and – rumour has it – find some serious problems.

The piece of software is called “Tango Programmer“, a third party tool (software and hardware) used to make transponders. This has been available since at least 2009.

Tango Programmer is readily available, but it appears that it needs to be bought alongside a physical programmer. I strongly suspect that the software would be available on file sharing sites illegally, or possibly even legitimately on another site if you look hard enough.

Another company, Bicotech, produce a similar tool called RwProg. The software is downloadable from their website. The executable is packed, but I am sure it would be perfectly possible to reverse engineer the algorithm from the binary.

The court decision itself contains valuable information on Megamos as well, notably from paragraphs 4 and 5:

In detail the way this works is as follows: both the car computer and the transponder know a secret number. The number is unique to that car. It is called the “secret key”. Both the car computer and the transponder also know a secret algorithm. That is a complex mathematical formula. Given two numbers it will produce a third number. The algorithm is the same for all cars which use the Megamos Crypto chip. Carrying out that calculation is what the Megamos Crypto chip does.

When the process starts the car generates a random number. It is sent to the transponder. Now both computers perform the complex mathematical operation using two numbers they both should know, the random number and the secret key. They each produce a third number. The number is split into two parts called F and G. Both computers now know F and G. The car sends its F to the transponder. The transponder can check that the car has correctly calculated F. That proves to the transponder that the car knows both the secret key and the Megamos Crypto algorithm. The transponder can now be satisfied that the car is genuinely the car it is supposed to be. If the transponder is happy, the transponder sends G to the car. The car checks that G is correct. If it is correct then the car is happy that the transponder also knows the secret key and the Megamos Crypto algorithm. Thus the car can be satisfied that the transponder is genuine. So both devices have confirmed the identity of the other without actually revealing the secret key or the secret algorithm. The car can safely start. The verification of identity in this process depends on the shared secret knowledge. For the process to be secure, both pieces of information need to remain secret – the key and the algorithm.

In standard cryptography terminology:

A car \text{C} and a transponder \text{T} share a secret key K. A pseudo-random function family \textsf{PRF} is keyed using key K i.e. \textsf{PRF}_K. The output from this PRF is split into two parts F and G.

  1. \text{C} generates a random number r.
  2. \text{C} calculates (F,G) = \textsf{PRF}_K(r)
  3. \text{C} \to \text{T}: r, F
  4. \text{T} calculates (F',G') = \textsf{PRF}_K(r)
  5. \text{T} checks that F = F'
  6. \text{T} \to \text{C}: r, G
  7. \text{C} checks that G = G'

This process means that the transponder believes the car knows the key and PRF, and the car believes the transponder knows the key and PRF. They should have authenticated themselves with each other.

What is a PRF? A pseudo-random function is similar in many respects to a psuedo-random number generator (PRNG), except instead of sequentially generating output, you can randomly access any of the outputs using an index (r in the example above). The key is analogous to the seed of the PRNG. Using a certain key, a given input will map to a determined output.

Importantly, the output of a PRF should be indistinguisable to an observer from a random function, and by extension you should not be able to derive the key even if inputs, outputs, or free access to the function is given. You should also not be able to tell which PRF is in use even if you can control the inputs and read the outputs.

So – if this is a secure, solid, verified PRF, the protocol should be secure, even if we know what the PRF is. The only thing that needs to be kept secret is the key.

But the court decision says:

The verification of identity in this process depends on the shared secret knowledge. For the process to be secure, both pieces of information need to remain secret – the key and the algorithm.

This suggests a few things:

  1. The PRF used is not secure
  2. They don’t know what they are talking about

Both are entirely possible, but I would strongly suspect that the PRF has issues and they want to keep it secret. This would be a clear example of “security through obscurity”.

How could a PRF be insecure?

  • Using one or more input/output pairs, it might be possible to derive the key.
  • You might not need a key to derive the output given the input.
  • The key length might not be long enough to prevent bruteforcing.
  • F and G might not depend on the whole key i.e. you might be able to calculate G given part of the key.

The protocol itself might suffer from further issues:

  • There does not appear to be any protection from replay attacks (prevented from being used as a direct vulnerability because the authentication is bidirectional).
  • Is the random nummber actually random? Does it matter if it isn’t? If they are re-used (i.e. it’s not a nonce), it probably does matter.
  • The transponder can bypass the check for F = F’ – it can be a “yes” key. If we don’t need the entire key to compute G, this matters.
  • The key might be constant across an entire line or make of cars. Recover the key from one transponder and there would be no secrets left.
  • The key might be derived from an open piece of information like the car VIN number
  • The key might be derived from something like the manufacture date/time of the car, massively reducing keyspace
  • Probably a million more things

Let’s look at the attacks described in the court decision.

Firstly, note:

The attacks are not, themselves, trivial things to do. However, they allow someone, especially a sophisticated criminal gang with the right tools, to break the security and steal a car.

This makes it sound like some of these attacks are practical i.e. it won’t take 2 weeks of effort after decapping and reading the key from EEPROM.

Attack 1:

One attack relies on weaknesses in the secret keys that are used in certain cars. That “weak key” weakness arises because certain car makers have used weak secret keys which are easier to guess than they need to be. In effect, it is a bit like using the word “password” for a password.

As I mentioned above, there are a number of situations where the keys chosen might be poor. It might be the case that the researchers need 2 weeks to work out the key given a car and transponder, but then if the same key is used across all cars, it doesn’t really matter.

Attack 2:

Another is concerned with key updates. The details do not matter.

This is very vague.  Maybe you can alter or add keys easily if you already have access to the car?

Attack 3:

The third attack relates to weaknesses in the Megamos Crypto algorithm itself. The academics explain this attack in the paper, and, as I say, the paper also sets out the whole of the algorithm. It is these two elements that the claimants seek to prevent publication of. The claimants wish to remove the Megamos Crypto algorithm and information about the attack based on the weakness in it from the paper.

This is where we get to the point that it sounds like the PRF is not secure. It sounds like this attack may take days of work with access to both the car and transponder.

This could be like the insecurities found in Keeloq. The first step was determining the details of the algorithm. The first few papers detailed weaknesses that meant the protocol was insecure, but the weakness could not practically be exploited. After this, papers were released that detailed faster, more effective attacks, until finally we are at the stage where Keeloq can be called “broken”.

A quick look at some of the software

I haven’t got hold of Tango Programmer, but I do have RwProg up and running. Here is a screenshot:2013-08-07 22_37_06-RwProg   v2.17.0002

What can we tell from this? Well, the crypto key looks to be 96bits long – too long to bruteforce.

There are a few videos as well:

Nothing really groundbreaking. I can’t see how the software reads and then writes the crypto key.


Regardless of the court decision, it looks like there is enough information out there for other people to start work on this. Download the software, maybe buy Tango Programmer, reverse the algorithm and then let the world loose on it!


We need an antidote to the anti-code

In the last post, I briefly went over the process of reverse engineering the algorithm behind an anti-code generator for an alarm system.

It turned out that the algorithm was very simple indeed. For a given 5-digit numeric quote code, we can derive a 5-digit reset code using a “secret” 8-bit (256 possibilities) version number as a key. This has a lot in common with a keyed hash function or a message authentication code.

There are some pretty serious security implications with this mechanism.

5 digit numeric codes are never going to be strong

Even if I had to enter a pin at random, a 5-digit numeric code only has 100,000 options – I have a 1/100,000 chance of getting it right.

If we made this into a 5-digit hexadecimal code, we would now have a 1/1,048,576 chance – a factor of over 10 times less likely.

Up this to a 6-digit alphanumeric code, and it is now 1/2,176,782,336 – a factor of over 20,000 times less likely we could guess the code.

It doesn’t take many alterations to the limits on codes to make them much more secure.

For this reason it surprises me that alarms are still using 4-digit pins, but most internet forums insist on 8-character passwords with alphanumeric characters and punctuation.

The algorithm isn’t going to stay secret

There is no way to reliably protect a computer application from reverse engineering. If you can run it, at all, it is highly likely the operation can be observed and reversed. Relying on the secrecy of an algorithm or a key hidden within the software is not going to afford any level of security.

Once we know the algorithm, the odds massively improve for an attacker

The algorithm takes a version number from 0-255. For a given quote code, I can try each version number, giving me a list of up to 256 potentially valid reset codes (sometimes, two version numbers will generate the same reset code).

If I enter a code from this list, I now have a 1/256 chance of getting it right. Not great compared to 1/100,000 for a purely random guess.

This is entirely due to the short version number used.

Given a quote/reset code, most of the time we can infer the version

It quickly became apparent that for most quote/reset pairs, there was only a single version number than could produce this pair. I’m awful at probability and decision maths, so I thought running a simulation would be better.

I like running simulations – generally when the number of simulations becomes large enough, the results tend towards the correct value. So I tried the following:

1. Generate a genuine quote/reset pair using a random quote.

2. Use a brute force method to see which version numbers can produce this pair

3. Record if more than one version number can produce this quote/reset pair.

I started doing this exhaustively. This would take a long time though… someone on the Crypto stack exchange answered my question with a neater, random simulation.


I ran this test over 20 million times. From this it turns out that 99.75% of quote/reset code pairs will directly tell me the version number. Most of the remaining 0.25% require yield two version numbers. A tiny number (<0.001%) yield more than four version numbers. You are almost certain to know the version number after two quote/reset pairs as a result.

What does this mean in the real world?

The version number is treated as the secret, and I am informed that this secret is often constant across an entire alarm company. All ADT alarms or all Modern Security Systems alarms may use the same version number to generate reset codes.

This means I could get hold of any quote/reset pair, infer the version number, and then use that later to generate my own anti-codes for any ADT alarm. I could get hold of these quote/reset pairs by going to an accomplice’s house with a ADT alarm system, or by eavesdropping on communications.

With that anti-code I could either reset a system presenting a quote code, or impersonate an alarm receiving centre (there are other speech based challenge-response requirements here to prove the caller is genuine, which are easily gamed I would imagine).


A 5-digit reset code using an 8-bit key is never going to be secure.

When computer passwords are 8 characters and 128-bit keys are the norm, this anti-code mechanism seems woefully inadequate.

Reversing an anti-code

A contact in the alarm industry recently asked if I could take a look at a quick reverse engineering job. I’m trying to gain some credibility with these guys, so I naturally accepted the challenge.

Many alarms have the concept of an “anti-code”. Your alarm will go off and you will find it saying something like this on the display:


QUOTE 12345

The idea is then that you call the alarm receiving centre, quote 12345, they will input this into a PC application, get a reset code back, tell the customer, and then they can use this to reset the alarm. This means that you need to communicate with the alarm receiving centre to reset the alarm.

Alarm manufacturers provide their own applications to generate these codes. This particular manufacturer provides a 16-bit MS-DOS command line executable, which will refuse to run on modern PCs. This is a pain – it’s not easy to run (you need to use a DOS emulator like DOS-BOX) and it doesn’t allow for automation (it would be convenient to call a DLL from a web-based system, for example).

So I was asked if I could work out the algorithm for generating the unlock codes. x86 reverse engineering is not my forté, especially older stuff, but I thought i would have a quick go at it.

Turns out it was easier than expected! I find documenting reverse engineering incredibly difficult in a blog format, so I’ll just cover some of the key points.

Step 1: Observe the program

First things first, let’s get the program up and running. DOS-BOX is perfect for this kind of thing.

The program takes a 5 digit input and produces a 5 digit output. There is also a version number which can be input which varies from 0-255.

I spent a while playing around with the inputs. Sometimes things like this are so basic you can infer the operation (say, if it is XORing with a fixed key, flipping the order of some bits or similar). It didn’t look trivial, but it was plain to see that there were only two inputs – the input code and version. There was no concept of time or a sequence counter.

At this stage, I’m thinking it might be easiest to just create a lookup for every single pin and version. It would only be 2,560,000 entries (10,000 * 256). That’s a bit boring though, and I don’t have any idea how to simulate user input with DOS-BOX.

Step 2: Disassemble the program

To disassemble a program is to take the machine code and transform it into assembly language, which is marginally more readable.

There are some very powerful disassemblers out there these days – the most famous being IDA. The free version is a bit dated and limited, but it allowed me to quickly locate a few things.

An area of code that listens out for Q (quit) and V (version), along with limiting input characters from 0-9. Hex values in the normal ASCII range along with getch() calls are a giveaway.

Keyboard input
Another area of code appears to have two nested loops that go from 0-4. That would really strongly indicate that it is looping through the digits of the code.

Other areas of code add and subtract 0x30 from keyboard values – this is nearly always converting ASCII text numbers to integers (0x30 is 0, 0x31 is 1 etc. so 0x34 – 0x30 = 4)


A block of data, 256 items long from 0-9. Links in with the maximum value of the “version” above. Might just be an offset for indexing this data?

IDA’s real power is displaying the structure of the code – this can be a lot more telling than what the code does, especially for initial investigations.

Code structure
It’s still assembly language though, and I’m lazy…

Step 3: Decompile the program

Decompiling is converting machine code into a higher level language like C. It can’t recover things like variable names and data structures, but it does tend to give helpful results.

I used the free decompiler dcc to look at this program. I think because they are both quite old, and because dcc has signatures for the specific compiler used, it actually worked really well.

One procedure stood out – proc2, specifically this area of code:
DCC outputIt’s a bit meaningless at the moment, but it looks like it is two nested while loops, moving through some kind of data structure, summing the results and storing them. This is almost certainly the algorithm to generate the reset code.

Now, again, I could work through this all and find out what all the auto named variables are (i.e. change loc4 to “i” and loc5 to “ptrVector”. Or I could step through the code in a debugger and not have to bother…

Step 4: Run the code in a debugger

A debugger allows you to interrupt execution of code and step through the instructions being carried out. It’s generally of more use when you have the source code, but it is still a helpful tool. DOS-BOX can be run in debug mode and a text file generated containing the sequence of assembly instructions along with the current registers and what is being written and read from them. It’s heavy going, but combined with IDA and the output from DCC, it’s actually quite easy to work out what is going on!

Step 5: Write code to emulate the behaviour

Shortly after, I had an idea how the algorithm worked. Rather than work it through by hand, I knocked up a quick Python program to emulate the behaviour.The first cut didn’t quite work, but a few debug statements and a couple of tweaks later, and I was mirroring the operation of the original program.

Overall, it was only a few hours work, and I’m not really up on x86 at all.

I’m not releasing the algorithm or the software, as it could be perceived as a threat. In the next post, I am going to discuss some of my security concerns around the idea of an anti-code and this specific implementation.

Reverse engineering a wireless burglar alarm, part 8

Last time we worked out that we could drive the CC1150 daughterboard using the Seeeduino Mega Arduino board easily.

The Seeeduino Mega uses a ATmega2560 microcontroller, which has a number of features which will help us do this:

  • Hardware SPI interface – this makes high speed SPI connections almost effortless. No nasty bit-banging here.
  • Pin change interrupts – this means we can quickly and efficiently respond to the clock signal coming into the microcontroller. 6.25kHz isn’t so fast that it precludes the use of a polling technique, but polling is wasteful and limits what we can do with the rest of our time.
  • Lots of memory – we don’t need to care about creating wasteful buffers. The 8KB in the 2560 is far more flexible that 2KB in the 328.

Connecting the hardware is easy – we need to use PB0 (CS), PB1 (CLK), PB2 (MOSI) and PB3 (MISO). These map through to pins 53, 52, 51 and 50. We are going to use the adjacent PL0 port (pin 49) to clock out the data to GDO0. VCC is supplied from the 3.3V supply on the Seeeduino Mega (it is switchable from 3.3 to 5V operation – a valuable addition).

It was at this time I found out that if you connect the CC1150 incorrectly it gets very hot and starts letting out the magic smoke. It also still works after this..

Now, as for the software – I’m not really sure where to begin. It’s quite hard to explain bit-by-bit. I hope it is self-explanatory, to a large extent.

The code is available on github. There isn’t really much fancy going on here – I am using some CC11xx headers from previous projects to help identify the settings registers. All this aims to do is take the 48 bit (6 bytes) of data we have sniffed from a door contact, and replay this signal. It configures the CC1150 exactly as per the sniffed traffic, and then we use a pin change interrupt to clock the data out.

Oooo 2-FSK!

Oooo 2-FSK!

Once all of this is wired up, I can immediately see that my spectrum analyser is showing bursts of 2-FSK. That must mean things are approximately correct. I used to hate working with RF – I could never tell if it was the transmitter, receiver or just idiocy that was preventing things from working. A spectrum analyser gives you reassurance that you are doing at least something right.

So now I need to see if the bell box can learn the door contact from this signal. I connect up the bellbox, press and hold learn, and… nothing. I try the door contact, and it is learnt immediately. I must be doing something wrong.

(when working with alarm sirens, you need to be careful. If you leave the siren connected, they can be loud enough to actually damage your hearing. I never trust the soft setting to turn off the siren for testing, so always disconnect it physically. This then presents another danger – siren drivers can produce large voltages when the siren is open circuit – sometimes hundreds of volts, enough to give you a fair belt.)

The trusty Saleae Logic Analyser comes out again, and I connect it up to my cloned alarm sensor. It’s apparent that the signal isn’t quite right – a few extra bits here and there. A really quick and easy debug method is to use extra IO ports – PL1 and PL2 in this case – to  observe timing and code flow. Serial is just too slow for this kind of stuff – it might help me, but normally it slows down execution enough to cause problems. ATmega chips do have on-chip debug, but it is always more trouble than it is worth – these are very simple microcontrollers!

He's learning, we're learning.

He’s learning, we’re learning.

It turns out it is a simple bug – I’d used an incorrect comparison to signify the end of a packet. The code is fixed, and I try again. This time the panel learns the remote!

So where to from here? I need to start experimenting with the data in the packet, and seeing what the panel will and won’t accept. Next time.

Reverse engineering a wireless burglar alarm, part 7

So, where were we? We’ve worked out how several components of the alarm use the CC1150 to send out signals, and now I want to emulate the signal and then play with it.

By far the easiest way to do this is to use the same CC1150 chip as in the authentic transmitters. I can be absolutely sure the 2-FSK modulation with ~50kHz deviation will be reproduced exactly. There are many other RF transmitter chips out there that can do this, but we know that these work and we already have the correct settings.

So, the first challenge is – where do I get a board with a CC1150 on it? Well, we have plenty of alarm sensors with neat little CC1150 daughterboards on them!

Door contact RF board

Door contact RF board

I chose a door contact as my victim – they are the cheapest and simplest sensors.

The daughterboard has an 8-pin interface, with a 0.1″ pitch pin header connecting the two. I need to desolder the board to use it. Desoldering pin headers is always a pain – you can’t desolder all 8 pins at a time. My technique now is:

  1. Chop the pin header along it’s length, separating the two boards. I have a nice pair of snips for this, but a Dremel with cut-off wheel can do it if they don’t fit.
  2. Chop the plastic spacer away from the pins, leaving you with just half the pins stuck in the board.
  3. Desolder the pins one at a time.


The daughterboard

Now I have a small CC1150 board. I soldered a new pin header onto it, and a pin socket onto the rest of the door contact. This allows me to easily reconnect it to the door contact if I need to.

You’ll notice in the picture above that there is a small bridge wire. I desoldered and resoldered the CC1150 QLP package (just to see if I could) and burned a track in the process.


Testing testing 1-2

Now we need to drive this board using something. There are a plethora of options available to me:

  • An Arduino board
  • A mbed board
  • A Bus Pirate
  • A GoodFET
  • A CC1110 development board (this would skip using the CC1150 board, as it has a built-in transceiver)
  • One of the Ciseco CC11xx boards (again, this would skip the CC1150 board)
  • Any of the other development boards I have lying around…

I decided I would use an Arduino board – specifically the Seeeduino Mega.

Why the Arduino? In reality, I’ve stopped using the Arduino environment and now use AVRGCC, so this is really just a nice little AVR development platform.

It won out over the mbed as I have had SPI issues with the mbed before.

The Bus Pirate and GoodFET are very similar, but require a host PC. I would like this standalone.

It seemed silly to implement this on a CC1110 chip, and I have had problems in implementing proprietary encoding schemes on it before.

Why the Mega? I find myself often using the extra pins and RAM as compared to the ATmega328p used in the Uno

So, now I have a RF board and something to control it. Next, I need to connect the two and write some code!