
	Hexprobe

	what's new?
	key features
	screen shots
	history & awards

	download

	how to install?
	how to uninstall?

	buy now

	order FAQ
	how to register license code?

	supports

	product FAQ
	online manual
	online template

	contact us

	links

Product List

Current Page: Hexprobe --> how to analyze a binary file...

How to analyze a binary file?

In this article, we will describe how to understand a binary file format by mapping the file to a hierarchical list of data structures and analyzing their properties and relations. We will first introduce the private key blobs briefly and provide a private key blob as a sample binary file. We will then create a document template for the private key blobs using Hexprobe template techniques and parse the sample file to a hierarchical list of simpler data members. We will finally analyze some components contained by the sample blob using two Hexprobe utilities to verify their properties and mathematic relations.

To follow this article, you will need two utilities mentioned above: Hexprobe Hex Editor and Hpmbcalc Hex Calculator, which can be downloaded from this website. For convenience in name, the following content will use Hexprobe and Hpmbcalc to denote the two utilities respectively.

About private key blobs

There are some situations where you need export keys from the secure environment of the cryptographic service provider (CSP) and into your application's data space. Keys that have been exported are stored in encrypted data structures known as key blobs. Later, the key blob can be imported into a provider (often a different CSP on a different computer). This will create a key in the CSP that is a duplicate of the one that was exported. In this way, key blobs are used as the medium for securely transferring keys from one CSP to another.

1. Private key blobs

Private key blobs are used to store RSA public/private key pairs. They have the following format:

BLOBHEADER blobheader;
RSAPUBKEY rsapubkey;
BYTE modulus[rsapubkey.bitlen/8];
BYTE prime1[rsapubkey.bitlen/16];
BYTE prime2[rsapubkey.bitlen/16];
BYTE exponent1[rsapubkey.bitlen/16];
BYTE exponent2[rsapubkey.bitlen/16];
BYTE coefficient[rsapubkey.bitlen/16];
BYTE privateExponent[rsapubkey.bitlen/8];

If the key blob is encrypted, then everything but the BLOBHEADER portion of the blob is encrypted. Note that the encryption algorithm and encryption key parameters are not stored along with the private key BLOB. It is the responsibility of the application to manage this information.

The following table describes each private key BLOB component (Table 1):

Field	Description
*blobheader*	A BLOBHEADER structure as described in the below. The *bType* field must always have a value of 7.
*rsapubkey*	A RSAPUBKEY structure as described in the below. The *magic* field must always have a value of 0x32415352 ("RSA2"). The *pubexp* field is often known as "e".
*modulus*	The modulus. This has a value of "*prime1* * *prime2*" and is often known as "n".
*prime1*	Prime number 1, often known as "p".
*prime2*	Prime number 2, often known as "q".
exponent1	Exponent 1. This has a numeric value of "d mod (p - 1)".
*exponent2*	Exponent 2. This has a numeric value of "d mod (q - 1)".
*coefficient*	Coefficient. This has a numeric value of "(inverse of q) mod p".
*privateExponent*	Private exponent, often known as "d".

2. BLOBHEADER structure

The BLOBHEADER structure, indicates a key's blob type and the algorithm that the key uses.

typedef struct _BLOBHEADER {
  BYTE bType;
  BYTE bVersion;
  WORD reserved;
  ALG_ID aiKeyAlg;
} BLOBHEADER;

To get detailed description about the structure, please refer to Microsoft Cryptography Reference.

3. RSAPUBKEY structure

The RSAPUBKEY structure contains information specific to the particular public key contained in the key BLOB.

typedef struct _RSAPUBKEY {
  DWORD magic;
  DWORD bitlen;
  DWORD pubexp;
} RSAPUBKEY;

magic: Set to RSA1 (0x31415352) for public keys and to RSA2 (0x32415352) for private keys. The hexadecimal values are the ASCII encoding of RSA1 and RSA2.
bitlen: Number of bits in the modulus. In practice, this must always be a multiple of eight.
pubexp: The public exponent.

Sample private key blob

The following hex dump shows a sample private key blob that will be used in this article, generated by the Microsoft Base Cryptographic Provider. It contains a key exchange public/private key pair and is not encrypted.


0x00000000  07 02 00 00 00 a4 00 00  ........
0x00000008  52 53 41 32 00 02 00 00  RSA2....
0x00000010  01 00 01 00 6b df 51 ef  ....k.Q.
0x00000018  db 6f 10 5c 32 bf 87 1c  .o.\2...
0x00000020  d1 4c 24 7e e7 2a 14 10  .L$~.*..
0x00000028  6d eb 2c d5 8c 0b 95 7b  m.,....{
0x00000030  c7 5d c6 87 12 ea a9 cd  .]......
0x00000038  57 7d 3e cb e9 6a 46 d0  W}>..jF.
0x00000040  e1 ae 2f 86 d9 50 f9 98  ../..P..
0x00000048  71 dd 39 fc 0e 60 a9 d3  q.9..`..
0x00000050  f2 38 bb 8d 5d 2c bc 1e  .8..],..
0x00000058  c3 38 fe 00 5e ca cf cd  .8..^...
0x00000060  b4 13 89 16 d2 07 bc 9b  ........
0x00000068  e1 20 31 0b 81 28 17 0c  . 1..(..
0x00000070  c7 73 94 ee 67 be 7b 78  .s..g.{x
0x00000078  4e c7 91 73 a8 34 5a 24  N..s.4Z$
0x00000080  9d 92 0d e8 91 61 24 dc  .....a$.
0x00000088  b5 eb df 71 66 dc e1 77  ...qf..w
0x00000090  d4 78 14 98 79 44 b0 19  .x..yD..
0x00000098  f6 f0 7d 63 cf 62 67 78  ..}c.bgx
0x000000a0  d0 7b 10 ae 6b db 40 b3  .{..k.@.
0x000000a8  b2 eb 2e 9f 31 34 2d cb  ....14-.
0x000000b0  bf a2 6a a6 1f e9 03 42  ..j....B
0x000000b8  f2 63 9b b7 33 d0 fe 20  .c..3.. 
0x000000c0  83 26 1f 56 a8 24 f5 6d  .&.V.$.m
0x000000c8  19 51 a5 92 31 e4 2b bc  .Q..1.+.
0x000000d0  11 c8 26 75 a0 51 e9 83  ..&u.Q..
0x000000d8  ca ee 4b f0 59 eb a4 81  ..K.Y...
0x000000e0  d6 1f 49 42 2b 75 89 a7  ..IB+u..
0x000000e8  9f 84 7f 1f c3 8f 70 b6  ......p.
0x000000f0  7e 06 5e 8b c9 53 65 80  ~.^..Se.
0x000000f8  b7 16 f2 5e 5e de 0b 57  ...^^..W
0x00000100  47 43 86 85 8a fb 37 ac  GC....7.
0x00000108  66 34 ba 09 1a b1 21 0b  f4....!.
0x00000110  aa fa 6c b7 75 a7 3e 23  ..l.u.>#
0x00000118  18 58 95 90 b5 29 a4 1e  .X...)..
0x00000120  15 76 52 56 bb 3d 6b 1d  .vRV.=k.
0x00000128  2a d1 9f 5c 8a c0 55 ea  *..\..U.
0x00000130  c3 29 a2 1e              .)..

You can download the private key blob, or create a private key blob by yourself according to the following steps:

Select and copy the above hex dump to the clipboard;
Run Hpmbcalc and activate its Number Formatter;
Paste the hex dump from the clipboard to the Number Formatter;
Format the hex dump and save it as binary file.

To get the detailed information about the Number Formatter, please refer to: how to format number using Hpmbcalc?

You can also use Hpmbcalc to create a public/private key pair and export it as an encrypted private key blobs.

Private key blob template

To present and analyze the private key blobs much clear and easier, you can develop a special template using Hexprobe template techniques. The following text shows such a template (keyblob_private.tem) that can be used to map a private key blob into the simpler components. If you want to know about the template techniques or want to know how to write a template, please refer to Hexprobe Online Reference Manual.

/**************************************************************
 * Hexprobe Hex Editor Document Mapping Template
 *
 * File:     keyblob_private.tem
 * Author:   Hexprobe System
 * Revision: 2.21
 * Purpose:  Map the Private key blobs (type PRIVATEKEYBLOB). 
 *           The blobs are used to store RSA public/private 
 *           key pairs.
 *************************************************************/

SetDocOnlineHelp("http://msdn.microsoft.com/library/default.asp?
    url=/library/en-us/seccrypto/security/private_key_blobs.asp");

// Exported key blob definitions
var SIMPLEBLOB         = 0x1;
var PUBLICKEYBLOB      = 0x6;
var PRIVATEKEYBLOB     = 0x7;

// Define structures used in Private key blobs 

<%
typedef struct BLOBHEADER {
    BYTE   bType;
    BYTE   bVersion;
    WORD   Reserved;
    DWORD  aiKeyAlg;
};

typedef struct RSAPUBKEY {
    DWORD magic;
    DWORD bitlen;
    DWORD pubexp;
};
%>

// Document mapping entry point function

function MainMapping()
{
    var str, nVal, nVer, bytelen, dwMagic;

    // Define the headers
    <%
    BLOBHEADER blobheader;
    RSAPUBKEY rsapubkey;
    %>

    nVal = GetVariableInt("blobheader.bType");
    nVer = GetVariableInt("blobheader.bVersion");
    bytelen = GetVariableInt("blobheader.Reserved");
    dwMagic = GetVariableInt("rsapubkey.magic");
    if (nVal != PRIVATEKEYBLOB || nVer != 0x02 || 
        bytelen != 0 || dwMagic != 0x32415352) {
        Message( "Template stopped: invalid private key blobs." );
        return;
    }

    // The size of the public key modulus data 
    bytelen = GetVariableInt("rsapubkey.bitlen");
    if (bytelen > 0) {
        str  = "BYTE modulus[" + (bytelen/8) + "];\n";
        str += "BYTE prime1[" + (bytelen/16) + "];\n";
        str += "BYTE prime2[" + (bytelen/16) + "];\n";
        str += "BYTE exponent1[" + (bytelen/16) + "];\n";
        str += "BYTE exponent2[" + (bytelen/16) + "];\n";
        str += "BYTE coefficient[" + (bytelen/16) + "];\n";
        str += "BYTE privateExponent[" + (bytelen/8) + "];\n";
        Evaluate(str);
    }
}

MainMapping();

Parse private key blobs

So far, you have prepared a sample private key blob and a sample template for the binary file. Now, you can take the following steps to parse the file:

Run Hexprobe;
Open the sample private key blob: privatekey.blob;
Load and run the private key blob template: keyblob_private.tem;
Operate the data members in the Template Result List (see figure below).

Notice that the BLOBHEADER and RSAPUBKEY structures have been assigned the following values:

With the Hexprobe, you can easily copy the numeric values of each of the large numeric fields contained by the sample private key blob to the clipboard:

In the Template Result List of the Hexprobe, select the variable you want to copy. The same range of data would be selected in the Hex-data Panel.
Copy the selection in the Hex-data Panel to the clipboard.

The following table shows the numeric values (Table 2). As with all of the numbers in the public and private key blobs, these are in little-endian byte order.

Field name	Value
*modulus*	6b df 51 ef db 6f 10 5c 32 bf 87 1c d1 4c 24 7e e7 2a 14 10 6d eb 2c d5 8c 0b 95 7b c7 5d c6 87 12 ea a9 cd 57 7d 3e cb e9 6a 46 d0 e1 ae 2f 86 d9 50 f9 98 71 dd 39 fc 0e 60 a9 d3 f2 38 bb 8d
*prime1*	5d 2c bc 1e c3 38 fe 00 5e ca cf cd b4 13 89 16 d2 07 bc 9b e1 20 31 0b 81 28 17 0c c7 73 94 ee
*prime2*	67 be 7b 78 4e c7 91 73 a8 34 5a 24 9d 92 0d e8 91 61 24 dc b5 eb df 71 66 dc e1 77 d4 78 14 98
*exponent1*	79 44 b0 19 f6 f0 7d 63 cf 62 67 78 d0 7b 10 ae 6b db 40 b3 b2 eb 2e 9f 31 34 2d cb bf a2 6a a6
*exponent2*	1f e9 03 42 f2 63 9b b7 33 d0 fe 20 83 26 1f 56 a8 24 f5 6d 19 51 a5 92 31 e4 2b bc 11 c8 26 75
*coefficient*	a0 51 e9 83 ca ee 4b f0 59 eb a4 81 d6 1f 49 42 2b 75 89 a7 9f 84 7f 1f c3 8f 70 b6 7e 06 5e 8b
*privateExponent*	c9 53 65 80 b7 16 f2 5e 5e de 0b 57 47 43 86 85 8a fb 37 ac 66 34 ba 09 1a b1 21 0b aa fa 6c b7 75 a7 3e 23 18 58 95 90 b5 29 a4 1e 15 76 52 56 bb 3d 6b 1d 2a d1 9f 5c 8a c0 55 ea c3 29 a2 1e

Analyze private key blob components

You can now use with Hexprobe and Hpmbcalc to analyze and verify the number properties of the large numeric fields and their math relations.

1. Copy the variable value to Hpmbcalc

To analyze the number properties of the large numeric fields and their math relations, you often need copy the Hexprobe's template variables to the Hpmbcalc's operand edit boxes.

In the Template Result List of the Hexprobe, select and copy the variable that you want to use.
Paste the data as Hex Text format from the clipboard to Hpmbcalc's one of the operand edit box.

Because the numbers in the public and private key blobs are in little-endian byte order, and the Hpmbcalc needs big-endian numbers to perform multiple-precision math operations, each operand came from key blobs should be reversed using Hpmbcalc operator Reverse Operand.

2. Is prime1 or prime2 a prime?

To test the primality of the variable prime1 or prime2 contained by the sample private key blob, you can take the following steps:

Copy variable prime1 (prime2) to Hpmbcalc's one of operand edit box.
Reverse prime1 (prime2) using Hpmbcalc Reverse Operand operator.
Test the primality of prime1 (prime2) using Hpmbcalc General Primality Test command.

3. If modulus equals to prime1 * prime2?

You can take the following steps to verify if modulus equals to prime1 * prime2.

Copy variable prime1 to Hpmbcalc's First Operand, and variable prime2 to Second Operand.
Reverse prime1 and prime2 using Hpmbcalc Reverse Operand operator.
Multiply prime1 by prime2 using Hpmbcalc Multiplication operator.
Copy Operation Result to First Operand using Hpmbcalc r->x operator.
Copy variable modulus to Hpmbcalc's Second Operand.
Subtract Second Operand from First Operand.
Check if the operation result equals to 0.

4. Verify private exponent

Private exponent, often known as "d", should be satisfied with:

f = (p-1)(q-1)
ed = 1 (mod f).

The identifiers p, q, e, d are defined on the Table 1.

To verify the private exponent contained by the sample private key blob, firstly, let us calculate: f = (p-1)*(q-1):

Copy variable p to Hpmbcalc's First Operand, and variable q to Second Operand.
Reverse p and q using Hpmbcalc Reverse Operand operator.
Calculate (p-1) and (q-1) using Hpmbcalc Decrement operator.
Multiply (p-1) by (q-1) using Hpmbcalc Multiplication operator to get f.
Save f to memory using Hpmbcalc MS operator.

Next, let us calculate: e*d:

Copy variable e to Hpmbcalc's First Operand, and variable d to Second Operand.
Reverse e and d using Hpmbcalc Reverse Operand operator.
Multiply e by d using Hpmbcalc Multiplication operator.
Copy Operation Result to First Operand using Hpmbcalc r->x operator.

Last, let us verify the equation: e*d = 1 (mod f):

Restore f from memory to Second Operand using Hpmbcalc MR operator.
Calculate e*d (mod) f using Hpmbcalc Mod operator.
Check if the operation result equals to 1.

5. Verify coefficient

The variable coefficient should have a numeric value of "(inverse of q) mod p" (p, q are defined on the Table 1).

Copy variable q to Hpmbcalc's First Operand, and variable p to Second Operand.
Calculate "(inverse of q) mod p" using Hpmbcalc m^-1 operator.
Copy Operation Result to First Operand using Hpmbcalc r->x operator.
Copy variable coefficient to Hpmbcalc's Second Operand.
Subtract Second Operand from First Operand.
Check if the operation result equals to 0.

6. Others

Similarly, you can verify if the variables exponent1 and exponent2 satisfy with:

exponent1 = d mod (p - 1);
exponent2 = d mod (q - 1);

The identifiers p, q, d are defined on the Table 1.

Where to go from here?

Thus, to understand a binary file format, you need mainly the following knowledges, techniques and tools:

You should obtain as much information about the binary file format that you want to master as possible. You should search the internet to see what is already known about the format, or to find out if someone already offers some conversion tool to the format. One place to look at, is Wotsit, a site dedicated to file formats.
You need a hex editor with some advanced features for analyzing binary files. Hexprobe allows you to write templates for describing the structure of the binary file, and the familiar JScript languages is used. Thus, if you choose Hexprobe as your hex-prober, you need:
- Download and try Hexprobe.
- Learn how to write Hexprobe document templates.
- Take advantage of the some books and classes to master the JScript language.
You may need some other hex tools to help your works on the binary files, such as:
- A hex calculator like Hpmbcalc for you to perform big number math operations and hex, dec, oct, bin conversion.
- A hex compare tool for you to compare two binary files byte by byte, Hexprobe provides such a tool.
- A hex find/replace tool for you to search/replace certain bytes in the binary file, Hexprobe provides such a tool.