Registry hive basics part 4: SK records

February 2, 2015, 8:30 pm

≫ Next: XWFIM and XWFRT v1.0 released!

$

0

0

In part 3, we covered VK records. Next up is the SK record, or Security Key

SK record

An SK (security key) record contains the information necessary to define access controls for the Registry. An example of an SK record as it exists on disk is shown below.

In total, there are up to 17 pieces of information in an SK record. The SK record is a bit stranger (and far more complicated) than the other records we have seen before in that it is somewhat nested. The more interesting pieces of information include:

• Size
• Signature
• Flink and Blink
• Reference count
• Descriptor
• Control
• Owner offset
• Group offset
• SACL (Security ACL, used for auditing purposes) offset
• DACL (Discretionary ACL, used for controlling access) offset
• SACL
• DACL
• Owner SID (Security IDentifier)
• Group SID

Size

The size starts at offset 0x00 and is stored as signed 32-bit integer. In the example above, the hex value is 0x28FFFFFF (stored in little endian format). Converting the value to decimal results in -216 (negative 216).

The size works the same way in SK records as it did in the NK and VK records, so our actual record size is 216 bytes.

Signature

The signature for an SK record is found at offset 0x04. The signature for an SK record is the ASCII string "sk" which is 0x736B in hex.

Flink and Blink

Flink (Forward link) starts at offset 0x08 and is stored as a 32-bit unsigned integer. Flink is the offset to the next SK record in the hive.

Blink (Backward link) starts at offset 0x0C and is stored as a 32-bit unsigned integer. Blink is the offset to the previous SK record in the hive.

In the example above, the Flink is 0x10C30000 and the Blink is 0xF0C30800. As with other offsets, these are relative offsets, so we would have to add 0x1000 to get to their actual position on disk.

Flink and Blink are used to build the "chain" of SK records in a hive.

Reference count

The Reference count starts at offset 0x10 and is stored as a 32-bit unsigned integer. Reference count reflects the total number of references from NK records a particular SK record has.

In the example above, Reference count is 0x010000, or 1 in decimal. Recall from our discussion on NK records that they contain a Security Key offset. The NK record's Security Key offset would point to an SK record, and for each one doing so, Reference count would be increased by 1.

Descriptor

The Descriptor starts at offset 0x18. The length of the Descriptor data starts at offset 0x14 and is stored as a 32-bit unsigned integer. 

In the example above, the Descriptor length is is 0xBC000000, or 188 in decimal. If we start at offset 0x18 and sweep 188 bytes, we get the following:

Before we jump into the Descriptor, notice how we again see some extra stuff after the Descriptor bytes. This is padding. The same calculations we saw before can be used to verify the SK record is a multiple of 8:

20 (bytes before Descriptor data) + 188 (Descriptor data) + 8 (Padding) == 216 which is what our Size reflects.

Descriptor breakdown

After determining if this record does in fact have Descriptor data (Descriptor length is greater than zero), we can parse out the parts that make up the Descriptor.

As mentioned above, the Descriptor has the following parts we are interested in:

• Control
• Owner offset
• Group offset
• SACL offset
• DACL offset
• SACL
• DACL
• Owner SID
• Group SID

Control

The Control starts at offset 0x1A and is stored as a 16-bit unsigned integer. In the example above, Control is 0x1498, or 0x9814 once we convert it from little endian.

Control is a flag that has the following values:

• SeDaclAutoInherited = 0x0400
• SeDaclAutoInheritReq = 0x0100
• SeDaclDefaulted = 0x0008
• SeDaclPresent = 0x0004
• SeDaclProtected = 0x1000
• SeGroupDefaulted = 0x0002
• SeOwnerDefaulted = 0x0001
• SeRmControlValid = 0x4000
• SeSaclAutoInherited = 0x0800
• SeSaclAutoInheritReq = 0x0200
• SeSaclDefaulted = 0x0020
• SeSaclPresent = 0x0010
• SeSaclProtected = 0x2000
• SeSelfRelative = 0x8000

In this particular instance, Control is equivalent to the following:

 SeDaclPresent | SeDaclProtected | SeSaclAutoInherited | SeSaclPresent | SeSelfRelative

These flag values can be calculated exactly the same as we have previously seen, so we won't repeat that here.

Control will be used later and we will check it to know whether or not to try to parse a SACL or DACL structure from our bytes.

Owner offset

The Owner offset starts at offset 0x1C and is stored as a 32-bit unsigned integer. In the example above, Owner offset is 0xA0000000. 

After we discuss the Group, SACL and DACL offset below we will see exactly what lives in these offsets.

Group offset

The Owner offset starts at offset 0x20 and is stored as a 32-bit unsigned integer. In the example above, Owner offset is 0xB0000000. 

The Owner offset and Group offset hold the relative offsets inside the Descriptor data to where the Owner and Group SIDs (Security IDentifier) can be found. Before we can get that data tho, we need a few more pieces of information.

SACL offset

The SACL offset starts at offset 0x24 and is stored as a 32-bit unsigned integer. In the example above, Owner offset is 0x14000000. 

DACL offset

The DACL offset starts at offset 0x28 and is stored as a 32-bit unsigned integer. In the example above, Owner offset is 0x1C000000. 

Now that we have all of our offsets to the relevant data structures, we can determine the length of each of these structures as follows:

• SACL size == DACL offset - SACL offset, or 0x08
• DACL size == Owner offset - DACL offset, or 0x84
• Owner SID size == Group offset - Owner offset, or 0x10
• Group SID size ==Descriptor length - Group offset, or 0x0C

We now know the length of each of the structures in bytes and can grab those bytes and parse them out.

In our example above, the raw SACL bytes would be:

02-00-08-00-00-00-00-00

and the raw DACL bytes would be:

02-00-84-00-05-00-00-00-00-03-24-00-3F-00-0F-00-01-05-00-00-00-00-00-05-15-00-00-00-82-F6-13-90-30-42-81-99-23-04-C3-8F-51-04-00-00-00-03-14-00-3F-00-0F-00-01-01-00-00-00-00-00-05-12-00-00-00-00-03-18-00-3F-00-0F-00-01-02-00-00-00-00-00-05-20-00-00-00-20-02-00-00-00-03-14-00-19-00-02-00-01-01-00-00-00-00-00-05-0C-00-00-00-00-00-18-00-19-00-02-00-01-02-00-00-00-00-00-0F-02-00-00-00-01-00-00-00

In our example above, the raw Owner SID bytes would be:

01-02-00-00-00-00-00-05-20-00-00-00-20-02-00-00

and the raw Group SID bytes would be:

01-01-00-00-00-00-00-05-12-00-00-00

Decoding raw SID bytes to a more common format

To decode the raw bytes to a format we are more used to seeing, we have to do some manipulation of the bytes as described here. If you are curious as to exactly how to do this, please visit that link. 

When we convert our raw bytes for Owner and Group SID, we get the following:

• Owner SID: S-1-5-32-544 
• Group SID: S-1-5-18

Seeing a SID in that format is less than helpful, so it would be nice if we could convert those SIDs into something that means a little more, such as whether or not the SID is used for a service account, the users group, an anonymous logon group, and so on.

My Registry parsing code has a large collection of lookup values that can be used to cross reference the SID to a description. Of course this all happens under the covers for you when using the parser, but if we look up those SIDs manually (via the link or via Google, etc), we get the following descriptions:

• S-1-5-32-544  is equivalent to Built in Administrators
• S-1-5-18 is equivalent to Local System

This certainly makes life easier than memorizing or constantly looking up a bunch of SIDs for various keys in a hive.

Decoding raw SACL and DACL bytes

Finally we get to the last pieces of the horrible nastiness that is SK records.

Recall earlier to what our Control flag value was. Since it contains SeDaclPresent and SeSaclPresent we know we can parse out those structures.

The format for both SACL and DACL structures is the same. To simplify things a bit, lets refer to the common structure as an ACL. The more interesting fields in an ACL are:

• Size
• ACE (Access Control Entry) count
• ACE records

The ACL breaks down as follows:

• Size starts at offset 0x02 and is stored as a 16-bit unsigned integer.
• ACE count starts at offset 0x04 and is stored as a 16-bit unsigned integer.
• The ACE records start at offset 0x08.

Since our DACL was the bigger of the two structures, lets process that one by hand.

Recall the raw bytes for the DACL were:

02-00-84-00-05-00-00-00-00-03-24-00-3F-00-0F-00-01-05-00-00-00-00-00-05-15-00-00-00-82-F6-13-90-30-42-81-99-23-04-C3-8F-51-04-00-00-00-03-14-00-3F-00-0F-00-01-01-00-00-00-00-00-05-12-00-00-00-00-03-18-00-3F-00-0F-00-01-02-00-00-00-00-00-05-20-00-00-00-20-02-00-00-00-03-14-00-19-00-02-00-01-01-00-00-00-00-00-05-0C-00-00-00-00-00-18-00-19-00-02-00-01-02-00-00-00-00-00-0F-02-00-00-00-01-00-00-00

Following our template we can determine the following:

Size = 0x8400, or 132 in decimal
ACE count = 0x0500, or 5 in decimal

This leaves the following bytes which contains our ACE records:

03-24-00-3F-00-0F-00-01-05-00-00-00-00-00-05-15-00-00-00-82-F6-13-90-30-42-81-99-23-04-C3-8F-51-04-00-00-00-03-14-00-3F-00-0F-00-01-01-00-00-00-00-00-05-12-00-00-00-00-03-18-00-3F-00-0F-00-01-02-00-00-00-00-00-05-20-00-00-00-20-02-00-00-00-03-14-00-19-00-02-00-01-01-00-00-00-00-00-05-0C-00-00-00-00-00-18-00-19-00-02-00-01-02-00-00-00-00-00-0F-02-00-00-00-01-00-00-00

An ACE record looks like this:

• Ace type: 1 byte long starting at offset 0x00
• Ace flags: 1 byte long starting at offset 0x01
• Ace Size: 16-bit unsigned integer starting at offset 0x02
• Mask: 32-bit unsigned integer starting at offset 0x04
• SID: Starting at offset 0x08

Based on what we saw above, the ACE count is five, so we would expect there to be enough bytes to find five ACE records from the bytes above.

If we follow our template, we get the following five chunks of bytes:

1. 00-03-24-00-3F-00-0F-00-01-05-00-00-00-00-00-05-15-00-00-00-82-F6-13-90-30-42-81-99-23-04-C3-8F-51-04-00-00
2. 00-03-14-00-3F-00-0F-00-01-01-00-00-00-00-00-05-12-00-00-00
3. 00-03-18-00-3F-00-0F-00-01-02-00-00-00-00-00-05-20-00-00-00-20-02-00-00
4. 00-03-14-00-19-00-02-00-01-01-00-00-00-00-00-05-0C-00-00-00
5. 00-00-18-00-19-00-02-00-01-02-00-00-00-00-00-0F-02-00-00-00-01-00-00-00

For each of these chunks of bytes, we can build an ACE record. If we take the first chunk and process it, we get the following:

• Ace type: 0x00
• Ace flags: 0x03
• Ace Size: 0x2400, or 24 in decimal
• Mask: 0x3F000F
• SID: 01-05-00-00-00-00-00-05-15-00-00-00-82-F6-13-90-30-42-81-99-23-04-C3-8F-51-04-00-00

Ace type is equivalent to:

 AccessAllowedAceType

Ace flags is equivalent to:

 ContainerInheritAce | ObjectInheritAce

Mask is a flag and is the equivalent to:

FullControl

When the raw SID is converted we get S-1-5-21-2417227394-2575385136-2411922467-1105. This corresponds to a user SID which could be looked up accordingly in another hive.

Now we have to repeat that for the last four ACE records and we have completed one SK record!

If you made it this far, congratulations! I told you SK records were complicated!

Putting it all (back) together

Let's take our example and break it down into the most important parts.

For a full breakdown of the SK record layout and all its parts, review the documents posted earlier.

If we look at the ToString() output from my Registry parser, we get this:

Size: 0xD8

Relative Offset: 0xB0

Absolute Offset: 0x10B0

Signature: sk

Is Free: False

Forward Link: 0xC310

Backward Link: 0x8C3F0

Reference Count: 1

Security descriptor length: 0xBC

Security descriptor: Revision: 0x1

Control: SeDaclPresent, SeSaclPresent, SeSaclAutoInherited, SeDaclProtected, SeSelfRelative

Owner offset: 0xA0

Owner SID: S-1-5-32-544

Owner SID Type: BuiltinAdministrators

Group offset: 0xB0

Group SID: S-1-5-18

Group SID Type: LocalSystem

Dacl Offset: 0x1C

DACL: ACL Size: 0x2

ACL Type: Discretionary

ACE Records Count: 5

------------ Ace record #0 ------------

ACE Size: 0x24

ACE Type: AccessAllowedAceType

ACE Flags: ObjectInheritAce, ContainerInheritAce

Mask: FullControl

SID: S-1-5-21-2417227394-2575385136-2411922467-1105

SID Type: UnknownOrUserSID

SID Type Description: SID does not map to a common SID or this is a user SID

------------ Ace record #1 ------------

ACE Size:

 0x14

ACE Type: AccessAllowedAceType

ACE Flags: ObjectInheritAce, ContainerInheritAce

Mask: FullControl

SID: S-1-5-18

SID Type: LocalSystem

SID Type Description: S-1-5-18: An account that is used by the operating system.

------------ Ace record #2 ------------

ACE Size: 0x18

ACE Type: AccessAllowedAceType

ACE Flags: ObjectInheritAce, ContainerInheritAce

Mask: FullControl

SID: S-1-5-32-544

SID Type: BuiltinAdministrators

SID Type Description: S-1-5-32-544: A built-in group. After the initial installation of the operating system, the only member of the group is the Administrator account. When a computer joins a domain, the Domain Administrators group is added to the Administrators group. When a server becomes a domain controller, the Enterprise Administrators group also is added to the Administrators group.

------------ Ace record #3 ------------

ACE Size: 0x14

ACE Type: AccessAllowedAceType

ACE Flags: ObjectInheritAce, ContainerInheritAce

Mask: Qu

eryValue, EnumerateSubkeys, Notify, ReadControl

SID: S-1-5-12

SID Type: RestrictedCode

SID Type Description: S-1-5-12: This SID is used to control access by untrusted code. ACL validation against tokens with RC consists of two checks, one against the token's normal list of SIDs and one against a second list (typically containing RC - the RESTRICTED_CODE token - and a subset of the original token SIDs). Access is granted only if a token passes both tests. Any ACL that specifies RC must also specify WD - the EVERYONE token. When RC is paired with WD in an ACL, a superset of EVERYONE, including untrusted code, is described.

------------ Ace record #4 ------------

ACE Size: 0x18

ACE Type: AccessAllowedAceType

ACE Flags: None

Mask: QueryValue, EnumerateSubkeys, Notify, ReadControl

SID: S-1-15-2-1

SID Type: AllAppPackages

SID Type Description: S-1-15-2-1: All applications running in an app package context.

Sacl Offset: 0x14

SACL: ACL Size: 0x2

ACL Type: Security

ACE Records Count: 0

Registry hive basics part 5 will cover the various list records in the Registry.

---

XWFIM and XWFRT v1.0 released!

February 4, 2015, 7:04 am

≫ Next: ShellBags Explorer v0.5.2.0 released!

≪ Previous: Registry hive basics part 4: SK records

$

0

0

XWFIM and XWFRT version 1.0 now available!

XWFIM Changes include:

NEW: Full proxy support via Tools menu
NEW: Support for BYOD downloads
CHANGE: You can now make a portable installation on any device vs removable only
FIX: Detect when 'Append version #' option is active when creating portable install

You can download the update here or just start XWFIM to get the latest version.

BYOD

You don't have to do anything differently for BYOD downloads. Just enter the username and password you received for the BYOD download and XWFIM will detect it and do the rest

Proxy support

You can access Proxy settings via the Tools menu. XWFIM has been tested by myself and several users. I tested it against a Squid proxy without issue.

XWFRT Changes include:

CHANGE: Fix parsing of v18 report files

Note: Only the latest version of X-Ways Forensics is supported!

You can download the update here or just start XWFRT to get the latest version.

ShellBags Explorer v0.5.2.0 released!

February 4, 2015, 2:22 pm

≫ Next: Exploring the Registry at the hex level

≪ Previous: XWFIM and XWFRT v1.0 released!

$

0

0

Changes include:

NEW: Detect deleted ShellBags. When deleted ShellBags are found they will have a red X after the name of the bag. Deleted ShellBags will also be denoted in the Miscellaneous column
NEW: Added GUID for Network Neighborhood

CHANGE: Expanded coverage for MTP type 2 devices (a lot of new detail available in Details view)
CHANGE: Expanded coverage for MTP type 1 devices (a lot of new detail available in Details view)

FIX: Cosmetic clean up in SBECmd output (showing results twice)

You can get it here or by starting ShellBags Explorer and using the update functionality.

Expanded MTP coverage

MTP type 1 and 2 improvements include showing details on GUIDs, file system name, ClassID, and Storage ID name.

Type 2:

Type 1:

Deleted ShellBags

Registry Explorer has the ability to recover and display deleted keys and values. Looking at a hive with deleted ShellBags looks like this:

Since Registry Explorer and SBE both use the same back end to parse hives, SBE now has the ability to detect deleted ShellBag entries. When they are found, a red X will be displayed to the right of the name in the tree and the Details view will also note the fact that the ShellBag was deleted.

Here we see the grid reflects the deleted ShellBags as well.

Exploring the Registry at the hex level

February 10, 2015, 1:30 pm

≫ Next: Introducing Registry Explorer

≪ Previous: ShellBags Explorer v0.5.2.0 released!

$

0

0

If you want to open your hives and browse the structure you can of course use any hex editor. There are several hex editors out there that let you define templates to parse data. One of these hex editors is 010 Editor.

010 has a C/C++ like syntax for defining templates. I wrote a template for Registry hives that you can get on the Template Archive (at some point), but it will also always be available here.

The template currently supports all the major structures found in hives including NK, VK. SK. and lists (LH, LF, and LI).

How to use it

After saving the template to your computer, add it to 010 editor via the Templates | Edit Templates List option.

Click New, change the name to "Registry hive", then browse to the template you saved above. Once that is done click OK.

Open a hive in 010 and then select Templates | Registry hive from the list.

Here is an example of what you would see before applying the template:

And after applying the template.

As you browse the data at the bottom the relevant bytes are selected in the hex view. In the image below I selected the Descriptor struct and all the bytes that make up that struct are selected. If I select a member of a struct, the bytes belonging to that member are selected.

From here you can explore at will and verify hives by hand (just be sure to add 0x1000 to any offsets to get to the proper position in the file).

Enjoy!

Introducing Registry Explorer

February 11, 2015, 12:32 pm

≫ Next: ShellBags Explorer 0.5.4.0 released!

≪ Previous: Exploring the Registry at the hex level

$

0

0

Registry Explorer is a new approach to interacting with Registry hives. It has several unique capabilities not found in other programs.

The version being discussed here is 0.0.1.8 (some of the screenshots may reflect v0.0.1.7). It is an early beta version of RE compared to where it will be but the features discussed below are ready for general use.

Known issues:

• Large hives (SOFTWARE hives especially) can be slow to load in RE due to the number of keys that exist (often hundreds of thousands). Give RE a few minutes to load them tho and it will work. This is being addressed.

Getting started

To get started, lets take a look at the main interface.

There is of course a standard menu bar at the top of the screen. Most of the options are self-explanatory, but lets look at a few of them.

View | Messages

The Messages window is automatically displayed when RE starts. It will contain information about loading hives, warnings, errors, etc. If you have used ShellBags Explorer, it looks and feels pretty much the same as in SBE.

Options menu

We will drill down into these options in detail below, but at a high level, the items in the Options menu work as follows:

• Recover deleted keys/values: When enabled, RE will search for and reassociate deleted Registry keys and values from a hive.
• Show hidden keys: When enabled, any keys that have been hidden will be visible in the hive (and a different icon will be used to denote the key is hidden)
• Manage hidden keys: Brings up an interface to delete any keys that have been added to the auto hide list

Bookmarks

The Bookmarks menu contains two submenus: Common and User created. Common bookmarks are those included in RE and User created are those created by an end user of RE. We will discuss bookmarks in detail below.

There is also an option for managing user created bookmarks which will be included in the more detailed bookmark discussion below.

Loading hives

There are two ways to load Registry hives in RE. The easiest is to simply drag and drop hives on the left side of the main interface. This method allows for loading several hives at once. The File menu can also be used to load hives. 

Once one or more hives are loaded, they will be seen on the left side of RE.

To expand nodes, click the arrow to the left of the folder for a hive or simply double click on a node.

In the first hive that is expanded (1UsrClass.dat) we see the root node (S-1-5-21-1141529136-2431258765-826847743-1000_Classes), the number of values present in that key, and the last write timestamp of the key. Of course the status bar will also update to show this information.

In the last two hives we see a few additional child objects of the hive. Since the Recover deleted keys/values was enabled when we loaded the hives, RE recovered keys and values.

Registry Explorer and deleted keys/values

RE handles deleted keys and values differently than other viewers. In the image above we can see there is a node for Associated deleted records and Unassociated deleted records. The difference between the two is that the associated records are associated with keys in the active registry (ie keys that are not deleted).  The unassociated records are all of the keys and values that could not be associated with an active key. Deleted values that could not be associated with a key are not displayed (but could be. If this interests you, please let me know).

Associated deleted records

If we expand the Associated deleted records node a few times, we see the following.

The green Registry icon means that the path exists in the active Registry and as such, the same path will be found under the root node above. If we find the same key in the active Registry, it looks like this:

As you can see, the associated deleted keys are available in the active tree as well as Associated deleted records. The green icons in Associated deleted records correspond to the paths that exist in the active Registry. This tells you where you can find the deleted keys in the active Registry.

In both places, deleted keys have a different icon vs. an active key.

You can also see the number of values and the last write timestamps of each of the keys now that we have some rows expanded.

Unassociated deleted records

In the image above Unassociated deleted records does not have the indicator showing there are subkeys. This is because for this hive there are no unassociated deleted records (in the version you can get now, this will not be there if there are no unassociated records to display).

When there are unassociated records, they would be displayed just as we saw with Associated deleted records.

Viewing key values

For both active and deleted keys, selecting a key on the left results in the values for that key being displayed in the upper right.

In the Values grid we see the value name, type, data, and any value slack.

Clicking on a value will display the details of that value in the viewer area below the values grid. The type viewer will dynamically change depending on the Value type of the selected value.

Here we see a RegBinary value selected:

And here we see a RegDWord value selected:

In upcoming versions, the viewers available will be expanded and improved to include dynamically generating viewers based on plugins available for Registry keys, displaying value slack in a hex editor (as well as decoding common items such as strings, etc), and so on.

Bookmarks

The bookmarking capabilities of RE makes it easy to see exactly what keys of forensic interest are available in a given hive. Bookmarks are available in the menu at the top of RE.

Bookmarks are specific to certain Registry hive types such as ntuser, usrclass, software, etc. Depending on what kind of hive is selected in the tree, the available bookmarks will dynamically update to only show bookmarks available for that kind of hive.

In the example below, notice the NTUser.dat hive is selected (based on the black triangle to the left of the name). After selecting this hive (or any subkey in it), the Bookmarks menu adjusts to show:

• In the main menu, the total number of bookmarks available (3 in this case)
• In the submenus, the total number of Common and User created bookmarks available

In this example, usrClass.dat is selected. Since this is a different hive type, the ntuser bookmarks are not even shown.

This lets you quickly see how many bookmarks are available in a given hive without clicking on a bunch of bookmarks to see if a particular key is present. This can save a lot of time, especially with more esoteric keys related to malware, etc.

But thats not all! After bookmarks are filtered by hive type, RE goes a step further and checks to see if the path to the Registry key for a bookmark actually exists in the hive. This further reduces needless clicking on bookmarks that point to non-existent keys in a hive.

Creating bookmarks

To create bookmarks, load a hive and find a key you are interested in. Right click on the key and select Add bookmark.

The Bookmark editor is now displayed. Since RE knows what kind of hive this is, it has prepopulated the Hive type (NTUSER in this case). Next, select a Category from the dropdown. To add a new one, simply enter a new one and it will be saved. The initial list of categories was borrowed from @corey_harrell and his Auto_Rip script.

Once a category is selected, enter short and long descriptions. Short description will be used in the Bookmarks menu to give you some context as to what the key means. The long description will be used in reports so others will have some background on what the key is used for, why its important, and other links. A completed bookmark will look like this:

Click OK to save the bookmark. 

The Bookmarks menu will be immediately updated showing the availability of the new bookmark.

Managing bookmarks

The Bookmarks menu also has an option to manage user created bookmarks. 

The bold columns indicate read-only values. All other columns can be edited  by clicking on the cell you wish to edit. In the image above, I am editing the long description column.

Bookmarks are saved on closing the Bookmark Manager.

Sharing bookmarks

All bookmarks (and for that matter, all settings) in RE are simple json files. Bookmarks live in a subdirectory of where the RE executable is called Bookmarks. Under that directory are two more directories, one for Common and one for User bookmarks.

Here we see the bookmark we just created. On disk, bookmarks are named using the following method:

• Category_HiveType_Name_InternalGUID

The internal GUID is automatically assigned by RE and is simply used to guarantee a unique name exists when saving bookmarks.

If we look at the bookmark in a text editor, we see a very simple json formatted file:

Using a common and well understood format like json to store bookmarks allows for scripting of existing Registry key repositories to RE bookmarks as well as editing said bookmarks using whatever other means people want to come up with.

Notice in the json file the KeyPath points to the key below the root key in a hive (vs. including the root hive name). This is because root keys, depending on which hive you are talking about, are all named differently. When creating bookmarks outside RE, keep this in mind.

There is no difference between a Common and User bookmark other than the directory the bookmark lives in. The reason for a Common folder and a User folder is so that any User created bookmarks will not be overwritten on version updates (although this is highly unlikely since RE uses a GUID for uniqueness).

It is my hope the community will contribute bookmarks to the RE project. To help facilitate this, I have set up a Github repository to push bookmarks into. These bookmarks will become the Common bookmarks in RE as time goes on. You can find the repository here.

Using bookmarks

To use a bookmark, simply click it and the key associated with the bookmark will be instantly selected in the tree. If you have multiple hives of the same type open, the key in the currently selected hive will be selected.

Context menu on keys

In the bookmark discussion, we saw the context menu that is available for keys. The context menu changes depending on what is selected in the tree. If a hive is selected, the following options are available:

Since some context options do not make sense depending on what is selected, certain options get disabled (bookmarking, hiding, etc. in this case)

If we select a key (other than the root key), more options become available:

The context menu options are as follows:

• Key: <Key name>: This is the name of the key that was selected when the context menu was invoked.
• Remove hive: Only available for hives. When selected, you will be prompted to remove the hive from RE.
• Add bookmark: Creates a new bookmark for the selected key
• Hide key
• For this session only: Temporarily hides the selected key (and all subkeys) until RE is restarted
• Hide and add to autohide: Hides the selected key (and all subkeys) which persist between RE being restarted
• Copy
• Key name: Copies only the key name to the clipboard
• Key path: Copies full path to the clipboard
• Last write time: Copies last write time to the clipboard
• Expand child nodes: Recursively expands the selected key's subkeys
• Collapse child nodes: Collapses the selected key's subkeys

You can also copy the Key Path and Last write time by double clicking on the relevant section of the status bar.

Finally, selecting a key and pressing CTRL-C copies details about that key to the clipboard

Key name# valuesLast write timestamp
eMule39/2/2014 7:56:39 PM +00:00

Hiding keys (for fun and profit)

The Registry is a busy place with a lot of distractions. To help alleviate this problem, RE can hide keys so you do not have to look at them when doing your work.

To hide a key, select one of the two context menu items according to your needs.

As an example, take the screen shot below as a starting point.

As you can see, this is an NTUSER hive. There is a lot of good information in there, but if I were to expand all of the keys and try to navigate to the keys I am interested in there would be a lot of keys in the way that are not of interest to me. 

In this case, lets assume the Printers key (CsiTool-CreateHive-{00000000-0000-0000-0000-000000000000}\Printers) and the MediaProperties key (CsiTool-CreateHive-{00000000-0000-0000-0000-000000000000}\System\CurrentControlSet\Control\MediaProperties) are of no interest to me.

If I right click on MediaProperties and select Hide key | For this session only the key (and all subkeys) will be immediately hidden from the tree.

For the Printers key, I will select Hide key | Hide and add to autohide. This also results in the key being immediately hidden.

Our tree now looks like this:

We just cleaned up some of the noise that distracts from the core keys we are interested in. 

At some point you may want to view the keys you have hidden. To do this, simply select Options | Show hidden keys from the menu at the top of RE. When this option is enabled, our tree looks like this:

Notice how the hidden keys have a different icon (a bigger example of that icon is shown to the right). This lets you easily differentiate hidden and non-hidden keys when Show hidden keys is enabled. To hide the keys, simply disable the option.

As mentioned above, any keys hidden for the current session will be shown normally after RE is started and any keys added to the auto list will always be hidden. To manage the auto hidden keys, select Options | Manage hidden keys and a dialog will be displayed allowing you to remove auto hidden keys. In this example I added a few more keys to the auto hide list.

So to wrap up, if we hide keys and then show them via the Options menu, the interface would change. Here is a separate example using different hidden keys to illustrate the point further. Here, the Appearance, Printers and WAB keys have been hidden

Other features to try

• To quickly select a key in a hive, select a hive and just start typing the key name. For example, if loading a UsrClass hive, start typing 'BAGMRU' and RE will start expanding the keys and selecting key that matches what you are typing.
• With a key selected in the tree, press CTRL-F to bring up a search interface. You can then enter a value for Re to find in VISIBLE key names. This does not auto expand things but if you recursively expand a hive and then use find you can narrow down the keys you are looking at in no time.
• Filtering on key names is also a powerful way to narrow down your focus. You can enter a key name to filter on, enter a minimum number of values, or any other criteria.

That does it for now. Please give it a try and let me know what else you want to see it do!

If you have an earlier version of RE, it should auto update on startup. If not, you can always get the latest version here.

ShellBags Explorer 0.5.4.0 released!

February 17, 2015, 1:30 pm

≫ Next: Registry Explorer 0.0.2.0 released!

≪ Previous: Introducing Registry Explorer

$

0

0

This release contains the following changes:

CHANGE: Replace custom logging code with NLog
FIX: A few cosmetic fixes to ShellBag details output
FIX: Registry parser bug fixes on some fringe cases

The biggest change for this version is swapping out old custom logging stuff with NLog. This makes things much more flexible as any of the NLog targets can now be used for logging. Future releases will possibly support using an NLog.config file so end users can control what gets logged and to where (files, databases, email, etc).

ShellBags Explorer is also using the new logging under the hood, but it will look and feel pretty much the same since all output goes to the Messages window.

Here is what SBECmd.exe looks like now:

Registry Explorer 0.0.2.0 released!

March 6, 2015, 6:22 am

≫ Next: The end of XWFRT?

≪ Previous: ShellBags Explorer 0.5.4.0 released!

$

0

0

Lots of good changes in this version including:

Known issues:
- Large hives (SOFTWARE 100+ Mb) can be slow to load the tree due to the number of data in these hives (100s of thousands of keys and values). They will load thought!

NEW: Added new tab in upper left, Available bookmarks, that shows all available bookmarks across all loaded Registry hives
NEW: Added 'Technical details' option to context menu. Use this to view all the down and dirty details about a key including its bytes, its security key, subkeys, values, etc. This provides an easy to use way to explore and validate Registry tools
NEW: Added several hotkeys for commonly used key context menu items
NEW: Allow exporting of keys either individually or recursively to .reg format via the context menu
NEW: Add 'Collapse all hives' button to status bar. 
NEW: Added more bookmarks

CHANGE: Prevent illegal file name characters in category names (\, /, |, and so on). Any illegal characters will be replaced with an underscore
CHANGE: Nlog logging added
CHANGE: Registry parsing is now ~150% faster and memory usage reduced by 40-80%
CHANGE: Prevent the same hive from being loaded more than once
CHANGE: Expand the top level node after loading a hive
CHANGE: Hide or unhide all matching keys in all open hives vs only the active hive

FIX: When removing keys from auto hide list, remove any hidden keys in the tree that match as well
FIX: Actually export the timestamp when exporting Messages
FIX: GUI polish

General changes

The first thing to notice about this version is SPEED! The back end Registry parser has undergone significant performance tuning as well as memory usage optimization. The result is much faster loading of hives.

Several of the most used context menu items were given shortcut keys.

An option to collapse all opened Registry hives was added to the lower right corner. This is useful to reset things if many hives/keys are expanded vs scrolling around all over the place. Clicking this button resets the Registry hives tree to what it looked like after initially loading hives.

The Quick help menu now works and a Legend menu item has been added to the Help menu.

This version also adds a lot of GUI polish, new bookmarks, and general fixes.

Changes to hiding and unhiding keys

The last change to point out is that hiding a key in one hive results in that key being hidden in all other open hives where that key exists.

In the screenshot below, i have hidden several keys (MuiCache and VirtualStore). Note the Windows key in each of the hives (highlighted in green). The Options | Show hidden keys option is on so we can see currently hidden keys (the keys that have the little red mark on them).

In the screen shot below, The key indicated was selected and the hide key option was selected for the key.

After selecting a hide option, notice how ALL of the 'Local Settings\Software\Microsoft\Windows' keys are now marked hidden

Unhiding keys works the same way in that when you unhide a key in a hive, any other keys that match that key are also unhidden.

Finally, in the lower right corner is an indicator of how many keys are hidden.

Available bookmarks

This is a new tab that lives next to the original tree in previous versions. Here we see where the new tab is located. The "Registry hives" tab works exactly as the tree did in previous versions.

Before showing what the "Available bookmarks" tab looks like and how it works, lets talk about bookmarks in general again. Recall from the last post that the Bookmarks menu contains a list of bookmarks that a) exist for the currently selected hive type and b) the hive actually contains the key reflected by the bookmark. 

This is all well and good for quickly moving around a Registry hive, but there can be a lot of noise in the way in the Registry. The Available bookmarks tab seeks to make things easier by showing you the keys that relate to all available bookmarks for the currently loaded hives.

At first, Available bookmarks will be empty, like this:

But if we load a few hives, say a UsrClass and NTUser hive, things change. The Registry hive tab will show the hives in their entirety.

But if we now look at the Available bookmarks tab, we see something very different.

The first thing to notice is the numbers at the end. The first is the total number of Common bookmarks that were found across all hives. The second is the total number of User bookmarks that were found across all hives. These numbers will dynamically update as new bookmarks are added or hives are loaded.

Here we see each of our loaded hives. but only the bookmarks that exist in each hive show up. The rest of the keys are not shown at all (but subkeys of bookmarked keys are).

Of course you can click on any of those keys and view all of the values, etc.

As you click on any of the bookmarked keys (or subkeys), the Bookmark information will be updated to show the details related to the active bookmark. This lets you drill down into the key areas of all loaded Registry hives in a succinct and easy to use way.

You can also sort and filter on the available bookmarks tab, just like the main Registry hives tab.

Technical details

This feature bridges the gap between using a hex editor and blindly trusting Registry Explorer (or any tool for that matter) when it comes to reviewing Registry hives.

The idea is to be able to browse Registry data structures for a given key and see all the gory details in an easy to use way that also allows for exploring up or down from a given key. In other words, you can browse to a key's parent or any of its subkeys, explore values, security keys, hive details, and so on. 

Let's look at some examples.

To view technical details, select a key and invoke the appropriate option from the context menu, or press F5.

Once that is done, the technical details form will be displayed.

At the top of the form are tabs that convey information about the selected key.

• NK record: Shows all the details about the key itself
• Values: Contains a list of values. Selecting a value will display the associated VK record
• Subkeys: Contains a list of subkeys. Double clicking a subkey will bring up a Technical details report for that key
• SK record: Shows all the details about the key's security key
• Full details as text: All the details about this key in text format
• Hive details: Metadata about the hive file itself

If a key doesn't have subkeys, values, or an SK record (in the case of a free NK record), those tabs will not be displayed. In the example below, the Values tab is hidden because there are no values to show.

As you click around on the properties on any of the tabs. the hex editor will highlight the bytes that make up the selected property. If nothing is selected for a property, the property value comes from elsewhere (such as the relative offset, whether or not its free, etc). 

Here the Name property is selected. Notice how the hex editor has highlighted the bytes that make up the Name property. The number of bytes selected and the offset are also displayed in the lower right.

On the values tab, all values for the key are listed on the top left. Selecting a value will display its properties on the top right as well as displaying the raw VK record in the hex editor at the bottom. Selecting a property works the same as it did in the NK record.

On the subkeys tab is a list of all subkeys for the selected key. Double clicking a subkey will bring up another Technical details form for the double clicked subkey.

On the SK record tab, things again work as we have already seen. In this example, I have selected the "ACE record 0" structure. As a result, Registry Explorer highlighted the 36 bytes for that record. As we have already seen selecting any of the properties at any level selects those bytes.

On the hive details tab we can see things like the sequence numbers, time stamps, checksum, embedded file name, etc.

Exporting Registry keys

On the context menu in the Registry hives tree is an Export option.

If a key is selected that has subkeys, the recursive export option is available, else it is disabled.

After selecting one of the export options you will be prompted where to save the hive.

By default, the name of the key being exported will be used for the file name. When exporting recursively, '_Recursive' will be appended to the key name to indicate this.

Clicking 'Save' will export the hive

The resulting file is a plain text file that can then be imported back into the Registry.

Lines starting with a semicolon are comments.

An example of an exported key with values looks like this:

And finally, when exporting deleted keys, Registry Explorer will note this in the .reg file.

Registry Explorer exports keys in almost the same exact format as RegEdit does. Many hives and keys were compared using Beyond Compare to ensure the data being exported was consistent with RegEdit. In some cases, Registry Explorer will have a slightly different layout for some binary values, but in every case the data is imported exactly as it would be if the export were done via RegEdit.

The end of XWFRT?

March 22, 2015, 2:51 pm

≫ Next: ShellBags Explorer 0.6.0.0 released!

≪ Previous: Registry Explorer 0.0.2.0 released!

$

0

0

Starting with X-Ways Forensics 18.1, the report format generated by X-Ways Forensics was dramatically changed, breaking XWFRT.

Today X-Ways Forensics 18.2 Beta 3 was released. One of the changes included this (the key part is in bold):

* "Create main report" is now a 3-state checkbox in the case report options dialog. If only half checked, details about the evidence objects are not included in the case report, the evidence objects are merely listed. Evidence objects details, if included, now precede report tables in the report. Links to report tables now work even if the report is optionally split into multiple HTML files, and there is a link back from each report table to the report table overview. The report is now split based on the number of items that are referenced, not based on the number of pictures that are displayed in the report. If the report is split, the next segment is now linked from the bottomof the previous segment.

The previous behavior of X-Ways reporting was the primary reason XWFRT was created. The breaking changes in 18.1 necessitated a rewrite of XWFRT anyways but with the above change I do not know if the effort is worth it unless the community deems otherwise.

I have tested the new functionality and it works very well, linking to other HTML files and back to the main report as links are clicked.

I have suggested a few other tweaks to default X-Ways reports that hopefully make it into 18.2 release.

If XWFRT does something you cannot live without, please let me know and I will see what I can do.

---

ShellBags Explorer 0.6.0.0 released!

April 11, 2015, 10:17 am

≫ Next: AppCompatCache changes in Windows 10

≪ Previous: The end of XWFRT?

$

0

0

This version contains the following changes:

NEW: Added NodeSlot info. This column is hidden by default in the grid, but will always be shown in Details view

CHANGE: Updated Registry parser code (~150% faster and significant memory reduction)

FIX: Correct FAT vs exFAT file system hint in BEEF0004 blocks
FIX: Correctly detect end of ShellBag in C3 bags

Thanks to Dan P @4n6k for the NodeSlot request and testing!

Node slot

Both the grid and the details view now contain information on the ShellBag's NodeSlot value. This value points to a subkey under Bags. This key contains other keys and values that contain such information such as sort order, icon settings, size, and so on.

This is what NodeSlot looks like in RegEdit.exe

One thing to be aware of is that the NodeSlot value seen in RegEdit does NOT correspond to the ShellBags in the key where the NodeSlot value is. The NodeSlot shown is associated with the currently open key (BagMRU\0\0\0 in the example above). To see the ShellBag associated with a given NodeSlot, go up one level and find the value as shown below.

ShellBags Explorer does all of this for you, so you can just go to the corresponding Bags subkey to find the related data.

NOTE: Not all Shellbags have a NodeSlot value (Control Panel categories for example). In these cases, a Node Slot value of -1 will be shown in SBE.

Node Slot is hidden by default in the grid, so use the field chooser to show as required.

Node Slot will always be visible in Details view.

AppCompatCache changes in Windows 10

April 22, 2015, 2:32 pm

≫ Next: Introducing AppCompatCacheParser

≪ Previous: ShellBags Explorer 0.6.0.0 released!

$

0

0

The AppCompatCache value lives at SYSTEM\CurrentControlSet\Control\Session Manager\AppCompatCache.

In Windows 7, 8.0, and 8.1, the header is 128 bytes in size (more details here)

I recently added AppCompatCache support to my live response tool and when I tested it on Windows 10, I didn't get any shimcache entries.

Looking at a Windows 10 SYSTEM hive in a hex editor shows:

As you can see, the header is now 48 bytes in length. The signature, 10ts, remains the same in Windows 10 as it was for Windows 8.1.

As one would expect when it comes to this artifact, Microsoft has once again introduced another format. From my testing it looks like it breaks down as follows (subject to change):

Using this information I was able to extract 350 cache entries from my Windows 10 SYSTEM hive. This matches the value we see in the header as well.

Another very interesting thing that I found in cache entries were things like this:

I haven't done enough research into what these may mean for us yet. 

Time to update your code vendors =)

Introducing AppCompatCacheParser

May 1, 2015, 9:24 am

≫ Next: AppCompatCacheParser v0.0.5.1 released

≪ Previous: AppCompatCache changes in Windows 10

$

0

0

Yesterday I got a request to put together a utility to dump shimcache entries from the Registry. Today it is available!

Windows 7 (x86 and x84), Windows 8.x (Server 2012* as well), and Windows 10 are all supported.

You can get the binary here or browse the source on GitHub, so if you want to contribute or see how it works, check it out.

There are 2 files included:

AppCompatCacheParser.exe
NLog.config

Both are required. You can adjust the nlog.config file as needed to push logging to any other source you need to.

Usage is pretty straightforward:

The only required option is -s, so the program knows where to save things to. If you omit the -h option, the live Registry is dumped.

Notice the name of the machine is used (HAX0R in this case) when dumping the live system.

Here are some more examples. The program will take less than a second to run.

Output is in TSV format and entries are saved in files as follows:

Windows7X64_SYSTEM_AppCompatCache.tsv

Windows7X86_SYSTEM_AppCompatCache.tsv

Windows10_SYSTEM_AppCompatCache.tsv

Windows81_SYSTEMWin81_AppCompatCache.tsv

In short, its {OperatingSystem}_{HiveName}_AppCompatCache.tsv

All times are UTC.

Please hit me up if you want any additional features.

Enjoy!

PS If you are wondering what that console thing is, its cmder and its awesome.

AppCompatCacheParser v0.0.5.1 released

May 2, 2015, 5:42 am

≫ Next: AppCompatCacheParser v0.0.5.2 released

≪ Previous: Introducing AppCompatCacheParser

$

0

0

Changes include:

NEW: Added CacheEntryPosition to allow for sorting in the order cache entries were found in the Registry
NEW: Changed format of last modified timestamp to allow for sorting. Format is yyyy-MM-dd HH:mm:ssZ
NEW: Added -t switch that sorts the timestamps in descending order (most recent first)
NEW: Added a sweet icon

CHANGE: Removed -f switch (it was fun while it lasted)
CHANGE: Added GitHub URL to console window
CHANGE: Cleaned up command line switches and added explanation for short vs. long names
CHANGE: Reflect Windows 2012 along with Windows 8.1 as they use the same format

You can get the update here or via the Software page above

AppCompatCacheParser v0.0.5.2 released

May 20, 2015, 6:39 am

≫ Next: ShellBags Explorer 0.6.1 released!

≪ Previous: AppCompatCacheParser v0.0.5.1 released

$

0

0

This version tweaks the format of the LastModifiedTimeUTC column so that it is in the form MM-dd-yyyy HH:mm:ss which allows for easy sorting and grouping in Excel.

You can get the update here or via the Software page above

ShellBags Explorer 0.6.1 released!

May 20, 2015, 7:39 am

≫ Next: Introducing RECmd!

≪ Previous: AppCompatCacheParser v0.0.5.2 released

$

0

0

This version adds support for Windows XP hives. For the most part, things should "just work" with XP hives (NTUSER.DAT), but as you can see in the image below, there are some differences I need to work out/add support for.

Right now, SBE is only looking at the "Software\Microsoft\Windows\ShellNoRoam\BagMRU" key when it sees an NTUSER hive. I believe ShellBags can also be found in "Software\Microsoft\Windows\Shell\BagMRU" but I need to generate more test data to be sure.

What I really need now is Windows XP NTUSER.DAT hives so I can test things and make sure I am extracting as much detail as possible from XP hives.

Please email them or get them to me via Dropbox or other means.

You can get the update by starting SBE and looking in the lower right corner or downloading it from the Software page above.

Enjoy!

Introducing RECmd!

May 22, 2015, 7:15 am

≫ Next: RECmd 0.6.0.0 released!

≪ Previous: ShellBags Explorer 0.6.1 released!

$

0

0

RECmd is a command line tool to access offline (for now, live Registry support is planned) hives.

The command line options look like this:

There are 3 categories of commands, General, Query, and Search.

General

This category includes the Hive and Recover switches. Hive is always required for offline hives. Once online Registry is supported, if the Hive switch is missing, it will be queried/searched.

Query

This category includes the KeyName, ValueName, SaveToName, MinSize, and Recursive switches.

If either the key or value has spaces in them, be sure to enclose them in quotes.

When passing in key names, the root key name is optional. This is because most of the time you won't even know the root key name in order to be able to pass it in.

To get default values, use a value name of "(default)" as can be seen in the examples below.

Usage is very simple. To dump a key, you would do something like this:

In this example, i asked for the ".citrixonline" key and when data was returned we can see the full key path, including the root, after the Key: label.

To dump just a value, something like this would be used:

Notice how you get more detail when you query a key and a value. This allows you to choose how much detail you want based on your needs.

If you need to save the contents of a value's data to a file, use the SaveToName switch.

If you open c:\temp\MruListEx.bin, it looks like this:

Which is the same info we can see in a tool like Registry Explorer for the same value:

If the Recursive switch is used, full details about the initial key, its values, and all subkeys and their values are dumped. The data when using the Recursive switch is MUCH more comprehensive than when the Recursive switch isn't used. This again puts you in control when it comes to how much detail you want/need to see.

This is a partial output from what that might look like:

Finally, the MinSize switch can be used to find all values that are at least a given size. In the example below, all values whose value data is larger than 4,096 bytes are located:

This could be used to look for large blobs of data in the Registry that may correspond to executables, persistence mechanisms, etc.

Once a key is found you can use the previously discussed switches to query that key/value and export as needed for further analysis.

Search

This is a particularly useful feature to locate data across hives in key names, value names, and perhaps most importantly, in value data.

Searching is broken down into two types, simple and regular expression (RegEx) based.

Simple searching

Simple searching looks for the provided string in key names, value names, and value data. It is not anchored in any way. If a search target contains the search string, it is returned as a hit.

In the example below, all keys with "MuiCache" somewhere in the key name are returned via the sk switch:

To find all values with "edit" somewhere in the value name, we use the sv switch:

Notice how the key name and value name are separated by two colons (::).

To search for a string inside a value's data, we use the sd switch. To find value data containing "edit", we can use this:

You can also search for binary patterns too. For a simple search, just string the hex characters together with a dash (04-00-EF-BE for example). Here is the partial output when doing something like this:

This allows you to find signatures for common data structures ANYWHERE in the Registry. The binary signature used above is that of a BEEF0004 extension block, commonly used in ShellBags. It contains information such as MAC dates/times, MFT info, etc.

To actually see the data itself, use one of the query methods.

RegEx searching

RegEx searching is much more powerful than simple searches, but the trade off is that it can be slower depending on how complicated your RegEx is.

To use RegEx to search for key names, use the nk switch. Enclose the RegEx in quotes to make sure the shell doesn't try to interpret anything in there.

Values work the same way via the nv switch (this is partial output, but finding 320 entries took 0.393 seconds on my system):

and finally and perhaps most powerfully, we can search value data using RegEx via the nd switch:

All regular expressions must of course be valid .net regular expressions. Different flavors of RegEx providers allow for different syntax, so be sure to use the proper syntax.

Here are some RegEx tutorials for .net.

https://msdn.microsoft.com/en-us/library/az24scfc%28v=vs.110%29.aspx

http://regexhero.net/reference/

https://msdn.microsoft.com/en-us/library/hs600312%28v=vs.110%29.aspx

http://www.codeproject.com/Articles/9099/The-Minute-Regex-Tutorial

RegExBuddy is an awesome tool for building and testing RegEx against data sets. 

Why do we need another Registry tool?

The capabilities of RECmd can be leveraged to find new places where understood data is located in a systematic way. 

Earlier we talked about extension blocks. Extension blocks have a signature that looks like this: XX-00-EF-BE. Using RECmd (and Registry Explorer once searching is included in the next release) we can now find ALL extension blocks wherever they live via a search like this:

RECmd.exe --Hive "C:\Temp\UsrClass 1.dat" --nd "(04|03)-00-EF-BE"

This particular search looks for BEEF0003 and BEEF0004 blocks, but that can easily be extended to any pattern one wishes to search for.

No longer will it be required to export to hives to some kind of text format and use another tool to search through the Registry for patterns.

The full source code for RECmd is available here on Github. RECmd uses my Registry code on the backend and that is available here.

Right now you can get RECmd from here or via the Software page above. ReCmd will be bundled with Registry Explorer starting with the next release of RE.

This post merely serves as an introduction to what I hope the community will come up with as it relates to use cases for RECmd. Please let me know if there is anything else you want RECmd to do as well as any problems you have solved or other cool uses of RECmd you come up with.

Enjoy!!

---

RECmd 0.6.0.0 released!

May 23, 2015, 3:39 pm

≫ Next: RECmd 0.6.1.0 released!

≪ Previous: Introducing RECmd!

$

0

0

This version adds some new features and a bit of polish. Changes include:

New: Added --Sort that will sort the output according to the type of data returned (size for MinSize, time stamps for date range searches, etc)
New: Added --SuppressData. If present, do not show data when using --sd or --nd

Change: Make output consistent
Change: When searching for value data, show the value's data vs just the key and value name

Here is an example of the adjusted output for time stamp searches (time stamps are first now):

A sorting example:

And finally, when searching for value data, show the data itself vs. just value name.

You can suppress data values from being shown by using the --SuppressData switch.

Additionally, when searching for key names, only keys are shown and when searching for values, the key name and value name are shown.

Get the update from here.

RECmd 0.6.1.0 released!

June 3, 2015, 10:22 am

≫ Next: Reintroducing Registry Explorer and RECmd!

≪ Previous: RECmd 0.6.0.0 released!

$

0

0

This version is mostly cleanup and slight formatting changes to make things easier to use.

You can get the updated version here or on Github

Changes include:

CHANGE: Clean up command line option language, added another example with AM/PM vs 24 hour time format
CHANGE: Add root key name to top of output since its always the same
CHANGE: Strip root key name from output so output is more succinct.
CHANGE: General output formatting clean up ('value' vs 'values' when one result found, etc.)
CHANGE: Added subkey info (# of subkeys and basic subkey details) when dumping a key via --KeyName
CHANGE: Updated Registry back end to tweak formatting of output when dumping values via --ValueName

Here you can see the updated output showing the root key once at the top vs in every key. This cuts down on the amount of data displayed, especially when a lot of matches are found

When dumping a key, the number of subkeys is shown as well as a listing of the subkeys. Previously only the values were listed.

Finally, the format for showing values has been updated to condense the info. Previously, each property of the value was reported on a separate line. Now the type is in line with the name and slack is only shown when it exists.

Reintroducing Registry Explorer and RECmd!

July 1, 2015, 10:11 am

≫ Next: Registry Explorer/RECmd 0.7.1.0 released!

≪ Previous: RECmd 0.6.1.0 released!

$

0

0

I am happy to announce the availability of the next releases of Registry Explorer and RECmd! Extensive work has been done in a variety of areas which will be detailed below.

This post serves to hit the highlights of this release. The included manual should be consulted for a full description of new features and changes.

Finally, from this release forward, Registry Explorer and RECmd will be distributed together since they serve similar purposes.

The following describes version 0.7.0.0 of both RECmd and Registry Explorer.

NOTE: David Cowen, @HECFBlog, is having me on the Forensic Lunch this Friday, July 3, to discuss deleted record recovery in the Registry and the latest changes in Registry Explorer and RECmd. Check it out here.

Before diving into the changes, here are some examples as to what to expect in this version of Registry Explorer.

Example 1: 18 hives loaded, search term 'mui'

• Searching in key name resulted in 62 hits in 0.14533 seconds
• Searching in value name resulted in 75 hits in 0.19966 seconds
• Searching in value data resulted in 57 hits in 4.22546 seconds
• Searching in value slack resulted in 10 hits in 1.08956 seconds
• Searching in key name, value name, value data, and value slack at the same time resulted in 204 hits in 5.42693 seconds

Example 2: 84 hives loaded, search term 'comp'

• Searching in key name resulted in 870 hits in 0.36592 seconds
• Searching in value name resulted in 2,745 hits in 0.59400 seconds
• Searching in value data resulted in 6,862 hits in 8.58572 seconds
• Searching in value slack resulted in 97 hits in 3.42388 seconds
• Searching in key name, value name, value data, and value slack at the same time resulted in 10,574 hits in 14.06698 seconds

Registry backend

More than 25 commits have been made to the code base since May, 2015. Some of the changes include:

• Regular expression search support
• Find by value size
• Find in key and value name
• Find by last write timestamp
• Ability to search value data and value slack for strings regardless of capitalization or encoding
• Bug fixes
• Code clean up

Registry Search improvements

The majority of the improvements centered around search improvements in the core parser. After exploring several avenues, I came up with a way to allow for searching for strings regardless of how they are stored and in a case insensitive manner. This includes the ability to report back to the user the exact capitalization/byte string that was hit on.

Since the Registry parser powers both RECmd and Registry Explorer, both programs reap the benefits of these improvements. We will see examples of these improvements in the sections below.

RECmd

The last release of RECmd was mostly about cleanup and tweaking. This version includes the following changes:

• NEW: Added –Literal switch. When present, --sd and --ss switches will not be interpreted
• NEW: Added --ss switch for searching Value slack space
• NEW: Search terms are now highlighted in search results. Edit nlog.config to adjust colors for foreground and background
• NEW: Added --RegEx switch. When present, treat <string> in --sk, --sv, --sd, and --ss as a regular expression
• NEW: If nlog.config is missing, add default config and warn user
• CHANGE: Switches are NOT case sensitive any more
• CHANGE: Remove RegEx specific switches (See --RegEx above)
• CHANGE: Tweak command line option descriptions
• CHANGE: Updated nlog

Lets unpack some of these changes. The first thing to notice is that the command line switches have been simplified. Capitalization of switches is also not required any more (so --hive or --Hive works fine).

This includes removing the dedicated RegEx switches (starting with n) in favor of a single --RegEx switch that is used in conjunction with the standard search switches. 

A new search, -ss, has been added. This option looks in value slack space.

The other new switch is the --Literal switch that prevents RECmd from converting the entered string into its ASCII and Unicode byte string formats for search purposes. The literal switch is honored only when using -sd and -ss searches. 

Finally, the best (IMO) and most important change is RECmd's ability to highlight the search hit in the returned information. 

The colors used to highlight hits are completely user configurable. The default can be seen below, but you can adjust the colors via the nlog.config file as shown below.

A few examples are in order.

Key name and value name searches

Here a search against key names is performed for key names containing MuI, Notice however, that the keys returned contain a variety of capitalization that is different for the string we searched for.

RECmd is even smart enough to highlight Regular Expression based matches, like this:

This functionality really shines when searching in value data and value slack. 

One other thing to notice is that you are told the hits were found via RegEx at the bottom when the --RegEx switch is used.

Value name searching works the same way.

Value data searches

Value searches are where the new back end search capabilities start to shine. In the example below, we are searching for the string UsErs. Two hits were found, but neither is a plain text search hit.

The highlighted sections above break down as follows:

55-73-65-72-73 == Users (ASCII)

55-00-73-00-65-00-72-00-73-00 == Users (Unicode)

So what happened here? RECmd was able to take the search term, UsErs, and search for it in a way that ANY combination of capitalization will be found AND the exact bytes (and therefore capitalization of the hit) that make up the hit are highlighted regardless of the encoding of the string (Western 1252 and Unicode to be exact).

This feature provides enormous power to find data in Registry hives as search terms will be found both in binary and string based searches. In RECmd 0.6.1.0, the end user would have been required to search for each byte string in every possible capitalization format in order to find the hit. Needless to say, this would be near impossible and very tedious to attempt.

Value slack searches

Slack searches work the same way as value data searches. As an example, a search for sk was done in value slack. Since all value slack is treated as binary data, the importance of the search improvements can be seen here as well.

Other examples

Value data search for cdrom. We get back a variety of hits, some as strings of differing capitalization and some as encoded Unicode strings.

Registry Explorer

The vast majority of changes in this release were focused on Registry Explorer. In fact, this is pretty much a complete rewrite of every line of code in Registry Explorer. This was necessary to some initial bad design decisions which just did not scale.

The changes in this version include:

Hive processing is fully asynchronous, but very large hives can take a few seconds to display once the hive is loaded. This is due to the need to load all 

NEW: Full support for searching including key names, value names, and value data, both with simple searches and RegEx. Searching based on last write timestamps is supported as well

NEW: Fully asynchronous loading of hives which keeps the GUI responsive, even when loading 100+ MB hives (I am looking at you SOFTWARE hive)

NEW: Tech details hex editors now update with offset and selection length when bytes are selected

NEW: Added value context menu to copy value summary (a combination of name, type, and data), name, type, data, and slack to clipboard

NEW: Add value context menu to export data and slack to a file

NEW: Settings for things persist

NEW: Search strings are remembered and autopopulate when typing on the Find form. Use the Tools menu to clear

NEW: Added Convert | To hex ASCII and To hex Unicode to Find. This allows you to look for encoded strings in value data without having to manually convert strings to hex

NEW: Allow deleting of user created bookmarks in Bookmark Manager via Ctrl-Delete

NEW: Added context menu to Available bookmarks (Copy, expand/collapse, tech details) that work the same as the 'Registry hives' context menu options

NEW: Added 'Jump to key' context menu item on Available bookmarks tab that will select the hive's key on the 'Registry hives' tab

NEW: More hot keys added to main/context menus

NEW: Added 'Root key name' to Tech details | Hive details properties and strip root key from Tech details window title to save space

NEW: Added Export 'Registry hive' menu to File menu. This exports the tree exactly as it is shown to the selected format

NEW: Enable/disable expand/collapse subkey options depending on the expanded state of the selected key

NEW: Save positions of vertical and horizontal splitters

NEW: Trees and grids all save settings (sorting, filtering, conditional formatting rules) between sessions

NEW: Save size of main form

NEW: Improved hex editor control for RegBinary keys. Added offset, selection length, and data interpreter

NEW: Added 'Show associated deleted records' and 'Show unassociated deleted records' to Options menu

NEW: Added 'Slack viewer' tab for values that have slack space

NEW: Added 'Show parent keys when filtering' to options menu. Turning this OFF shows only the keys that match the filter. When ON, parent keys to keys matching the filter are also shown

NEW: Added a Total messages counter to lower status bar (far right) that indicates the total number of messages available on the Messages form

NEW: Added skinning support. Active skin can be changed from the Options menu

NEW: Added icon for Registry hive in the Registry hives tree to visually separate it from keys

NEW: Make hive name bold to make it stand out from keys

NEW: Tech details info can be copied via Ctrl+C (just the value) or Ctrl+Alt+C (Name: Value)

NEW: All hex viewers now support Ctrl+C to copy selected bytes to clipboard

NEW: Search for minimum value sizes added

NEW: Search in value slack added

CHANGE: Allow resizing of window below 800x600

CHANGE: Drag and dropping of hives supported on any of the 3 main sections of Registry Explorer

CHANGE: Status bars adjusted. Added options to hold Shift when double clicking in order to copy different parts of the key/value

CHANGE: Add vertical scroll bar to Technical details hex editors

CHANGE: Rename tree context menus from 'child nodes' to 'subkeys'

CHANGE: Hide Messages form by default since things load and process faster when its hidden

CHANGE: Icon for existing key placeholder in Associated deleted records updated

CHANGE: Icon for Associated deleted records updated

CHANGE: Made legend icons bigger

CHANGE: Bookmarks manager now allows editing/deleted both common and user created bookmarks

FIX: Bug fixes in Registry parser (yay unit tests)

FIX: Show SK record in Technical details form

Whew! That's a long list! Lets look at some details to see them in action.

General improvements

Fully asynchronous loading of hives. This means hives get loaded in their own thread in parallel. As hives are finished, they are displayed. This lets smaller, less complicated hives load quicker while bigger hives finish processing. You can start interacting with hives as soon as they show up.

Settings for window size, position, and options persist between sessions and are stored in the Settings folder. Pretty much everything is remembered between program executions.

Interface improvements

The first thing you will notice when loading hives is that the Registry hives is that some icons have changed. The hive itself has a green icon and the hive name is in bold. The number at the end of the Registry hives tab is the total number of hives currently loaded.

The status bars across the bottom were tweaked as well. Hovering over the key path, value name, or last write time will show you a tool tip with different options to copy data to the clipboard.

The checkbox to the far left toggles showing the root key name in the key path. It is not shown by default.

When looking at RegBinary and value slack, a new hex viewer has been created. The current offset and selected bytes count is shown. After selecting a range of bytes, Ctrl+C copies those bytes to the clipboard.

In the lower right corner is a button to show a data interpreter. The data interpreter decodes the binary data from the current offset into a variety of formats. Sweeping a range of bytes does not affect the data interpreter, only the current offset. The data interpreter can be seen below.

In the lower right of the data interpreter is a ? that contains a tool tip for how to copy values (and optionally the value name) to the clipboard.

The data interpreter will update in real time as the offset is changed in the hex viewer.

Values now have a context menu, as seen below.

The options are straightforward with the exception of Value summary. This option copies basic info about the value in text format to the clipboard, like this:

Value: MRUListEx (RegBinary)

Data: 00-00-00-00-FF-FF-FF-FF

Slack: 6F-00-20-00

The Export options allow for saving either value data or slack to a file (as binary data).

The Technical details form has been tweaked slightly. In the lower right is a ? that shows a tool tip for how to copy the properties (and optionally the property name) via Ctrl+C.

When a value is selected, the value's data and, if present, slack, is shown at the bottom. This allows you to verify both the VK record and the data (found at offset 0x0C) in the same place.

The available bookmarks tab has also been updated with the icons from the Registry hives tab. A new context menu option, 'Jump to key', has been added.

Using this option will select the bookmarked key in the currently active hive and switch to the Registry hives tab. This is useful to see the bookmark in context with other keys in the hive.

The options menu has been tweaked and improved, adding support for a different way to filter the Registry hives tree and full skin support.

The 'Show parent keys when filtering' option works as follows. The default option is to show all parent keys of keys that match the filter. In this case, the filter value is 0. The parent keys are highlighted in yellow.

Turning this option off and doing the same search results in a much different result. Notice in the screen shot below only the keys that match the filter criteria are shown. This can greatly reduce noise in the results in addition to lessening the need to scroll to the keys that match the filter criteria.

The other aspect this option controls is whether to show subkeys of keys that match a given filter. With the option on, any subkeys of keys that match the filter will continue to be shown. With the option off, subkeys of keys matching the filter are hidden.

Registry Explorer contains the ability to apply Excel like conditional formatting to any of the tree or grid controls. To set up conditional formatting, right click on a column header and choose the context menu option for conditional formatting at the bottom.

If we choose the Text that contains option and configure it like this:

The Registry hives tab changes to match the rule (instances of 'Bag' are circled).

This lets you highlight conditions that are meaningful to you and your investigations.

Finally, as mentioned above, Registry Explorer supports skinning so you can get into the holiday spirit when the time comes.

This is just an overview of the changes in this version of Registry Explorer. For full details, see the manual.

Searching

After loading one or more hives, use the Tools | Find option (or simply press Ctrl+F) to bring up the Find dialog. 

All searching is also fully asynchronous and results are returned as they are available. This results in smaller hive search results showing up more quickly than larger hives. 

The Options menu lets you clear recent Standard searches as well as convert strings to their ASCII or Unicode byte string equivalent.

The Help menu contains a link to a web page that provides help on constructing .net regular expressions as well as some search tips.

Key name and value name search

To conduct a key or value name search, enter a string in the 'Search for' box and check the Key name and/or Value name boxes. Adjust the search type to be either a simple string search or a regular expression. To start the search, click the 'Search' button or just press Enter on the keyboard. Registry Explorer will adjust the default button depending on which search you are conducting, so it is often easiest just to enter your search conditions and pres Enter.

In the example above, we found 143 search hits across 13 loaded hives in 0.31 seconds. 

The Hive Name indicates which hive the hit was found in and the Hit Location will indicate where the hit was found.

Depending on the number of search hits, it is often helpful to use the grouping feature. If we drag the Hive Name and Hit Location columns to the top of the grid, we can now browse the search results by hive and hit type.

To view a hit in context, simply double click it and Registry Explorer will select the right hive, key, and value (assuming it is a Value Name hit). If we double click row with the MuiVerb hit above, we see the following in Registry Explorer.

Notice how the search term is highlighted in the value name. If we double click on a key name hit, we get the following in Registry Explorer.

Value data and value slack search

When conducting Value data and/or value slack searches, the 'Search for' string is looked for in such a way as to find hits regardless of case and encoding (i.e. the string is converted to ASCII and stored as byte array, or converted to Unicode and stored as a byte array). 

If you do not want Registry Explorer to search for encoded strings, check the Literal check box. When Literal is checked, the search term will be looked for exactly as it is entered. This would be useful for entering a string of bytes, separated by dashes, that you wish to look for. Without Literal enabled, Registry Explorer would look for the byte string exactly as entered and in RegBinary fields. The Literal option makes searching more efficient when you want to search for an already encoded search term. Literal is only honored for Value data and Value slack searches.

When a search hit is found in a non RegBinary value, all instances of the search term are highlighted as seen below.

When a hit is found in a RegBinary value, the corresponding hit is highlighted in the hex viewer, as seen below. In this case, notice how the string we searched for was An, but we still found the hit for an.

Only the first instance of the search term is highlighted in the hex viewer, but you can use the preview to the right of the hex viewer to see other string values in the binary data.

The other thing to notice is how the search results can be filtered. These filters can be edited by clicking the 'Edit Filter' button to the far right of the filter condition at the bottom of the grid.

When looking at a value slack hit, the Slack viewer tab will be active and the hit will again be highlighted.

Last write timestamp search

Registry Explorer allows for searching for keys with last write times before, between, or after certain time stamps. 

Once the search is complete, double clicking on a row will again select the key in Registry Explorer.

Minimum value size search

Registry Explorer can find all values with data size greater than or equal to a given size in bytes. In the example below, 228 hits across all open hives were found in 0.4 seconds.

Clicking on the Value Data Length column sorts the column and makes it easy to find the largest values in the loaded hives.

As as have already seen, double clicking will select the key and value in the main Registry Explorer window.

Miscellaneous

The Bookmarks manager has been updated to allow for editing both common and user created bookmarks. You can also delete bookmarks from this interface as well.

Finally, a manual is now included that fully explains all of these things and more in detail. Be sure to read it so you can get the most out of Registry Explorer and RECmd.

Whats next?

With searching and the performance overhaul out of the way with this release, the next release will focus on adding plugins. 

The general concept for plugins is to allow plugin authors to register interest in a Registry path and or value. When a user clicks on a key or value that matches what a plugin is interested in, the plugin will process the data which will then be made available to the end user. 

For plugins that process entire keys, a new tab will be added next to the Values tab in the upper right. For plugins that process a single key, a new tab will be added to the bottom with the other viewer tabs (type viewer, slack viewer, etc). 

This has the advantage of having multiple plugins written for keys/values without having to choose which one to use.

For example, if a plugin was interested in shellbags, whenever a user navigated to BagMRU or a subkey, a new tab would show up next to the Values tab. This tab would then contain all of the decoded shellbag entries for the selected key. Since the plugin would get its own tab, both the original data AND the decoded data is available and visible at all times.

I am also interested in your thoughts, ideas, and feature requests. Please let me know how Registry Explorer and RECmd can be improved to make your work easier.

I hope you enjoy this release!

You can get the update from the Software page.

Registry Explorer/RECmd 0.7.1.0 released!

July 7, 2015, 3:02 pm

≫ Next: Introducing bstrings, a Better Strings utility!

≪ Previous: Reintroducing Registry Explorer and RECmd!

$

0

0

This is the #DFIRSummit release!

Several people asked for some tweaks such as recursively searching Registry hives via RECmd and allowing Registry Explorer to be set as the default application for hives.

This release adds those things as well as an option in Registry Explorer to check for and download any new versions via the Help menu.

RECmd changes

New: Added --Dir switch. This recursively searches for hives in a given directory and searches each of them

As an example, if you used, say X-Ways to find all available hives in an image, then recover/copied them out to a directory using the 'Recreate original path' option, you may end up with something like this:

Using the new --dir switch, you can now search across every hive found in the specified directory and all subdirectories.  In the example output below, only beginning and ending of the console output is shown

When the search is complete, the total number of hits and so on is displayed. If there are any non-Registry hives found during the search they are skipped and this fact is also displayed in the summary (17 hives out of 18 files).

Registry Explorer changes

New: Registry Explorer can now function as a "default application" in that you can associate RE with *.dat and then double click hives. This also allows for setting up RE in other apps like X-Ways as an external viewer, dragging and dropping hives onto RE shortcut/executable, etc
New: Added 'Check for updates' to About menu

As an example of setting up Registry Explorer as a viewer in another application, let's set up X-Ways to send hives to Registry Explorer.

In Options | Viewer programs, add Registry Explorer to the Custom viewer programs list in the lower right by clicking on the + sign and selecting the Registry Explorer binary.

After finding Registry hives, select a hive, then right click and choose Viewer Programs | RegistryExplorer.

X-Ways will then export the hive to its temporary directory and tell Registry Explorer to open it.

As usual, get it here. Once you have 0.7.1.0, you can then check for updates via the Help menu in Registry Explorer.

Introducing bstrings, a Better Strings utility!

July 11, 2015, 9:40 am

≫ Next: bstrings 0.9.0.0 released

≪ Previous: Registry Explorer/RECmd 0.7.1.0 released!

$

0

0

Do you like strings? Do you like to find strings? How about in binary data? What about via regular expressions?

ME TOO!

bstrings works like other strings utilities out there, but it also has some additional features not found elsewhere such as regular expression searching, some useful built in regex patterns, and string cleanup (trimming white space from the returned string), sorting. duplicate removal, and so on.

Using bstrings is simple

The -p switch shows a list of built in regular expression patterns.

To use a pattern, simply type the name of the pattern you want to use along with the --lr switch.

Here we are looking for ipv4 strings.

and here we are looking for anything looking like a url and port number.

Of course, simple string searches are available too via the --ls switch.

You can sort by length or alphabetically via the --sl and --sa switches.

First, by length

And alphabetically

That's pretty much it! If there's something else you would find useful, please let me know!

You can get bstrings here.