Carving and hash signatures

This lab covers searching for files with hashes and file carving. File hashes present an important method of rapidly searching for and identifying known good and bad files. A file hash database of files to be searched for can be used to rapidly identify them on a system, even when their names have been changed in an attempt to obfuscate their true type.

File carving is a file extraction method for recovering files from a partition or disk image that may be corrupt. Or it may be used to recover deleted files.

KnownGoodFiles is a hash database of files which should be excluded from further analysis. These could be things like system files. The files have been hashed using a file source which has been validated (e.g. the files were downloaded from the manufacturer).

KnownBadFiles is a hash database of files which a forensics colleague of yours has created. These are hashes of files you are specifically interested in finding and examining. These could be be questionable jpegs which were discovered on another computer, or for instance rootkit executables which you think may have been used as part of a crime.

Question 2: Generate a hash database

Question 3: hfind

Now lookup the md5 hash of Frankie.xml which you saved into myhash.hdb in the last question. Using the hexadecimal hash value for Frankie.xml, search for that has using hfind in both KnownGoodFiles.hdb and KnownBadFiles.hdb.

Search of KnownGoodFiles produced:
Search of KnownBadFiles produced:
Expert opinion of Frankie.xml:

Tests - not attempted Details
KnownGoodFiles	UNTESTED
KnownBadFiles	UNTESTED
Opinion?	UNTESTED

Question 4: Sorter and filtering

Question 5: Linking the Techniques

In this question you will put all you have learned so far together to search the image for known bad files that have been obfuscated.

The search.dd image has a number of files on it that are identified in the KnownBadHashes.hdb file. However, some of these images may have been tampered with and obfuscated by a user. In some cases they have been put there to create a false positive result. The following table should be filled with the details of the files found on the image. You should provide the original image name as found in the KnownBadHashes.hdb file, the name of the File as found on the file system, as well as the md5 hash that links the two. Finally, select the method of obfuscation employed by the user to hide the true nature of the file.

You can fill in this table by using any method you deem necessary. You should use the output of solver from the previous question, as well as start a new case in Autopsy to quickly analyse the files in a GUI environment. Finally, remember that you should inspect header information to find out the true nature of a file.

In the analysis remember to drill down through any obvuscation. For instance, if you document a file which is compressed, also test the uncompressed version of that file against the KnownBadFiles and list that file in the table using its uncompressed name as reported by sorter. So for example if "gordon.xml" is really a gziped jpeg called "gordon.jpg", and this jpeg is also listed in the KnownBadFiles, then put this into the table too using "gordon.jpg" as the "Name in disk image" and make the observation that this is "obfuscated file extracted".

Name in HashDatabase	md5 hash	Name in disk image	Observation

Tests - not attempted Details
You have answer 1	UNTESTED
You have answer 2	UNTESTED
You have answer 3	UNTESTED
You have answer 4	UNTESTED
You have answer 5	UNTESTED
You have answer 6	UNTESTED
You have answer 7	UNTESTED
You have answer 8	UNTESTED
You have answer 9	UNTESTED
You have answer 10	UNTESTED
You have answer 11	UNTESTED

Question 6: Scalpel

List the names of the files found and extracted from carve.dd by scalpel by comparing the hashes of the files in the scalpel output directory with the KnownBadFiles.hdb.

Hint: use the md5deep tool to recursively analyse all the files in the scalpel output directory. Ensure you use the -b flag.

Hint 2: you will need to clean up the md5deep output file so that hfind will work with it. Use the following regular expression:

sed 's/\s*[0-9a-z]*.\(doc\|jpg\|txt\)//' FILE_CREATED_BY_MD5DEEP.hdb

Hint 3: The hfind command can take a file which contains a list of md5 hashes and look up each line of that file in its hash database.

Filenames carved from image which match a hash in KnownBadFiles

Tests - not attempted Details
answer 1	UNTESTED
answer 2	UNTESTED
answer 3	UNTESTED
answer 4	UNTESTED
answer 5	UNTESTED

Linux tutorials:	intro1 intro2 wildcard permission pipe vi essential admin net fwall DNS diag Apache1 Apache2 MySQL1 MySQL2
Caine 3.0:	Essentials \| Basic \| Search \| SysIntro \| 5a \| 5b \| 5c \| 6 \| 7 \| 8a \| 8b \| WebBrowserA \| WebBrowserB \| Registry
Useful:	Quiz \| Forums \| Privacy Policy \| Terms and Conditions
Site Links:	XMLZoo ActiveSQL ProgZoo SQLZoo