If you can see this check that

Main Page

File Searching and Management


Bytes and Sort

User:
Password:

Objectives

The tutorials this week are split into two parts. This part is focused on standard Linux, looking at byte manipulation techniques in binary files, as well as sort. In all cases a byte offset is the number of bytes into a file, where a byte offset of 0 is the first byte in the file.

Question 1: Binary Files

Press this button to create a simple file to manipulate. This will create a file "/home/caine/bindat".

Tests - not attempted
Create bindat UNTESTED

Use xxd to examine bindat. What is the hex value stored at offset 0x61? Format your answer in hex with a leading 0x, e.g. 0xff.
Byte at 0x61:

Tests - not attempted
bindat still valid UNTESTED
Byte value correct UNTESTED

Assuming a block size of 512, what is byte 6 of block 1 (where the first block is called block 0)?
Byte at block 1 offset 0x06:

Tests - not attempted
bindat still valid UNTESTED
Byte value correct UNTESTED

If you do

echo -ne "\x65"
the echo command will bring to the screen the binary code for 0x65. This is the ASCII code for 'e'. If this echo was in a pipe with the dd command, you could use this to set individual bytes.

For instance, to set the byte offset 5 of file1.dd to 0x55, you could do:

echo -ne "\x55" | dd of=file1.dd seek=5 bs=1 count=1 conv=notrunc

Demonstrate this technique by copying bindat to a file "bincopy", and setting byte offset 29 to 0x9f.

Tests - not attempted
bindat still valid UNTESTED
bincopy only changed as needed UNTESTED
Byte value correct UNTESTED

Challenge question... try this question for a maximum of 10 minutes. If you cannot get it dont worry, as it is a little tricky!

Copy bindat to bincopy2, and change bincopy2 so that byte offsets 0x310 to 0x330 inclusive are set to 0x55.

Hints: Remember 0x310 to 0x330 INCLUSIVE. Remember there is an offset 0x000. Check what you are doing with xxd into less. And dd uses decimals not hex... Finally "count=" is the maximum size transferred, so make sure there is sufficient data comming from the echo to make this work... For instance:

echo -ne "\x55"                  -- This supplies 1 byte of data, or count=1
echo -ne "\x55\x55"              -- This supplies 2 bytes, so could do up to count=2

Tests - not attempted
bindat still valid UNTESTED
bincopy2 only changed as needed UNTESTED
Byte value correct UNTESTED

Question 2: Sort

The next few questions check your understanding of the sort command. Sort can read from a pipe or from a specified filename. The basic way sort works is to sort on a column where the columns are seperated by spaces, so a file names.txt like:

gordon  russell
jim     jackson
bob     ludwiniak
can be sorted by the first column easily by doing
sort names.txt
If you want to sort by a column other than 1, you have to select the column. This is called the key, and has a start and end column numbered from 1. So to sort column 2 you need to do "-k" then the start and end column with a comma between them, i.e.:
sort -k 2,2 names.txt

To practice, press the button below to create a file "stuff.txt" in /home/caine.

Tests - not attempted
Create stuff.txt UNTESTED

Sort stuff.txt into alphanumeric order based on column 1 (the md5 value). Save this into /home/caine/sort1.txt.

Tests - not attempted
stuff.txt still valid UNTESTED
sort1.txt is correct UNTESTED

Now sort stuff.txt by column 2, and save the output as sort2.txt.

Tests - not attempted
stuff.txt still valid UNTESTED
sort2.txt is correct UNTESTED

Introduce the "-u" flag to sort, which removes any duplicate entries (such as /bin/ls in out example file). The 'u' stands for 'unique'. So now sort stuff.txt by column 2, but make sure all rows are unique. Save the output to sort3.txt.

Tests - not attempted
stuff.txt still valid UNTESTED
sort3.txt is correct UNTESTED

In these examples the character which split column 1 from column 2 was the space character. This is called the column delimiter. Sometimes columns will be seperated with other characters, such as "," for comma seperated files. So for instance if names.txt was:

gordon,russell
jim,jackson
bob,ludwiniak
can be sorted by the first column easily by doing
sort -t"," names.txt
and can be sorted by the second column easily by doing
sort -t"," -k 2,2 names.txt

Sort /etc/passwd (delimited by the ":" character) by column 5 and save that output into sort4.txt.

Tests - not attempted
sort4.txt is correct UNTESTED

If you are sorting a column which is composed of numbers, you need to decide if you want alphanumeric or numeric ordering. In alphanumeric ordering, "10" comes before "9" as the first character of "10" has a lower ascii value than "9". If you want numeric ordering where "10" is higher than "9", you must specify this in the "-k" command after each column number using the trailing character "n". So to sort column 10 numerically you need to do: "-k 10n,10n"

Firstly sort /etc/passwd on column 3 alphanumerically, saving the output to sort5.txt. Confirm that the last line of sort5.txt has a 9 in column 3.

Tests - not attempted
sort5.txt is correct UNTESTED

Repeat the sort, but this time sort numerically and save the output into sort6.txt. Confirm that the last line of sort6.txt has the number 65534 in column 3.

Tests - not attempted
sort6.txt is correct UNTESTED


Linux tutorials: intro1 intro2 wildcard permission pipe vi essential admin net fwall DNS diag Apache1 Apache2
Caine 6.0: Essentials | Basic | Search | SysIntro | 5a | 5b | 5c | 6 | 7 | 8a | 8b | WebBrowserA | WebBrowserB | Registry | Browser
Digital Investigation: Editing | Email | Logs | Strength
Kali: 1a | 1b | 1c | 2 | 3 | 4a | 4b | 5 | 6 | 7a | 8a | 8b | 9 | 10 |
Useful: Quiz | Forums | Privacy Policy | Terms and Conditions
Site Links:XMLZoo ActiveSQL ProgZoo SQLZoo

Copyright @ 2004-2014 Gordon Russell. All rights reserved.