File reader

As it is easy to guess, file reader allows you to read sequences of bits or bytes from the input file. The position from where the bits/bytes will be read is controlled by two internal variables - CurrentOffset and CurrentBit, which are not directly visible to your template. However, you can use a few special functions that control these variables. Both these variables are 32-bit wide unsigned integers, therefore it is not possible for TempLuator to process files bigger than 4G. They work together - the next bit to read is defined by the appropriate sum of both variables (CurrentOffset * 8 + CurrentBit). The first variable - CurrentOffset - contains offset of the current byte. This offset is counted relatively to the Hiew cursor position. The second variable contains the number of the bit that is the next one. Even though the CurrentBit variable may contain a big number, many functions internally normalize the Offset/Bit pair so that the CurrentBit variable is usually in range 0...7. When you start TempLuator, both variables are set to zero. It is not possible to go beyond the file. That is, you can't read bytes preceding the Hiew cursor position and past the end of the file.

ReaderData = ReadBytes(NumberOfBytes)
ReaderData = PeekBytes(NumberOfBytes)
 
ReaderData = ReadBits(NumberOfBits)
ReaderData = PeekBits(NumberOfBits)
 
ReverseBits(boolean)
Bit0First()
Bit7First()
MSVCBitOrder()
MPEGBitOrder()
 
SkipBytes(NumberOfBytes)
SkipBits(NumberOfBits)
 
Align(Alignment)
 
FilePos = GetFilePosition()
SetFilePosition(FilePos)
 
Seek(Pos[, Relative=false])
 
BytesRest = BytesTillEOF()

NumBytes = BytesBetween(FilePos1, FilePos2)
NumBytes = BytesProcessed(StartPos)
NumBits = BitsProcessed(StartPos)

ReaderData

All file reading functions return this Lua table. Many functions require exactly this table as an argument. This table keeps all information regarding the data retrieved from file. Here is the internal structure of this table:

ReaderData = {
    ["Offset"] = number,		-- CurrentOffset
    ["Bit"]    = number,		-- CurrentBit
    ["Bits"]   = number,		-- number of bits
    ["Data"]   = string,		-- actual data read from file
    ["Bf"]     = boolean		-- false for byte reader, true for bit reader
}

It is important to note that in case of bit reader the Data field always contains 64 bits of data (that is, this string consists of 8 bytes) and the unused bits are set to 0. Perhaps, it is a bad idea to use this field for any string operations for the bit reader data. However, it is entirely normal to use the Data field for any desired operation in case of byte reader. The Bits field always contains number of bits, not bytes, even in case of byte reader. It is definitely a BAD idea to change any of the fields of this table.

No byte reordering is done to the ReaderData.Data field and all bytes in this string are exacly in the same order as they are in the file. Therefore, you can use this field for string comparison operations. For example, if you want to check if read data is equal to some string, you can use the following code:

-- check for PE header
local Data = PeekBytes(4)
if Data.Data ~= "PE\0\0" then error("This is not a 'PE' file") end

ReaderData = ReadBytes(NumberOfBytes)

Read requested number of bytes from the input file and advance file pointers. The only mandatory parameter is the number of bytes to read from the input file (must be positive and greater than zero). Returns ReaderData table. Throws an error if current file position is not byte aligned (use Align(1) to align the current position to byte boundary). Throws an error if the requested amount of bytes cannot be read from the file.

ReaderData = PeekBytes(NumberOfBytes)

Exactly the same as ReadBytes() but preserves current file position.

ReaderData = ReadBits(NumberOfBits)

Read requested number of bits from the input file and advance file pointers. The only mandatory numeric parameter is the number of bits to read. It must be in range 1...64. Returns ReaderData table. Its Data member is always 64 bits wide with unused bits set to 0. That is, it is a unsigned __int64 number placed in Intel byte order with the LSB first.

This is a pretty wicked beast... Why? Just because there are sooo many ways the bits can be read from the file... Four, to be precise. The easiest way to explain all the stuff is to show an example. Let's suppose we are on a byte boundary. There are two bytes in the input file. Let's name all their bits so that we can track them later:

Now we want to read 12 bits from the file. Back to our four combinations. First of all, we may consider bit0 to be the first (Bit0First()) or bit7 to be the first (Bit7First()). Next, we may want to reverse the bits (ReverseBits()) in the resulting number. So here is the table explaining what we get in each case:

Condition
Bit63.........Bit0
Name
bit0 first, no reverse
00...0hgfedcbaponm
 
bit0 first, reverse bits
00...0mnopabcdefgh
bit7 first, no reverse
00...0abcdefghijkl
bit7 first, reverse bits
00...0lkjihgfedcba
 

If you look carefully at the table above, you'll definitely notice that at least two combinations have sense - these two in the middle of the table that have names. Yes, the first one is exactly how C bitfields are processed by the MSVC compiler, the second one is used in MPEG. Due to the fact that these bit orders are widely used, there are two special functions that select the corresponding bit order - MSVCBitOrder() and MPEGBitOrder(). The other two combinations are rare and intended just for pure aesthetes and perfectionists. I seriously doubt you're going to use one of them... By default, TempLuator uses MSVCBitOrder().

ReaderData = PeekBits(NumberOfBits)

Exactly the same as ReadBits() but preserves current file position.

ReverseBits(boolean)

This function tells the bit reader if it needs to reverse bits. The only mandatory parameter must evaluate to boolean and tells if bit reversing is necessary (true) or not (false). This is set to true by default. See ReadBits() for all the gory details. It is an error to change bit reversion mode if the current file position is not aligned to byte boundary.

Bit0First()

This function tells the bit reader that it should consider the LSB (bit0) of each byte to be the first bit in that byte. This is the default. See ReadBits() for all the gory details. It is an error to change bit order mode if the current file position is not aligned to byte boundary.

Bit7First()

This function tells the bit reader that it should consider the MSB (bit7) of each byte to be the first bit in that byte. See ReadBits() for all the gory details. It is an error to change bit order mode if the current file position is not aligned to byte boundary.

MSVCBitOrder()

This function tells the bit reader that it should use the same order of bits as the MSVC compiler when it processes C bitfields. It is a short way to say Bit0First() and ReverseBits(true) in one call. This is the default. See ReadBits() for all the gory details. It is an error to change bit order mode if the current file position is not aligned to byte boundary.

MPEGBitOrder()

This function tells the bit reader that it should use the same order of bits which is used in MPEG bit reader. It is a short way to say Bit7First() and ReverseBits(false) in one call. See ReadBits() for all the gory details. It is an error to change bit order mode if the current file position is not aligned to byte boundary.

SkipBytes(NumberOfBytes)

This function lets you to skip requested number of bytes. The only required parameter gives the number of bytes to skip. If this parameter is positive, the current file position moves towards the end of the file. If the parameter is negative, it is moved towards the beginning of the file. Please note that skipping bytes backward cannot get across the Hiew cursor position and you'll get an error in this case. Theoretically, you can move the current file position beyond the end of the file. However, you will receive an error when you consequently try to read bits or bytes from that position. Note that this function does not change the current bit - if you were in the middle of a byte, then after skipping bytes you'll be in the same position inside of the new byte. SkipBytes(0), though senseless, is not an error.

SkipBits(NumberOfBits)

Almost the same as SkipBytes() with the only exception that this function operates with bits.

Align(Alignment)

This function aligns current file position to the given boundary. In addition, it always resets bit position to the byte boundary. The only required numeric parameter must be a power of 2. Use Align(1) in order to skip all nonprocessed bits of the current byte (if any) and go to the next byte (or the current one in case no bits of this byte were read yet).

FilePos = GetFilePosition()
SetFilePosition(FilePos)

These two functions let you save and later restore the current file position. Both of them operate with the following Lua table:

FilePos = {
    ["Offset"] = number,		-- CurrentOffset
    ["Bit"]    = number,		-- CurrentBit
}

Seek(Pos, Relative)

This function provides another way to navigate through the input file. The first parameter is a number that defines the new file position. The second parameter is a boolean value. If the Relative parameter is true, then new offset is the sum of the current offset and the Pos parameter. We use signed arithmetics here so negative Pos value moves the current position towards the beginning of the file. Otherwise, when Relative parameter is false, new offset becomes equal to the Pos parameter. If the Relative parameter is omitted (that is, according to generic Lua rules it is nil or false in our case), absolute positioning will be used. Note that this function does not change the bit position and works on the byte level.

BytesRest = BytesTillEOF()

This function returns the number of bytes that is available for reading before the end of the file is reached.

NumBytes = BytesBetween(FilePos1, FilePos2)

This function returns the number of bytes between two file position (as returned by GetFilePosition()), both of which must be byte-aligned. This value may be used as a rough equivalent of the C sizeof operator. Please note that NumBytes is always positive (or zero) so if you swap parameters, you'll get the same result.

NumBytes = BytesProcessed(StartPos)

This function returns number of bytes between the current file position and file position given as StartPos parameter. Both file positions must be byte-aligned. If StartPos is greater than the current file position, the result is negative. Here is a usage examle:

-- remember the current file position. If it is not aligned to byte, call Align(1) first
local StartPos = GetFilePosition()
 
-- do the processing
Rect("MyRect", "This is the standard RECT structure")
 
-- determine and show how many bytes has the Rect() function consumed
print(string.format("sizeof(RECT) == %d", BytesProcessed(StartPos)))

NumBits = BitsBetween(StartPos)

This function is almost the same as BytesProcessed() with the only exception that it counts bits instead of bytes. Consequently, neither StartPos nor the current file position have to be byte-aligned.