Data

Data object represents array of bytes (octets) of arbitrary size. The data are provided in chunks, whose size is determined by the data provider.

The data object keeps no state information (there is no current position). Client must specify, what offset it wants to read and a chunk containing byte at that offset is returned.

Data object uses a data provider to access data, thus allowing uniform access to different data formats.

Because the data is accessed in chunks and not just bytes, the speed overhead is relatively small. This allows chaining the data object and combining it's power.

Data object unifies concepts of strings, files, memory chunks, disks, filters and other concepts found in other systems.

DataOffset

DataOffset represents position in data.

It should be 64 bit integer on platforms supporting 64 bit int types. On platforms without 64bit support, only 32 bit offsets can be used. This limits maximal size of data object to 4Gb.

As an option, we can support structure of two 32 bit numbers.

DataChunk

Data chunk represents consecutive array of bytes from data object. It is created by data provider as an result of DataRead call. It specifies offset of the first byte it contains and number of bytes it contains.

Data provider interface

This is interface implemented by data provider.

void DataFree(VoidPtr data)

Free all resources allocated by data provider.

Error DataSize(VoidPtr data, DataOffset * p_size)

Return the size of the data in bytes. In some data providers, this function can be expensive (for example, it may be necessary to decompress the data to get the correct size). Avoid calling this method, if it's not necessary.

To avoid reading data size in advance when processing the data, it is possible to read chunks repeatedly, until returned chunk is NULL.

Provider may return ERR_UNIMPLEMENTED error, in such case the system will try to compute the size by reading all data.

Error DataRead(VoidPtr data, DataOffset offset, DataChunk ** p_chunk)

Return a chunk of data containing the byte at specified offset from the first byte of the data.

The chunk can start at an arbitrary offset and be of size chosen by the data provider. If the offset is bigger then the size of the data, NULL chunk is returned.

The pointer to the requested byte can be obtained using 'chunk->buf.data + (offset - chunk->offset)'.

Chunk is an object and must be released using ObjFree method when no longer required.

Error DataModify(VoidPtr data, DataOffset offset, DataOffset size, Data * new_data)

Modify the data.

The section of data specified by offset and size will be replaced by the contents of specified data.

If the size = 0, the new_data will be inserted at specified offset. If the new_data is NULL or empty (it's size is 0), the specified section is deleted.