Data object represents array of bytes (octets) of arbitrary size. The data are provided in chunks, whose size is determined by the data provider.
The data object keeps no state information (there is no current position). Client must specify, what offset it wants to read and a chunk containing byte at that offset is returned.
Data object uses a data provider to access data, thus allowing uniform access to different data formats.
Because the data is accessed in chunks and not just bytes, the speed overhead is relatively small. This allows chaining the data object and combining it's power.
Data object unifies concepts of strings, files, memory chunks, disks, filters and other concepts found in other systems.
DataOffset represents position in data.
It should be 64 bit integer on platforms supporting 64 bit int types. On platforms without 64bit support, only 32 bit offsets can be used. This limits maximal size of data object to 4Gb.
As an option, we can support structure of two 32 bit numbers.
Data chunk represents consecutive array of bytes from data object. It is created by data provider as an result of DataRead call. It specifies offset of the first byte it contains and number of bytes it contains.
This is interface implemented by data provider.
To avoid reading data size in advance when processing the data, it is possible to read chunks repeatedly, until returned chunk is NULL.
Provider may return ERR_UNIMPLEMENTED error, in such case the system will try to compute the size by reading all data.
The chunk can start at an arbitrary offset and be of size chosen by the data provider. If the offset is bigger then the size of the data, NULL chunk is returned.
The pointer to the requested byte can be obtained using 'chunk->buf.data + (offset - chunk->offset)'.
Chunk is an object and must be released using ObjFree method when no longer required.
The section of data specified by offset and size will be replaced by the contents of specified data.
If the size = 0, the new_data will be inserted at specified offset. If the new_data is NULL or empty (it's size is 0), the specified section is deleted.