I have some question on the usability of HDFql:
The table I want to create will have any thing 1 to 2^n rows and cannot be determine before hand.
Here goes some info concerning your questions about HDFql:
If by "HDFql support created of a table where the row are of different type" you mean HDFql supports compound data type, the answer is not yet. (EDIT: since HDFql version 2.2.0 compound data types are now supported)
To append data into a dataset (where its size cannot be determined before hand), you have to go through several steps (I will assume that you are using the C programming language):
2.1. The dataset must be extendible. As an example, you can create an extendible dataset in HDFql as follows (this creates a dataset named dset
of integer data type with an unlimited size):
hdfql_execute("CREATE CHUNKED DATASET dset AS INT(UNLIMITED)");
2.2. Write a value in the last row of dataset dset
using a hyperslab as follows (replace my_value
with an integer that you would like to write into the dataset):
hdfql_execute("INSERT INTO dset[-1:1:1:1] VALUES(my_value)");
2.3. After writing a value to dataset dset
and if there are more values to write, first increase (i.e. alter) the dimension one unit like the following and then repeat step 2.2.:
hdfql_execute("ALTER DIMENSION dset TO +1");
dset
, you must first read it and then use function hdfql_cursor_get_int()
as follows: hdfql_execute("SELECT FROM dset");
while(hdfql_cursor_next(NULL) == HDFQL_SUCCESS)
{
printf("Value: %d\n", *hdfql_cursor_get_int(NULL));
}