I would like to find a best way to read just the keys from an object list saved in Ignite cache. Below is the use case in the source code. The current code works for a small size of object list but fails or taking lots of time for a big dataset. I think failure is due to the below code loads entire dataset into the thick client memory and then find the keys. For my requirment, I just want to read the keys (subkeys) from every object in the object list.
BinaryFormatter bf = new BinaryFormatter();
var cache = ignite.GetOrCreateCache<int, IBinaryObject>("my-cache");
IBinary binary = cache.Ignite.GetBinary();
// Populate persons.
cache[1] = binary.GetBuilder("PersonType")
.SetField("Name", "James Wilson")
.SetField("CompanyId", -1)
.Build();
cache[2] = binary.GetBuilder("PersonType")
.SetField("Name", "Daniel Adams")
.SetField("CompanyId", -1)
.Build();
Console.WriteLine("**************************************/n/n");
Console.WriteLine("Count : " + ignite.GetCache<int, IBinaryObject>("my-cache").GetSize());
// The below line fails or taking lots of time for a very big dataset becuase I feel ignite load the entire data into thick client memory to get the keys.
// is there anyway just to read the keys instead of loading the full dataset
[use case :]
cache.Query(new Apache.Ignite.Core.Cache.Query.ScanQuery<string, IBinaryObject>()).ToList().ForEach(item => Console.WriteLine(item.));
Console.ReadLine();
I've also tried the below code but still takes lots of time, so I am not sure that's the best way
ICollection<string> _keys = new List<string>();
var cache = ignite.GetCache<string, IBinaryObject>(name).WithKeepBinary<string, IBinaryObject>();
using (var cursor = cache.Query(new Apache.Ignite.Core.Cache.Query.ScanQuery<string, IBinaryObject>()))
{
foreach (var entry in cursor)
{
_keys.Add(entry.Key);
}
}
The most efficient way to query only a part of cache entry is SQL.
An example of key only retrieval:
var ignite = Ignition.Start();
var queryEntity = new QueryEntity
{
KeyType = typeof(int),
ValueTypeName = "Person",
TableName = "People"
};
var configuration = new CacheConfiguration("my-cache", queryEntity);
var cache = ignite.GetOrCreateCache<int, object>(configuration).WithKeepBinary<int, IBinaryObject>();
cache[1] = ignite.GetBinary().GetBuilder("Person").SetField("Name", "A").SetField("Id", 1).Build();;
// SQL
var cursor = cache.Query(new SqlFieldsQuery("select _key from \"my-cache\".People"));
foreach (var row in cursor)
{
var key = (int)row[0];
Console.WriteLine(key);
}
// LINQ to SQL
var keys = cache.AsCacheQueryable().Select(e => e.Key);
foreach (var key in keys)
{
Console.WriteLine(key);
}