ignitegridgain

Read keys only from Apache Ignite cache


I would like to find a best way to read just the keys from an object list saved in Ignite cache. Below is the use case in the source code. The current code works for a small size of object list but fails or taking lots of time for a big dataset. I think failure is due to the below code loads entire dataset into the thick client memory and then find the keys. For my requirment, I just want to read the keys (subkeys) from every object in the object list.

 BinaryFormatter bf = new BinaryFormatter();
            var cache = ignite.GetOrCreateCache<int, IBinaryObject>("my-cache");

            IBinary binary = cache.Ignite.GetBinary();

            // Populate persons.
            cache[1] = binary.GetBuilder("PersonType")
                .SetField("Name", "James Wilson")
                .SetField("CompanyId", -1)
                .Build();

            cache[2] = binary.GetBuilder("PersonType")
                .SetField("Name", "Daniel Adams")
                .SetField("CompanyId", -1)
                .Build();

            
           Console.WriteLine("**************************************/n/n");
           Console.WriteLine("Count : " + ignite.GetCache<int, IBinaryObject>("my-cache").GetSize());

           // The below line fails or taking lots of time for a very big dataset becuase I feel ignite load the entire data into thick client memory to get the keys.
           // is there anyway just to read the keys instead of loading the full dataset
   [use case :] 
   cache.Query(new Apache.Ignite.Core.Cache.Query.ScanQuery<string, IBinaryObject>()).ToList().ForEach(item => Console.WriteLine(item.));
            Console.ReadLine();

I've also tried the below code but still takes lots of time, so I am not sure that's the best way

ICollection<string> _keys = new List<string>();
        var cache = ignite.GetCache<string, IBinaryObject>(name).WithKeepBinary<string, IBinaryObject>();

        using (var cursor = cache.Query(new Apache.Ignite.Core.Cache.Query.ScanQuery<string, IBinaryObject>()))
        {
            foreach (var entry in cursor)
            {
                _keys.Add(entry.Key);
            }
        }

Solution

  • The most efficient way to query only a part of cache entry is SQL.

    An example of key only retrieval:

    var ignite = Ignition.Start();
    var queryEntity = new QueryEntity
    {
        KeyType = typeof(int),
        ValueTypeName = "Person",
        TableName = "People"
    };
    
    var configuration = new CacheConfiguration("my-cache", queryEntity);
    var cache = ignite.GetOrCreateCache<int, object>(configuration).WithKeepBinary<int, IBinaryObject>();
    
    cache[1] = ignite.GetBinary().GetBuilder("Person").SetField("Name", "A").SetField("Id", 1).Build();;
    
    // SQL
    var cursor = cache.Query(new SqlFieldsQuery("select _key from \"my-cache\".People"));
    
    foreach (var row in cursor)
    {
        var key = (int)row[0];
        Console.WriteLine(key);
    }
    
    // LINQ to SQL
    var keys = cache.AsCacheQueryable().Select(e => e.Key);
    
    foreach (var key in keys)
    {
        Console.WriteLine(key);
    }