java data-structures redis memory-optimization

Redis memory optimization for hashmap vs String keys

I have been trying to evaluate the memory size of the redis in two scenario.

Scenario-1:

    String commonString = "Test";
    long commonId = 1234567890L;
    int size = 0;
    for(int i = 1; i <= 100; i++)
    {
        jedis.set((commonString+"_"+commonId+"_KEY_"+i).getBytes(),("Value_"+i).getBytes());
        DebugParams debugParams = DebugParams.OBJECT(commonString+"_"+commonId+"_KEY_"+i);
        size += Integer.parseInt(debugged.split(":")[4].split(" ")[0]);
    }

In this scenario, the size is 892 bytes

Scenario-2:

    String commonString = "Test";
    long commonId = 1234567890L;
    int size = 0;
    for(int i = 1; i <= 100; i++)
    {
        jedis.hset((commonString+"_"+commonId).getBytes(),SafeEncoder.encode("KEY_"+i),("Value_"+i).getBytes());
    }
    DebugParams debugParams = DebugParams.OBJECT(commonString+"_"+commonId);
    String debugged = jedis.debug(debugParams);
    size = Integer.parseInt(debugged.split(":")[4].split(" ")[0]);

In this scenario, it takes 925 bytes.

Since Redis 2.2 many data types are optimized to use less space up to a certain size. Hashes, Lists, Sets composed of just integers, and Sorted Sets, when smaller than a given number of elements, and up to a maximum element size, are encoded in a very memory efficient way that uses up to 10 times less memory (with 5 time less memory used being the average saving).

I have tried for this optimization, but i haven't achieved it. The configuration is default configuration. What might have gone wrong?

Edited

Scenario-1

    String commonString = "Test";
    long commonId = 1234567890L;
    jedis.flushAll();
    String info1 = jedis.info();
    for(int i = 1; i <= 75; i++)
    {
        jedis.set(commonString+"_"+commonId+"_KEY_"+i, "VALUE_"+i);
    }
    String info2 = jedis.info();

In Scenario-1, used memory in info1 is "used_memory:844640" and used memory in info2 is "used_memory:852176". So, the difference between the info2 and info1 is 7536 bytes.

Scenario-2

    String commonString = "Test";
    long commonId = 1234567890L;
    jedis.flushAll();
    String info1 = jedis.info();
    for(int i = 1; i <= 75; i++)
    {
        jedis.hset(commonString+"_"+commonId,"KEY_"+i,"VALUE_"+i);
    }
    String info2 = jedis.info();

In Scenario-2, used memory in info1 is "used_memory:845576" and used memory in info2 is "used_memory:847208". So, the difference between the info2 and info1 is 1632 bytes.

Have I evaluated correctly?

Solution

If you refer to the documentation about DEBUG OBJECT, you'll learn that:

DEBUG OBJECT is a debugging command that should not be used by clients. Check the OBJECT command instead.

Put a little differently, whatever you're getting back from DebugParams.OBJECT is in most likelihood almost if not entirely irrelevant to what you are trying to achieve.

The good news are that Redis v4 introduces the new MEMORY (Doctor) command. The following is an excerpt from @antirez's 'The first release candidate of Redis 4.0 is out':

6. The new MEMORY command.

I love it, as much as I loved LATENCY DOCTOR that once introduced cut the percentage of “My Redis is slow” complains in the mailing list to a minor fraction. Now we have it for the memory issues as well.
127.0.0.1:6379> MEMORY DOCTOR
Hi Sam, this instance is empty or is using very little memory, my issues detector can't be used in these conditions. Please, leave for
your mission on Earth and fill it with some data. The new Sam and I will be back to our programming as soon as I finished rebooting.

Movie rights owners will probably sue me for taking inspirations of sci-fi dialogues but that’s fine. Bring oranges for me when I’ll be in jail.

MEMORY does a lot more than that.
127.0.0.1:6379> MEMORY HELP
1) "MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key"
2) "MEMORY STATS                         - Show memory usage details"
3) "MEMORY PURGE                         - Ask the allocator to release memory"
4) "MEMORY MALLOC-STATS                  - Show allocator internal stats"
The memory usage reporting of the USAGE subcommand is going to be very useful, but also the in depth informations provided by “STATS”.

For now all this is totally not documented, so have fun figuring out what the heck it does.