javaspring-bootredisresponse-time

Multi-Field Querying on Redis Using Redis Spring


this will be a very baic question since im new to Spring-Redis

Im currently in the process of learning about Redis database and I'm working on a feature on priority, im compelled to Use Redis for this feature. Below in the challenge/Query im having.

Right now we have a DataModel as below:

@RedisHash("Org_Work")
public class OrgWork {

   private @Id @Indexed UUID id;
   private @Indexed String CorpDetails;
   private @Indexed String ContractType;
   private @Indexed String ContractAssigned;
   private @Indexed String State;
   private @Indexed String Country; 

}
public interface OrgWorkRepository extends CrudRepository<HoopCalendar, String> {

List<OrgWork> findByCorpDetailsAndContractTypeAndStateAndCountry(String CorpDetails, String ContractType, String ContractAssigned, String State, String Country);

}

we are developing an API to query on the above Datamodel where the front-end will send us CorpDetails ,ContractType, ContractAssigned, State and Country fields and we have to query these against the Redis Database and return back the DurationOfWork object.

In this case I will be having a load of approx. 100000 calls per minute.

Please let me know on if this is the right way and some suggestions on improving response time.

***Updated the query


Solution

  • See Spring Data Redis - 8.5. Secondary Indexes and:

    The annotation @Indexed instructs Spring Data Redis (SDR) to create a secondary indexed as a set to index the field of the hash.

    This means when you insert data, SDR will run seven commands to Redis:

    HMSET "OrgWork:19315449-cda2-4f5c-b696-9cb8018fa1f9" "_class" "OrgWork" 
        "id" "19315449-cda2-4f5c-b696-9cb8018fa1f9" 
        "CorpDetails" "CorpDetailsValueHere" "ContractType" "ContractTypeValueHere" 
        ... "Country" "Costa Rica"
    SADD  "OrgWork" "19315449-cda2-4f5c-b696-9cb8018fa1f9"                           
    SADD  "OrgWork:CorpDetails:CorpDetailsValueHere" "19315449-cda2-4f5c-b696-9cb8018fa1f9"
    SADD  "OrgWork:ContractType:ContractTypeValueHere" "19315449-cda2-4f5c-b696-9cb8018fa1f9"
    ...
    SADD  "OrgWork:Country:Costa Rica" "19315449-cda2-4f5c-b696-9cb8018fa1f9"
    

    Using Query by Example:

    You want to create a repository:

    interface OrgWorkRepository extends QueryByExampleExecutor<OrgWork> {
    }
    

    And then implement the query as in the example service below:

    class OrgWorkService {
    
      @Autowired OrgWorkRepository orgWorkRepository;
    
      List<OrgWork> findOrgWorks(OrgWork probe) {
        return orgWorkRepository.findAll(Example.of(probe));
      }
    }
    

    And use as:

    OrgWork orgWorkExample = new OrgWork();                          
    orgWorkExample.setCorpDetails("CorpDetailsValueHere"); 
    orgWorkExample.setContractType("ContractTypeValueHere");
    ...
    List<OrgWork> results = orgWorkService.findOrgWorks(orgWorkExample);
    

    Behind the scenes, SDR will take care of converting this to Redis commands to get your data, using a combination of SINTER and HGETALL:

    SINTER   …:CorpDetails:CorpDetailsValueHere   …:ContractType:ContractTypeValueHere   ...
    HGETALL "OrgWork:d70091b5-0b9a-4c0a-9551-519e61bc9ef3" 
    HGETALL ...
    

    This is a two-step process:

    1. Fetch keys contained in the intersection of secondary indexes, using SINTER
    2. Fetch each key returned by <1> individually, using HGETALL

    A workload of 100,000 per minute should be manageable for Redis assuming you have a quality server, a reasonable dataset size, and the queries are somewhat specific in average.

    SINTER has a time complexity of O(N*M) worst-case where N is the cardinality of the smallest set and M is the number of sets. You have one set for every dimension on your query.

    HGETALL is O(N) where N is the size of the hash, 7 in your case.

    As always, it is recommended you do some benchmarking to test if you're getting the desired performance and adjust if needed.