[SOLVED] How to search list items by regex with Aerospike Expressions

How to search list items by regex with Aerospike Expressions

We are migrating from our old aerospike version 3.15.1.4 to version 6.2.0.7. This involves the migration from aerospike-client-go v4.5.2+incompatible to aerospike-client-go/v6 v6.6.0. Predicate Expressions (PredExp) have been deprecated since Aerospike Database 5.2. They were removed in Aerospike Database 6.0. So we have to move to Aerospike Expressions.

We have a bin with a string-list in it. And we want to do a search with a "begins-with" pattern. In the old world, we do that with following code:

as.NewPredExpStringVar("v"),
as.NewPredExpStringValue("^"+regexp.QuoteMeta("search_term")),
as.NewPredExpStringRegex(QueryRegexOptionExtended|QueryRegexOptionIgnoreCase),
as.NewPredExpListBin("ListField"),
as.NewPredExpListIterateOr("v"),

Following examples will search vor exact value in the "new world":

// As filter inside statement
as.NewContainsFilter("ListField", as.ICT_LIST, "search_term")

// As FilterExpression on query-policy
*as.ExpGreater(
    as.ExpListGetByValue(
        as.ListReturnTypeCount,
        as.ExpStringVal("search_term"),
        as.ExpListBin("ListField")),
    as.ExpIntVal(0),

But how can we achive a "begins-with-search" with the new Aerospike Expressions?

We tried the following, without success:

*as.ExpRegexCompare(
    "^"+regexp.QuoteMeta("search_term"),
    as.ExpRegexFlagICASE,
    as.ExpListBin("ListField"),
)

In the aerospike-logs, the following warning appears:

WARNING (exp): (exp.c:1469) build_cmp_regex - error 4 invalid arg type 4 (list) != 3 (str)
WARNING (query): (query.c:1485) basic query job failed msg field processing

For more background-info, this is the db-setup:

We have following model which is getting stored inside aerospike:

type MyModel struct {
    ListField []string
    [some-other-fields]
}

And following secondary index: Index name: idx_MYSET_ListField / namespace: myns / set: MYSET / bin: ListField / bin: string / index: list / State: RW

And if I read a sample record over aql:

aql> select PK,ListField from myns.MYSET where PK = 12345
+--------------------+-------------------------------------------------------------------+
| PK | ListField |
+--------------------+-------------------------------------------------------------------+
| 12345 | LIST('["myid:1234", "testid2", "some-other-value", "more-values"]') |
+--------------------+-------------------------------------------------------------------+

Solution

Expressions do not support iteration over lists or maps. The list and map APIs normally provide functionality that would solve the use-cases using the iteration provided by the old predexp. This is the first use-case I've come across that used iteration in a way that cannot be replicated with the list/map APIs.

As of 6.3, the only way to replicate this behavior may be to use a Stream UDF.