javaelasticsearchliferayportletliferay-7

Liferay 7.3.5GA6 custom index search and ddmFieldArray

I'm trying to develop a custom web content search portlet for Liferay 7.3.5 GA6, using SearchContext, IndexSearcherHelperUtil and all the other stuff.

I have some DDMStructure with different fields and, from what I've seen on the elasticsearch index, this fields are indexed in a nested document, like this:

"ddmFieldArray": [
{
"ddmFieldName": "ddm__text__37702__nome_it_IT",
"ddmValueFieldName": "ddmFieldValueText_it_IT",
"ddmFieldValueText_it_IT": "Nome esempio",
"ddmFieldValueText_it_IT_String_sortable": "nome esempio"
}
,
{
"ddmFieldName": "ddm__text__37702__descrizione_breve_it_IT",
"ddmValueFieldName": "ddmFieldValueText_it_IT",
"ddmFieldValueText_it_IT": "Esempio di descrizione breve da indicizzare",
"ddmFieldValueText_it_IT_String_sortable": "esempio di descrizione breve da indicizzare"
}
]

which is different from the old way I used to know, where the custom fields were indexed like ddm__[keyword/text]__[structure_id]__[field_name]

Now, I understand this different way of indexing is due to an improvement to avoid elastic issues (Limit of total fields has been exceeded) but...After executing the search, there is no ddmFieldArray in the com.liferay.portal.kernel.search.Document .getFields, so I'm unable to get the ddmstructure fields values from the elastic search index.

Here's the code:

long journalArticleClassId = ClassNameLocalServiceUtil.getClassNameId(JournalArticle.class.getName());

SearchContext searchContext = new SearchContext();
searchContext.setClassTypeIds(new long[] {journalArticleClassId});
searchContext.setCompanyId(companyId);
searchContext.setStart(QueryUtil.ALL_POS);
searchContext.setEnd(QueryUtil.ALL_POS);
        
BooleanQuery query = new BooleanQueryImpl();
            
        
MatchQuery approvedQuery = new MatchQuery(Field.STATUS, String.valueOf(WorkflowConstants.STATUS_APPROVED));

query.add(approvedQuery, BooleanClauseOccur.MUST.getName());


Hits resultHits = IndexSearcherHelperUtil.search(searchContext, query);

for (Document doc: resultHits.getDocs()) {          
    doc.getFields().forEach((k, v) -> _log.debug(k)); //No ddm structure field
}

Is this still an improvement or just an unexpected behaviour?

Any way to solve or extend this?

Thanks

Solution

  • The "ddmFieldArray" field is a nested field that it is not returned by default. You have to fetch it from the document source field that is stored in an internal "_source" field in Elasticsearch (see https://www.elastic.co/guide/en/elasticsearch/reference/7.9/search-fields.html )

    In order to do this in Liferay, you have to use some search methods that aren't available in the old portal-kernel search classes, you have to use the new search classes that are available in the portal-search-api module that is located in modules/apps/portal-search

    These are the changes you have to apply to your code:

        searchRequestBuilderFactory.builder(
        searchContext
    ).fetchSource(
        true
    ).build();

        /* Execute search */
    IndexSearcherHelperUtil.search(searchContext, query);
    
    /* Get results from search response */
    SearchResponse searchResponse = searchContext.getAttribute("search.response");
    List<SearchHit> resultHits = searchResponse.getSearchHits().getSearchHits();
    
    /* Iterate */
    for (SearchHit searchHit : resultHits)  {
        Document doc = searchHit.getDocument();
    
        ...your stuff...
    }

    You have some examples about fetching source and gettings SearchResponse in following Liferay classes:

    I have also implemented a groovy script example that you can execute it from: Control Panel => Server Administration => Script

    import com.liferay.registry.*;
import com.liferay.portal.kernel.search.*;
import com.liferay.portal.kernel.search.generic.*;
import com.liferay.portal.search.legacy.searcher.SearchRequestBuilderFactory;
import com.liferay.portal.search.searcher.SearchResponse;
import com.liferay.portal.search.hits.SearchHit;
import com.liferay.portal.search.document.Document;

/* Get SearchRequestBuilderFactory reference using RegistryUtil, because we cannot use "@Reference" in a groovy script */
Registry registry = RegistryUtil.getRegistry();
SearchRequestBuilderFactory searchRequestBuilderFactory = registry.getService(registry.getServiceReference(SearchRequestBuilderFactory.class.getName()));

/* Create SearchContext */
SearchContext searchContext = new SearchContext();
searchContext.setCompanyId(com.liferay.portal.kernel.util.PortalUtil.getCompany(actionRequest).getCompanyId());
searchContext.setStart(-1);
searchContext.setEnd(-1);

/* Line to fetch stored source of documents */
searchRequestBuilderFactory.builder(
    searchContext
).fetchSource(
    true
).build();

/* Get journal articles that are approved (status = 0) */
MatchQuery approvedQuery = new MatchQuery(Field.STATUS, String.valueOf(0));
MatchQuery journalArticleQuery = new MatchQuery("entryClassName", com.liferay.journal.model.JournalArticle.class.getName());

BooleanQuery query = new BooleanQueryImpl();
query.add(approvedQuery, BooleanClauseOccur.MUST.getName());
query.add(journalArticleQuery, BooleanClauseOccur.MUST.getName());

/* Execute search */
IndexSearcherHelperUtil.search(searchContext, query);

/* Get results from search response */
SearchResponse searchResponse = searchContext.getAttribute("search.response");
List<SearchHit> resultHits = searchResponse.getSearchHits().getSearchHits();

/* Iterate */
for (SearchHit searchHit : resultHits)  {
    Document doc = searchHit.getDocument();
    out.println("entryClassPK: " + doc.getValue("entryClassPK"));
    out.println("ddmFieldArray: " + doc.getValues("ddmFieldArray"));
    out.println("");
}

    import com.liferay.portal.kernel.search.*;
import com.liferay.portal.kernel.search.generic.*;
import com.liferay.portal.search.legacy.searcher.SearchRequestBuilderFactory;
import com.liferay.portal.search.searcher.SearchResponse;
import com.liferay.portal.search.hits.SearchHit;
import com.liferay.portal.search.document.Document;

import org.osgi.framework.Bundle;
import org.osgi.framework.BundleContext;

import com.liferay.portal.kernel.module.util.SystemBundleUtil;

/* Get SearchRequestBuilderFactory reference using the BundleContext, because we cannot use "@Reference" in a groovy script */
BundleContext bundleContext = SystemBundleUtil.getBundleContext();
SearchRequestBuilderFactory searchRequestBuilderFactory = bundleContext.getService(bundleContext.getServiceReferences(SearchRequestBuilderFactory.class.getName(), null)[0]);

/* Create SearchContext */
SearchContext searchContext = new SearchContext();
searchContext.setCompanyId(com.liferay.portal.kernel.util.PortalUtil.getCompany(actionRequest).getCompanyId());
searchContext.setStart(-1);
searchContext.setEnd(-1);

/* Line to fetch stored source of documents */
searchRequestBuilderFactory.builder(
    searchContext
).fetchSource(
    true
).build();

/* Get journal articles that are approved (status = 0) */
MatchQuery approvedQuery = new MatchQuery(Field.STATUS, String.valueOf(0));
MatchQuery journalArticleQuery = new MatchQuery("entryClassName", com.liferay.journal.model.JournalArticle.class.getName());

BooleanQuery query = new BooleanQueryImpl();
query.add(approvedQuery, BooleanClauseOccur.MUST.getName());
query.add(journalArticleQuery, BooleanClauseOccur.MUST.getName());

/* Execute search */
IndexSearcherHelperUtil.search(searchContext, query);

/* Get results from search response */
SearchResponse searchResponse = searchContext.getAttribute("search.response");
List<SearchHit> resultHits = searchResponse.getSearchHits().getSearchHits();

/* Iterate */
for (SearchHit searchHit : resultHits)  {
    Document doc = searchHit.getDocument();
    out.println("entryClassPK: " + doc.getValue("entryClassPK"));
    out.println("ddmFieldArray: " + doc.getValues("ddmFieldArray"));
    out.println("");
}

    In your code you should replace the RegistryUtil or BundleContext usage with the "@Reference" annotation.

    Let me know if you have any problem with my example.