I have configured Appstats on my Java Appengine application and noticed that a single JDO Query which returns several objects results in a separate RunQuery RPC call for every object retrieved by the Query.
Shouldn't a Query be done in a single RPC call?
I've tried configuring Fetchgroups and Fetchplans to try to avoid this but to no avail.
My code is something like this:
Query query = pm.newQuery(WidgetDSO.class);
String filter = "widgetId == param1 || widgetId == param2 || widgetId == param3";
String parameters = "String param1, String param2, String param3";
query.setFilter(filter.toString());
query.declareParameters(parameters.toString());
List<WidgetDSO> results = (List<WidgetDSO>) query.executeWithArray(widgetIds);
if (!results.isEmpty())
...
When this runs, Appstats tells me that the last line results.isEmpty() results in as much RPC calls as objects retrieved:
@104ms datastore_v3.RunQuery real=5ms api=21ms
@422ms datastore_v3.RunQuery real=4ms api=12ms
@428ms datastore_v3.RunQuery real=4ms api=12ms
@434ms datastore_v3.RunQuery real=3ms api=12ms
@439ms datastore_v3.RunQuery real=4ms api=12ms
@445ms datastore_v3.RunQuery real=4ms api=12ms
@451ms datastore_v3.RunQuery real=4ms api=21ms
@463ms datastore_v3.RunQuery real=5ms api=21ms
The stack trace for each of these calls is the same (just a partial stack trace):
com.google.appengine.tools.appstats.Recorder:290 makeAsyncCall()
com.google.apphosting.api.ApiProxy:184 makeAsyncCall()
com.google.appengine.api.datastore.DatastoreApiHelper:81 makeAsyncCall()
com.google.appengine.api.datastore.PreparedQueryImpl:144 runQuery()
com.google.appengine.api.datastore.PreparedQueryImpl:70 asIterator()
com.google.appengine.api.datastore.PreparedMultiQuery$FilteredMultiQueryIterator:165 getNextIterator()
com.google.appengine.api.datastore.PreparedMultiQuery$FilteredMultiQueryIterator:184 computeNext()
com.google.appengine.api.datastore.PreparedMultiQuery$FilteredMultiQueryIterator:98 computeNext()
com.google.appengine.api.datastore.AbstractIterator:52 tryToComputeNext()
com.google.appengine.api.datastore.AbstractIterator:47 hasNext()
com.google.appengine.api.datastore.BasePreparedQuery$UncompilablePreparedQuery$1:86 hasNext()
org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterator$1:50 get()
org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterator$1:46 get()
org.datanucleus.store.appengine.query.QueryExceptionWrappers$1:51 get()
org.datanucleus.store.appengine.query.QueryExceptionWrappers$2:86 get()
org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterator:105 hasNext()
org.datanucleus.store.appengine.query.LazyResult:115 resolveAll()
org.datanucleus.store.appengine.query.LazyResult:110 size()
org.datanucleus.store.appengine.query.StreamingQueryResult:130 size()
org.datanucleus.store.query.AbstractQueryResult:312 isEmpty()
org.instantplaces.im.server.dso.WidgetDSO:209 getWidgetsFromDSO()
org.instantplaces.im.server.resource.WidgetResource:239 doDelete()
org.instantplaces.im.server.resource.GenericResource:244 delete()
Is there a way to fetch all the objects in just one call?
Is that your exact code? If so I would expect at least 3 queries, minimum. There is no native "||" ("OR") query in the datastore. JDO is forced to translate your query into one query for each option. For more info, see the docs and a related blog post. It isn't stated directly - you need to combine the fact that ||'s are turned into .contains(), and .contains() requires multiple gets.
|| is only legal in situations where the filters it separates can be combined into a single contains() filter:
and
The contains() operator also performs multiple queries, one for each item in the provided list value where all other filters are the same and the contains() filter is replaced with an equal-to filter.