javamongodbjongo

Want to iterate through half of mongoDB and iterate through the rest of the half with another query


I'm getting this error:

Exception in thread "main" com.mongodb.MongoCursorNotFoundException: Query failed with error code -5 and error message 'Cursor 304054517192 not found on server mongodb2:27017' on server mongodb2:27017 at com.mongodb.operation.QueryHelper.translateCommandException(QueryHelper.java:27) at com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:215) at com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:103) at com.mongodb.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:46) at com.mongodb.DBCursor.hasNext(DBCursor.java:155) at org.jongo.MongoCursor.hasNext(MongoCursor.java:38) at com.abc.Generator.Generate(Generator.java:162) at com.abc.main.main(main.java:72)

which I assume is because the query ran for too long.
So I'm planning to query mongo using find() and iterate through half of the collections.
Then I want to use another find() query and iterate through the remaining half of the collections.

Could you help with how to directly place the cursor at the half'th position of the collection? The documentation does not seem to provide any functions for it.

I'm basically just using a find() and iterating thru a collection with 100000 records, while connected to a server via ssh.

MongoCollection history = jongo.getCollection("historyCollection");
MongoCursor<MyClass> allHistories = history.find().as(MyClass.class);

  //---Iterate thru all histories
  while (allHistories.hasNext()) {
  MyClass oneHistory = allHistories.next();
}

Solution

  • Solved it by having the Mongo collections ordered by ObjectId's that were timestamps. This way, I was able to use the greater than operator to find objectIDs and split the iterations.

    private MongoCursor<PersonDBO> ReadFewProfilesFromDB(final String objectIdAfterWhichToSearchFrom, final Integer FIND_LIMIT) {
    
        MongoCursor<PersonDBO> aBatchOfProfiles = null;
    
        try {
            if (objectIdAfterWhichToSearchFrom.equals(START_OBJECTID_OF_MONGO_BATCHES)) {
                aBatchOfProfiles = personProfile.find().limit(FIND_LIMIT).as(PersonDBO.class);
            } else {
                aBatchOfProfiles = personProfile.find("{_id: {$gt: #}}", new ObjectId(objectIdAfterWhichToSearchFrom)).limit(FIND_LIMIT).as(PersonDBO.class);
            }
        } catch(Exception e) {logger.error("Problem while trying to find {} personProfiles, starting from objectID {}. {}, {}", FIND_LIMIT, objectIdAfterWhichToSearchFrom, e.getMessage(), e.getCause());}
    
        if (aBatchOfProfiles == null) {
            logger.error("profiles collection is null. Nothing more to iterate OR there was an exception when finding profiles. If exception, there would be an error printed above.");
            return null;
        }         
    
        return aBatchOfProfiles;
    }