I had used Appstats to record my ndb usage in my GAE project, and it worked very well. My Appstats setup was following this doc.
Recently, I move some ndb code to be executed on background thread, but those ndb calls don't show on Appstats console UI any more.
I have tried Appstats in both the dev and prod, they don't record ndb RPCs made in background thread.
To make the question clear, I means: Appstats works for:
class MyHandler(webapp2.RequestHandler):
def put(self):
...
do_a_lot_of_ndb_work()
...
but Appstats doesn't work for:
class MyHandler(webapp2.RequestHandler):
def put(self):
...
background_thread.start_new_background_thread(do_a_lot_of_ndb_work, [])
...
Can I change some parameters in appengine_config.py or do something to make Appstats works for both?
Update: The above code snippets are running in backend (basic_scaling, max_instances=1), and the thread usage is referenced from https://developers.google.com/appengine/docs/python/modules/#Python_Background_threads
You shouldn't be using threads this way. The correct way to execute functions that run longer than the 60sec request window is to use the Taskqueue API. That gives you a 10 minute window before the Task times out.
https://developers.google.com/appengine/docs/python/taskqueue/
If you really need to do even more processing than that, look into using backends.
https://developers.google.com/appengine/docs/python/backends/
If you're looking to run ndb calls asynchronously for performance, the tasklet decorators described here are excellent and highly recommended:
https://developers.google.com/appengine/docs/python/ndb/async
(The newest release of the SDK, 1.8.4, allows you to run transactions in a tasklet as well using the @transactional_tasklet decorator.)
I use all three of those methods for stuff that doesn't need to hold up the main request thread and appstats works well in all those cases.
You should also take a good look at what you're trying to do and see if it can be reasonably split up into smaller chunks, because it's probably going to cost a lot if you need more than 10 minutes to do the processing.