In my particular case I'm using Rackspace CloudFiles with sorl-thumbnails. It seems to download images from CloudFiles slowly. I have 1 worker for handling requests and another one for celery tasks.
Looked for existing solutions and it seems there is no such one at the moment.
Maybe I missed something? How should it be done the right way?
This isn't going to solve your problem, but there are some things to note/think about:
Remote object storage (Amazon's S3, Rackspace's CloudFiles) is going to be slower than local filesystem access. This depends on what you're doing of course and who's fetching the thumbnail. For users, downloading from a CDN is going to be faster than from a server. It may serve you well to do the thumbnail creation locally on an SSD backed server then upload to CloudFiles, distributing it over the CDN. Rackspace now has beefier SSD based instances with much greater IOPS.
The sorlery module takes great care to queue thumbnail creation with Celery (for use with remote object storage) and avoid filesystem access.
On another note, sorl-thumbnail hasn't seen development in over a year with LOTS of pull requests and issues sitting out on GitHub. Have you thought about using easy-thumbnails with django-cumulus?