So I am building an app witch returns the users in nearby.
This functionality is provided by the GeoFlutterFire library.
The problem is I don't want to let some guy to just dump all of the locations from the database with a single quarry.
So how can I limit the max range of the quarry with the firestore security rules?
You won't be able to do this with security rules. The rules you use to allow access to a collection for any GeoFire query will have to allow read access to all documents. Once you allow that, you are allowing anyone to read all documents in the entire collection.
If you need to control the specific parameters to a query, you will need to force the user through a backend that will check the arguments, place limits on the query, and only return as many documents as they are allowed. That said, someone could still call that backend with lots of different paramters to try to get all the documents, so all you've really done is make it more difficult.