here is my problem, please see video: https://www.youtube.com/watch?v=UFXye1mMr04
When biggest body pushes my rope - it becomes crazy. I need to use mousejoint in my game and user can grab big object and hit that rope. If I increase size of each rope element I can fix this, however I need even less size objects in my game. And I need them stable. I know that box2d works not very well with small size objects, but I see in some games a thin ropes and all works well. Can someone please help me with my problem?
Check out the RopeJoint in the Box2D testbed. Okay, here's another hint. Enable the Joints display. See how there's more than just a bunch of revolute joints.
Okay, okay I'll explain.
The revolute joint is not designed to be the best at handling stretching. You can see as you disturb the rope with the huge box that the first few links of rope are spazzing like mad as they try to correct themselves. They are in distress this way because great forces and torques are being applied, and I guess the details of what's precisely gone wrong are rather difficult to predict but basically you have produced a pathological situation for the constraint system.
What's happening is that the individual corrective forces being computed per timestep are too big and there is an inefficiency in propagating the "information" consisting of "the last link needs to be closer for the rope to be stable", because as far as the engine is concerned, this information propagates through one link per timestep (or perhaps more accurately, one link per constraint iteration, of which a constant number of these is performed per timestep).
So for example, box2d goes:
... and it goes on and on, of course in reality it's a lot more complicated and terrifying than that. But you can see how by the time we get to link Z the total corrective force might have dissipated to something negligible. Pretty much what's seen in the video. This is what will happen to the naive solver system when you forcibly violate constraints like you did.
Alright so what's the solution?
Well the easy fix is to put a max-distance (box2d calls it a rope joint) joint on the last link. In fact, in the testbed app it even implements a keybind toggle that lets you toggle that rope joint with the J key, and turning it off produces exactly the problem you demonstrated.
This max-distance rope joint makes a rope that behaves much better in the situation because it is a constraint that explicitly enforces one of the invariants of a real rope rather than spreading out the work of conserving the length of the rope across as many different constraints as there are links.
As a result, it will not allow you to stretch it. You will still be able to temporarily disturb it hugely with a massive freaking box, as demonstrated, but it will bounce back much more readily because it knows to keep the other end from going far away where it should never be able to go.
It will definitely also help to have more constraints as well, such as the middle link can't be further away than half the length of the rope, or, screw it, just put a max-dist constraint between every single link and the rope anchor, make sure it's limits are exactly where they should be given the placement of the links, then you can go nuts on it with the box and you'll get pretty much zero weirdness. The tradeoff there is you'll have twice as many total joint constraints.