we made a character with a capsule collider and a ladder (smooth as a slide collider) and the character is not fixed in the middle of the stairs but just slips off.
I tried to create a physical material for the character (with friction force) did not help
When your character touches the ground you can deactivate the gravity affects for the rigidbody using Rigidbody.useGravity.