Why so? I really can't understand that. Why we can only select from numbers proposed by players?
The numbers used are spaced far apart on purpose (typically from the Fibonacci sequence). If you get numbers from all across the board from 1 to 23, you're supposed to ask why the person who voted 1 gave it such a low score ("Did you think about testing and deployment? What about these other acceptance criteria?") and why the person who voted 23 gave it such a high score ("Are you aware we already have an existing code base? Did you know that Karen knows a lot about this and can pair up with you?") and then re-vote. If you're really stuck because half the team says 8 and the other half says 13, you can take the 13 and move on with your lives.
The extra precision isn't necessary when your accuracy is not great. My team goes for even less precision and buckets stories into "small" (one person can do a bunch in an iteration), "medium" (one person can handle a few of these), "large" (one person a week or more), and "extra large" (too big and needs to be split).