So i'm curious, is there any info on a router's config page or manual, or is there any way to measure it's buffer? (I'm talking about the "memory" it has to keep packets until the router can transmit them)
You should be able to measure your outgoing buffer by sending lots of (UDP) data out, as quickly as possible, and see how much goes out before data starts getting dropped. You will need to send it really fast, and have something at the other end to capture it as it comes in; your send speed has to be a lot faster than your receive speed.
Your buffer will be a little smaller than that, since in the time it takes you to send the data, at least the first packet will have left the router.
Note that you will be measuring the smallest buffer between you and the remote end; if you have, for example, a firewall and a router in two separate devices, you don't really know which buffer you are testing.
Your incoming buffer is a lot more difficult to test, since you can't fill it fast enough (from the internet side) not to be deliverable quickly enough.