I understand that RFB protocol is used to remote display. X11VNC uses the RFB protocol so that any VNC viewer (RFB-based) can view the display.
x11vnc uses X11 requests to get your screen updates - via Composite/Damage extensions when available or just by doing GetImage requests at time interval and diffing it with local copy. You want to know not only current image of the screen at any point of time but also when it was changed and what area was affected. Also with x11vnc you can track individual window instead of whole screen - there is fair number of x11 features in addition to just rfb server.