http://www.mjmwired.net/kernel/Documentation/IO-mapping.txt
153 - remapping and writing:
154 /*
155 * remap framebuffer PCI memory area at 0xFC000000,
156 * size 1MB, so that we can access it: We can directly
157 * access only the 640k-1MB area, so anything else
158 * has to be remapped.
159 */
160 void __iomem *baseptr = ioremap(0xFC000000, 1024*1024);
161
162 /* write a 'A' to the offset 10 of the area */
163 writeb('A',baseptr+10);
164
165 /* unmap when we unload the driver */
166 iounmap(baseptr);
167
Can one explain why this holds We can directly access only the 640k-1MB area?
Short answer: Because Linus decided to do it that way.
Long answer: Actually, it think it's wrong that you should be able to access even that area without a driver signing up for it. But never mind that.
In the old days, sometime in the 1990's, PCI didn't exist, and graphics cards, network cards, etc, were using something called ISA bus. It didn't have fancy features that allow you to request information from the card, or configure things onto the card with regards to where it thinks it's hardware addresses should be. All of the ISA memory is between A0000 (640K) and FFFFF (1MB-1). So, in the early days of Linux, this was where graphics and such things lived, and the kernel couldn't really know where these things were. Either way, we don't have these sorts of hardware these days, thanks to evolution of better hardware. Good riddance!
For compatibility reasons, this memory space is still used during boot, since until you get the drivers loaded and PCI hardware set up, it behaves in "legacy mode", so you can still run really old DOS and other old software on the machine without it behaving weird.
However, once your graphics card, for example, is configured, it will have BAR's (Bus Address Range) set up to tell the world where it is located in the bus address space (physical address). ioremap will map the physical address to a virtual address that you can use in the kernel - baseptr in this example. [I could go into much more detail about ioremap, as I've been working on some code derived from that recently].
As the other answer says, you will need to ask the device what it's BAR is (which is part of the PCI configuration space), and then map it's memory to a virtual address. You can see where things are using lspci -v|grep Memory (of course, using just lspci -v will give you a lot more information, including which device has what memory). These addresses are the physical addresses.