Say you are implementing a simple SoC with a cpu, ram, rom, and a mmu to map the ram and rom into the cpu's address space. When instantiating the different components would it make more sense to do:
ram and rom are instantiated inside the mmu
top
cpu
mmu
ram
rom
or
all components are instantiated at the top level and wired together there
top
cpu
mmu
ram
rom
It's really a design question, to which the answer depends on your requirements. A few things to consider are:
Reuse
As is the case in many SoC projects, you may want to reuse blocks or subsystems from one project to the next. It is a good idea to group modules together depending on how they might be reused in a different context. This way, collateral other than the design RTL (e.g. testbenches, synthesis constraints, assertions, etc) can be reused as well.
Dependencies
This goes along with the considerations for reuse, but think about what the dependencies are between modules. In your example, would the mmu always require the RAM and ROM? If yes, this is an argument for instantiating them within the mmu. Would the mmu work with a different number of types of memories? If yes, this is an argument for instantiating the memories outside the mmu.
Verification Scope
In modern SoC designs verification takes more time and resources than the design itself. So structure the hierarchy in a way that helps and doesn't hinder the verification effort. For example, if the cpu in your example is to be verified by itself, the cpu should instantiate all the sub-components, wire them up internally, and only present the relevant cpu ports. That way, the verification environment (aka the testbench) does not have to duplicate all the connections between the cpu, mmu, and memories.
In general I think a hierarchical structure (the first of your two examples) makes the most sense and is the most common.