I am not 100% sure how Inno Setup deals with the registry, but from what I can tell, it adds a few items to the registry even though I do not have any code under [Registry]. This is something we would like to eliminate if possible and we would also like it to not show up in the programs/features menu (which I'm assuming is directly related to the registry). I am unsure how to approach this problem because I can't see where Inno Setup is directly doing this.
There are two related directives:
Setting CreateUninstallRegKey to no prevents Inno Setup from creating uninstallation registry keys. So the application won't be listed in Add/Remove Programs control panel. It still can be uninstalled though by locating the unins000.exe (default name) in the application program folder and running it. Note that the registry key created by CreateUninstallRegKey=yes is basically a simple reference to the unins000.exe.
Setting Uninstallable to no disables possibility to uninstall the application at all. So not only it behaves as if CreateUninstallRegKey=no, but does not even "install" the unins000.exe.
So you want the CreateUninstallRegKey=no.