Is Implementing IDTExtensibility2 And Registering An Add-In Not Enough For It To Be Visible To Office?

So, I am trying to create a COM Add-In for 64-bit MS Office (no application in particular, just trying to get something working). I am not trying to make an add-in for the VBE, just something for the Office application itself. I have implemented IDTExtensibility2 like this (top of the file):

<Guid("94164866-CD9D-497A-9A8B-B476BE39749F"), 
ProgId("COM_Add-In_Test.Connection"), 
ComDefaultInterface(GetType(IDTExtensibility2)), 
ClassInterface(ClassInterfaceType.None), ComVisible(True)>
Public Class Connection
    Implements IDTExtensibility2

I have added registry entries under HKEY_CURRENT_USER\SOFTWARE\Microsoft\Office\Excel\Addins\COM_Add-In_Test.Connection (FriendlyName, Description, and LoadBehavior).

The add-in is automatically registered for COM-Interop by Visual Studio (the box is checked). I have also tried adding registry entries manually under HKCU\Classes\CLSID{94164866-CD9D-497A-9A8B-B476BE39749F}, but to no avail.

When I load up Excel, the add-in is not in the COM Add-Ins dialog box and nothing happens (My OnConnection method is MsgBox("Hello World!")).

I am not using any add-in framework of any kind (VSTO, ExcelDNA, etc). I have used these before, but would very much like to understand how to do this process manually.

What am I missing here?

So, after doing some research, this is what I found:

1. Don't have Visual Studio register COM interop classes for you.
2. Use the RegAsm tool with the /reg argument to have it generate a .reg file for you.
3. Edit the .reg file and replace references to HKEY_CLASSES_ROOT with references to HKEY_LOCAL_MACHINE\SOFTWARE\Classes, if you don't want to require admin rights to install. Example:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}]
@="YOUR_PROG_ID"

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}\Implemented Categories]

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}\Implemented Categories\{62C8FE65-4EBB-45e7-B440-6E39B2CDBF29}]

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}\InprocServer32]
@="mscoree.dll"
"ThreadingModel"="Both"
"Class"="YOUR_PROG_ID"
"Assembly"="YOUR_ASSEMBLY_FULL_NAME"
"RuntimeVersion"="v4.0.30319"
"CodeBase"="file:///PATH_TO_YOUR_DLL"

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}\InprocServer32\1.0.0.0]
"Class"="YOUR_PROG_ID"
"Assembly"="YOUR_ASSEMBLY_FULL_NAME"
"RuntimeVersion"="v4.0.30319"
"CodeBase"="file:///PATH_TO_YOUR_DLL"

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{YOUR_GUID}\ProgId]
@="YOUR_PROG_ID"

4. Use add-in spy (available here: https://github.com/NetOfficeFw/AddInSpy) to help diagnose problems and monitor your progress along the way.