Importmap Rails and Stimulus when working with Rails Engines

I'm unable to get stimulus and importmaps to read files from an app engine. The current project I've been tasked to do involves me upgrading all legacy engines to use importmaps and stimulus JS. The documentation is a bit misleading as it does not work the way any of the documentation lays out to use them.

Note :: I did not choose app structure, I'm working with what I was given and it is not my place to decide that the current structure is correct or incorrect, so please assume that this is my only option for structure.

Structure

Main App (Rails ==> gem 'Engine 1', gem 'Engine 2') only it works as a nested rails app.
-- Engine 1
-- Engine 2
-- Engine 3

The Main App Code

• The main app includes all the manifest files and has a working stimulus setup for the items that the main app controls.

The Engine Code

• When Using a <%= javascript_importmap_tags %> in the engine, the main apps controls are loading, but none of the engines javascript controllers. Documentation says that it needs an importmap and it needs its own manifest etc.

engines/myengine/assets/config/myengine/manifest.js

//= link_tree ../../images/appetite
//= link_directory ../../stylesheets/appetite .css
//= link_directory ../../stylesheets/appetite .scss
//= link_tree  ../../../javascript

engines/myengine/config/importmap.rb

pin_all_from "app/javascript/controllers", under: "controllers/myengine"

engines/myengine/lib/myengine/engine.rb

# engines/appetite/lib/appetite/engine.rb
module MyEngine
  class Engine < ::Rails::Engine
    isolate_namespace MyEngine
    initializer :importmap, before: :importmap do |app|
      app.config.importmap.paths << root.join('config/importmap.rb')
    end
  end
end

engine/.../example_controller.js

import { Controller } from "@hotwired/stimulus";

export default class extends Controller {
  connect() {
    console.log("Example controller connected");
  }
}

This usually causes a crash that says we need to make it a path, however if the importmap was working correctly it wouldn't say that.

import "controllers/myengine"

Eager loading within the controller also has not been working

eagerLoadControllersFrom("controllers/myengine", application);

I've tried lots of steps in between and even tried just linking pinning the files from the main app, but nothing seems to work. Any ideas and help would be appreciated.

A few things are missing from the engine configuration;

module MyEngine
  class Engine < ::Rails::Engine
    isolate_namespace MyEngine

    # `before:` option has to be a string as it has to match
    # a named initializer exaclty as a String or a Symbol
    # "importmap" is a string, you can see this list to check
    # the order of initializers:
    #   Rails.application.initializers.tsort.map(&:name)
    initializer "my_engine.importmap", before: "importmap" do |app|
      # https://github.com/rails/importmap-rails#composing-import-maps
      app.config.importmap.paths << root.join("config/importmap.rb")

      # https://github.com/rails/importmap-rails#sweeping-the-cache-in-development-and-test
      app.config.importmap.cache_sweepers << root.join("app/javascript")
    end

    initializer "my_engine.assets" do |app|
      # my_engine/app/javascript needs to be in the asset path
      app.config.assets.paths << root.join("app/javascript")

      # manifest has to be precompiled
      app.config.assets.precompile += %w[myengine/manifest.js]
    end
  end
end

Fix engine's importmap:

pin_all_from MyEngine::Engine.root.join("app/javascript/controllers"), under: "controllers"

The first argument has to be an absolute path, otherwise it is assumed to be relative to Rails.root. :under option is relative to myengine/app/javascript (which is an asset path).

Manifest:

//= link_tree  ../../../javascript

Controller is in my_engine/app/javascript/controllers/example_controller.js

Test:

$ bin/importmap json

{
  "imports": {
    "application": "/assets/application-b1b2b9a824f2a0f16175e1498e5f1ddf1a923fde046d6832e2e0a9526545ab04.js",
    "@hotwired/turbo-rails": "/assets/turbo.min-cd3ce4205eaa3eb1f80c30fedaf47bccb15a7668eb53b1cb1a5e0dda16009d4d.js",
    "@hotwired/stimulus": "/assets/stimulus.min-dd364f16ec9504dfb72672295637a1c8838773b01c0b441bd41008124c407894.js",
    "@hotwired/stimulus-loading": "/assets/stimulus-loading-3576ce92b149ad5d6959438c6f291e2426c86df3b874c525b30faad51b0d96b3.js",

    "controllers/example_controller": "/assets/controllers/example_controller-66b44365679431ba15f4a290f152a294517a1be03220a946ed215a6601e33a5e.js",

    "controllers/application": "/assets/controllers/application-368d98631bccbf2349e0d4f8269afb3fe9625118341966de054759d96ea86c7e.js",
    "controllers": "/assets/controllers/index-2db729dddcc5b979110e98de4b6720f83f91a123172e87281d5a58410fc43806.js"
  }
}