pythonpluginsimportdynamic-class-loaders

Dynamic class loading in Python 2.6: RuntimeWarning: Parent module 'plugins' not found while handling absolute import


I am working on a plugin system where plugin modules are loaded like this:

def load_plugins():
   plugins=glob.glob("plugins/*.py")
   instances=[]
   for p in plugins:
      try:
         name=p.split("/")[-1]
         name=name.split(".py")[0]
         log.debug("Possible plugin: %s", name)
         f, file, desc=imp.find_module(name, ["plugins"])
         plugin=imp.load_module('plugins.'+name, f, file, desc)
         getattr(plugin, "__init__")(log)
         instances=instances+plugin.get_instances()
      except Exception as e:
         log.info("Failed to load plugin: "+str(p))
         log.info("Error: %s " % (e))
         log.info(traceback.format_exc(e))
   return instances

The code works, but for each import statement in the plugin code i get a warning like this:

plugins/plugin.py:2: RuntimeWarning: Parent module 'plugins' not found while handling absolute import
  import os

No errors are reported for the main program code, and the plugins work.

Can somebody explain what the warning means and what I doing wrong. Do I need to create an empty plugins module separately and import it to keep python happy?


Solution

  • If the directory plugins were a real package (contained __init__.py fine), you could easily use pkgutils to enumerate its plugin files and load them.

    import pkgutil
    # import our package
    import plugins
    list(pkgutil.iter_modules(plugins.__path__))
    

    However, it can work without a plugin package anyway, try this:

    import pkgutil
    list(pkgutil.iter_modules(["plugins"]))
    

    Also it is possible to make a package that only exists at runtime:

    import types
    import sys
    plugins = types.ModuleType("plugins")
    plugins.__path__ = ["plugins"]
    
    sys.modules["plugins"] = plugins
    import plugins.testplugin
    

    However that hack that was mostly for fun!