I'm trying to write a python script that executes a local ansible playbook using the ansible.executor.playbook_executor.PlaybookExecutor class like the second answer of this question. I'm using python 3.11.4
The script looks like this:
from ansible import context
from ansible.module_utils.common.collections import ImmutableDict
from ansible.executor.playbook_executor import PlaybookExecutor
from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
loader = DataLoader()
context.CLIARGS = ImmutableDict(tags={}, listtags=False, listtasks=False, listhosts=False, syntax=False, connection='ssh',
module_path=None, forks=100, remote_user='bitv', private_key_file="/Users/an/.ssh/ssh_rsa_keys",
ssh_common_args=None, ssh_extra_args=None, sftp_extra_args=None, scp_extra_args=None, become=True,
become_method='sudo', become_user='root', verbosity=True, check=False, start_at_task=None)
inventory = InventoryManager(loader=loader, sources='/Users/an/Documents/Repositories/dlamp-stack/inventories/simple_lamp_stack/ansible01')
variable_manager = VariableManager(loader=loader, inventory=inventory)
pbex = PlaybookExecutor(playbooks=['/Users/an/Documents/Repositories/dlamp-stack/test.yml'], inventory=inventory, variable_manager=variable_manager, loader=loader, passwords={})
results = pbex.run()
and I get the following error when executing:
[WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details
Traceback (most recent call last):
File "/Users/an/Documents/Projekte/ansible-java-client/src/main/playbook-exec.py", line 18, in <module>
results = pbex.run()
^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/executor/playbook_executor.py", line 111, in run
pb = Playbook.load(playbook_path, variable_manager=self._variable_manager, loader=self._loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/__init__.py", line 52, in load
pb._load_playbook_data(file_name=file_name, variable_manager=variable_manager)
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/__init__.py", line 107, in _load_playbook_data
entry_obj = Play.load(entry, variable_manager=variable_manager, loader=self._loader, vars=vars)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/play.py", line 157, in load
return p.load_data(data, variable_manager=variable_manager, loader=loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/base.py", line 171, in load_data
setattr(self, name, method(name, ds[name]))
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/play.py", line 198, in _load_pre_tasks
return load_list_of_blocks(ds=ds, play=self, variable_manager=self._variable_manager, loader=self._loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/helpers.py", line 68, in load_list_of_blocks
Block.load(
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/block.py", line 92, in load
return b.load_data(data, variable_manager=variable_manager, loader=loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/base.py", line 171, in load_data
setattr(self, name, method(name, ds[name]))
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/block.py", line 120, in _load_block
return load_list_of_tasks(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/helpers.py", line 323, in load_list_of_tasks
t = Task.load(task_ds, block=block, role=role, task_include=task_include, variable_manager=variable_manager, loader=loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/task.py", line 136, in load
return t.load_data(data, variable_manager=variable_manager, loader=loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/base.py", line 161, in load_data
ds = self.preprocess_data(ds)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/playbook/task.py", line 204, in preprocess_data
(action, args, delegate_to) = args_parser.parse()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/parsing/mod_args.py", line 307, in parse
context = action_loader.find_plugin_with_context(item, collection_list=self._collection_list)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/loader.py", line 586, in find_plugin_with_context
result = self._resolve_plugin_step(name, mod_type, ignore_deprecated, check_aliases, collection_list, plugin_load_context=plugin_load_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/loader.py", line 682, in _resolve_plugin_step
return self._find_plugin_legacy(name, plugin_load_context, ignore_deprecated, check_aliases, suffix)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/loader.py", line 787, in _find_plugin_legacy
return self._find_fq_plugin(fq_name=candidate_fqcr, extension=suffix, plugin_load_context=plugin_load_context, ignore_deprecated=ignore_deprecated)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/loader.py", line 476, in _find_fq_plugin
routing_metadata = self._query_collection_routing_meta(acr, plugin_type, extension=extension)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/loader.py", line 434, in _query_collection_routing_meta
collection_pkg = import_module(acr.n_python_collection_package_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ansible_collections.ansible.builtin'
And these python packages are installed:
an@lpt02 ansible % pip3 list
Package Version
-------------- -------
ansible 8.2.0
ansible-core 2.15.2
ansible-runner 2.3.3
cffi 1.15.1
cryptography 41.0.2
docutils 0.20.1
Jinja2 3.1.2
lockfile 0.12.2
MarkupSafe 2.1.3
packaging 23.1
pexpect 4.8.0
pip 23.2.1
ptyprocess 0.7.0
pycparser 2.21
python-daemon 3.0.1
PyYAML 6.0
resolvelib 1.0.1
setuptools 67.6.1
six 1.16.0
wheel 0.40.0
There is no ansible_collections package listed so I went to /usr/local/lib/python3.11/site-packages and found it there: drwxr-xr-x 52 an admin 1664 28 Jul 16:02 ansible_collections.
Then in /usr/local/lib/python3.11/site-packages/ansible_collections/ansible (following ansible_collections.ansible.builtin) there is no builtin, probably resulting in the ModuleNotFoundError:
an@lpt02 ansible % pwd && ls -la
/usr/local/lib/python3.11/site-packages/ansible_collections/ansible
total 0
drwxr-xr-x 6 an admin 192 28 Jul 16:02 .
drwxr-xr-x 52 an admin 1664 28 Jul 16:02 ..
drwxr-xr-x 16 an admin 512 28 Jul 16:02 netcommon
drwxr-xr-x 17 an admin 544 28 Jul 16:02 posix
drwxr-xr-x 19 an admin 608 28 Jul 16:02 utils
drwxr-xr-x 10 an admin 320 28 Jul 16:02 windows
Also I can't find a way to install the missing module.
The test.yml looks like this:
- hosts: db-server
become: yes
become_user: root
gather_facts: false
tasks:
- name: i was here
file:
path: /home/bitv/iWasHere.txt
state: touch
What can I do to solve this problem?
I believe the issue is that the ansible.builtin collection is missing even though you have ansible installed. The builtin modules like file, service, etc are now part of ansible.builtin collection.
To fix this, you need to install the ansible collections base package:
pip install ansible-base
This will install the ansible.builtin collection along with other base collections that ansible depends on.
You could also just install the ansible.builtin collection alone:
ansible-galaxy collection install ansible.builtin
After installing the collection, the ModuleNotFoundError should go away when trying to run your playbook.
The other option is to downgrade to ansible 2.9 or earlier, where modules were not packaged into collections. But upgrading and installing the collections would be the recommended solution.