Ambari User & Group Management for Custom Ambari Services

I have been working with Custom Ambari Services for quite some time. I have been able to install several different custom components. I have created several management packs and consider myself very experienced in making third party services work in Ambari.

Whenever I install a custom service I get a user KeyError, for example Elasticsearch:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 38, in <module>
    BeforeAnyHook().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 31, in hook
    setup_users()
  File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 50, in setup_users
    groups = params.user_to_groups_dict[user],
KeyError: u'elasticsearch'
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-15.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-15.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']

A known work around is to execute a python command to turn off user/group management:

python /var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k  ignore_groupsusers_create -v true

However, this leaves the cluster in an undesirable state if you want to install native services again. If I execute the python command to turn user/group management back on, the next native service install will again fail on the third party user key object error.

Is there a database table that contains the list or key value object of users and groups that ambari manages? Satisfying the original error seems like the only turnkey solution.

I have tried to locate the key value object myself, I have also tried creating the users groups, I have even tried modifying the agent/server code executing the install. Next I will try more but I thought maybe this would be a good first post for SO.

Stuck with the same error for a few hours, here is the result of the investigation.

First of all, we need to know that the Ambari has one main group for all services in stack. Secondly, the creation of the user is quite hidden with one look you will never guess when and where the creation will be and from where the parameters will come.

And for the last there is quiestion, how we setup the params.user_to_groups_dict[user]?

The 'main group' is set in <stack_name>/<stack_version>/configuration/cluster-env.xml, for me it was HDP/3.0/configuration/cluster-env.xml:

  <property>
    <name>user_group</name>
    <display-name>Hadoop Group</display-name>
    <value>hadoop</value>
    <property-type>GROUP</property-type>
    <description>Hadoop user group.</description>
    <value-attributes>
      <type>user</type>
      <overridable>false</overridable>
    </value-attributes>
    <on-ambari-upgrade add="true"/>
  </property>

That parameter will be used everywhere in services to claim the group, for example zookeeper has the env.xml such as:

  <property>
    <name>zk_user</name>
    <display-name>ZooKeeper User</display-name>
    <value>sdp-zookeeper</value>
    <property-type>USER</property-type>
    <description>ZooKeeper User.</description>
    <value-attributes>
      <type>user</type>
      <overridable>false</overridable>
      <user-groups>
        <property>
          <type>cluster-env</type>
          <name>user_group</name>
        </property>
      </user-groups>
    </value-attributes>
    <on-ambari-upgrade add="true"/>
  </property>

And there is a magic in value-attributes: user-groups with one property that links to the cluster-env to user_group parameter. This is the connection that we are looking for.

The answer is setup the user parameter like zookeeper user.

The Wizard searches the stack and find the right users/groups to manage of the services you have chosen with wizard.

The map that contains the params.user_to_groups_dict will be created in runtime the cluster wizard and avalialbe at /var/lib/ambari-agent/data/command-xy.json:

    "clusterLevelParams": {
        "stack_version": "3.0",
        "not_managed_hdfs_path_list": "[\"/tmp\"]",
        "hooks_folder": "stack-hooks",
        "stack_name": "HDP",
        "group_list": "[\"sdp-hadoop\",\"users\"]",
        "user_groups": "{\"httpfs\":[\"hadoop\"],\"ambari-qa\":[\"hadoop\",\"users\"],\"hdfs\":[\"hadoop\"],\"zookeeper\":[\"hadoop\"]}",
        "cluster_name": "test",
        "dfs_type": "HDFS",
        "user_list": "[\"httpfs\",\"hdfs\",\"ambari-qa\",\"sdp-zookeeper\"]"
    },