endeca

backup_log_dir_for_component_Dgraph2 failed in baseline update. Permission denied error coming in Dgraph log

During baseline update, I am getting error as backup_log_dir_for_component_Dgraph2 failed.

1. Below is the error from baseline_update.out file

            Setting flag 'baseline_data_ready' in the EAC.
                1 file(s) moved.
        [06.28.16 05:26:02] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
        [06.28.16 05:26:02] INFO: Definition has not changed.
        [06.28.16 05:26:02] INFO: Starting baseline update script.
        [06.28.16 05:26:02] INFO: Acquired lock 'update_lock'.
.
.
.
more logs in between
.
.
.
        [06.28.16 05:26:17] INFO: [ITLHost] Starting component 'Forge'.
        [06.28.16 05:45:14] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Dgidx'.
        [06.28.16 05:45:15] INFO: [ITLHost] Starting component 'Dgidx'.
        [06.28.16 06:00:59] INFO: [MDEXHost] Starting shell utility 'cleanDir_local-dgraph-input'.
        [06.28.16 06:01:01] INFO: [MDEXHost] Starting shell utility 'rmdir_dgraph-input-old'.
        [06.28.16 06:01:03] INFO: [MDEXHost] Starting copy utility 'copy_index_to_host_MDEXHost'.
        [06.28.16 06:01:26] INFO: Applying index to dgraphs in restart group 'A'.
        [06.28.16 06:01:26] INFO: [MDEXHost] Starting shell utility 'mkpath_dgraph-input-new'.
        [06.28.16 06:01:27] INFO: [MDEXHost] Starting copy utility 'copy_index_to_temp_new_dgraph_input_dir_for_Dgraph1'.
        [06.28.16 06:01:59] INFO: [MDEXHost] Starting shell utility 'move_dgraph-input_to_dgraph-input-old'.
        [06.28.16 06:02:01] INFO: [MDEXHost] Starting shell utility 'move_dgraph-input-new_to_dgraph-input'.
        [06.28.16 06:02:02] INFO: [MDEXHost] Starting backup utility 'backup_log_dir_for_component_Dgraph1'.
        [06.28.16 06:02:03] INFO: [MDEXHost] Starting component 'Dgraph1'.
        [06.28.16 06:02:10] INFO: [MDEXHost] Starting shell utility 'rmdir_dgraph-input-old'.
        [06.28.16 06:02:12] INFO: Applying index to dgraphs in restart group 'B'.
        [06.28.16 06:02:12] INFO: [MDEXHost] Starting shell utility 'mkpath_dgraph-input-new'.
        [06.28.16 06:02:13] INFO: [MDEXHost] Starting copy utility 'copy_index_to_temp_new_dgraph_input_dir_for_Dgraph2'.
        [06.28.16 06:02:38] INFO: Stopping component 'Dgraph2'.
        [06.28.16 06:02:39] INFO: [MDEXHost] Starting shell utility 'move_dgraph-input_to_dgraph-input-old'.
        [06.28.16 06:02:40] INFO: [MDEXHost] Starting shell utility 'move_dgraph-input-new_to_dgraph-input'.
        [06.28.16 06:02:42]

     INFO: [MDEXHost] Starting backup utility 'backup_log_dir_for_component_Dgraph2'.
        [06.28.16 06:02:43] SEVERE: Utility 'backup_log_dir_for_component_Dgraph2' failed. Refer to utility logs in [ENDECA_CONF]/logs/archive on host MDEXHost.
        Occurred while executing line 5 of valid BeanShell script:
        [[
        2|      
        3|    DgraphCluster.cleanDirs();
        4|    DgraphCluster.copyIndexToDgraphServers();
        5|    DgraphCluster.applyIndex();
        6|     
        7|   
        ]]

        [06.28.16 06:02:43] SEVERE: Error executing valid BeanShell script.
        Occurred while executing line 35 of valid BeanShell script:
        [[
        32|        Dgidx.run();
        33|       
        34|        // distributed index, update Dgraphs
        35|        DistributeIndexAndApply.run();
        36|
        37|        // if Web Studio is integrated, update Web Studio with latest
        38|        // dimension values
        ]]

        [06.28.16 06:02:43] SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'. Releasing locks.

2. Below is the error from backup_log_dir_for_component_Dgraph2.log file (Filepath PlatformServices\workspace\logs\archive)

Renaming G:\Endeca\MyEndecaApp\config\script\..\..\.\logs\dgraphs\Dgraph2 to G:\Endeca\MyEndecaApp\config\script\..\..\.\logs\dgraphs\Dgraph2.2016_06_28.06_02_42
Unable to rename G:\Endeca\MyEndecaApp\config\script\..\..\.\logs\dgraphs\Dgraph2 to G:\Endeca\MyEndecaApp\config\script\..\..\.\logs\dgraphs\Dgraph2.2016_06_28.06_02_42: Permission denied

I tried running the baseline update again and again, sometimes Dgraph1 fails and sometime Dgraph2. After failure the dgraph also stopped.

Edit 1: I have observed that when I stops both the dgraphs from workbench and then run baseline update, it always ran successfully. I tried this 4-5 times. We know baseline_update stops dgraph before doing backup of log folder. So I am assuming dgraph is not stopped properly before baseline_update do backup of log folder and so it generates error.

Please help me in resolving the issue. I am novice in Endeca Administration

Thank you

Solution

  • Changing the Dgraph property 'numIdleSecondsAfterStop' to 90 seconds from IAP workbench solved the problem.

    It shows that the failure was due to the Dgraph was not properly stopped before rename and log folder was locked by Dgraph.

    Setting 'numIdleSecondsAfterStop' causes the baseline to wait for 90 seconds to process next steps after Dgraph stops.