I'm using Cloudify 2.7 with OpenStack Icehouse.
I have developed a service recipe for Glassfish, that registers the instance onto an Apache Load Balancer.
The start event executes a glassfish_start.groovy file that runs the following command:
su - glassfish -c "asadmin --passwordfile /home/glassfish/passwd.gf --user $userAdmin start-domain $domain"
su - glassfish -c "asadmin --passwordfile /home/glassfish/passwd.gf --user $userAdmin deploy --contextroot \"/\" $warName"
After the start event ends, I can see:
2015-07-14 09:59:15,678 glassfish.glassfish [1] INFO ....
[org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy (startDetection): arePortsFree http=8080
2015-07-14 09:59:15,680 glassfish.glassfish [1] INFO [org.cloudifysource.usm.UniversalServiceManagerBean] - Process liveness test passed
2015-07-14 09:59:15,680 glassfish.glassfish [1] INFO [org.cloudifysource.usm.UniversalServiceManagerBean] - Executing Process Locators!
2015-07-14 09:59:15,684 glassfish.glassfish [1] INFO [org.cloudifysource.usm.locator.DefaultProcessLocator] - Looking for actual process ID in process tree
2015-07-14 09:59:15,692 glassfish.glassfish [1] INFO [org.cloudifysource.usm.locator.DefaultProcessLocator] - Located process (6446): /usr/lib/jvm/java-8-oracle/jre/bin/java [java, -jar, /opt/glassfish4/bin/../glassfish/lib/client/appserver-cli.jar, --passwordfile, /home/glassfish/passwd.gf, --user, <user>, deploy, --contextroot, /, hello.war]
2015-07-14 09:59:15,693 glassfish.glassfish [1] INFO [org.cloudifysource.usm.UniversalServiceManagerBean] - Monitored processes: [6446]
2015-07-14 09:59:15,700 glassfish.glassfish [1] INFO [org.cloudifysource.usm.USMEventLogger.glassfish.glassfish] - glassfish.glassfish-1 POST_START invoked
2015-07-14 09:59:15,707 glassfish.glassfish [1] INFO [org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy: glassfish Post-start - useLoadBalancer = true - Adding Glassfish instance onto ApacheLB...
2015-07-14 09:59:15,721 glassfish.glassfish [1] WARNING [org.cloudifysource.utilitydomain.admin.TimedAdmin] - Admin object might expire prematurely! The specified timeout for waiting for PU was set to 180 SECONDS while the admin timeout is 120 seconds
2015-07-14 09:59:15,740 glassfish.glassfish [1] INFO [org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy: add-node onto ApacheLB ...
2015-07-14 09:59:15,742 glassfish.glassfish [1] INFO [org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy: ipAddress is <IP> ...
2015-07-14 09:59:15,745 glassfish.glassfish [1] INFO [org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy: Sto per aggiungere l'URL http://<IP>:8080 ad ApacheLB ...
2015-07-14 09:59:21,686 glassfish.glassfish [1] INFO [glassfish-stdout] - [exec] Application deployed with name hello.
2015-07-14 09:59:21,687 glassfish.glassfish [1] INFO [glassfish-stdout] - [exec] Command deploy executed successfully.
2015-07-14 09:59:21,687 glassfish.glassfish [1] INFO [glassfish-stdout] - [exec] startOnUbuntu.sh: End
2015-07-14 09:59:21,687 glassfish.glassfish [1] INFO [glassfish-stdout] - [exec] + echo 'startOnUbuntu.sh: End'
2015-07-14 09:59:23,007 glassfish.glassfish [1] INFO [org.cloudifysource.dsl.internal.BaseDslScript] - glassfish-service.groovy: glassfish Post-start ended
2015-07-14 09:59:23,008 glassfish.glassfish [1] INFO [org.cloudifysource.usm.USMEventLogger.glassfish.glassfish] - glassfish.glassfish-1 POST_START completed, duration: 7.3 seconds
2015-07-14 09:59:23,008 glassfish.glassfish [1] INFO [org.cloudifysource.usm.UniversalServiceManagerBean] - Starting async tasks
2015-07-14 09:59:25,011 glassfish.glassfish [1] INFO [org.cloudifysource.usm.USMLifecycleBean] - Stop detection - service has stopped!
2015-07-14 09:59:25,011 glassfish.glassfish [1] INFO [org.cloudifysource.usm.UniversalServiceManagerBean] - Detected death of underlying process
2015-07-14 09:59:25,011 glassfish.glassfish [1] INFO [org.cloudifysource.usm.USMEventLogger.glassfish.glassfish] - glassfish.glassfish-1 POST_STOP invoked
Why has it invoked the stop detection?
Moreover, the Cloudify docs state this:
Known Limitation: In case your start script returns, Cloudify will treat your service instance as having stopped and will therefore attempt to provision it again. To avoid this behavior, make sure your start script sleeps forever.
What should I do?
By default, Cloudify monitors the process launched by the start lifecycle event. If that process dies, Cloudify will consider this an failure of the service instance and will try to restart it.
If the process launched by the start event is not the one you should monitor, (perhaps because you ran the process as a service, or with nohup) you can add the locator section to the recipe, which tells Cloudify where to look for the monitored process. You can also configure the locator to not monitor processes at all, as follows:
locator {
NO_PROCESS_LOCATORS
}
There is a similar question here: cloudify 2.7 locator NO_PROCESS_LOCATORS