When I tried to update
a topology running on the Heron Cluster
, failed message as following:
The output of update command using --verbose
as follows:
[2018-07-03 12:07:27 +0800] [FINE] com.twitter.heron.scheduler.RuntimeManagerMain: Exception when submitting topology
com.twitter.heron.spi.packing.PackingException: Could not initialize containers using existing packing plan
at com.twitter.heron.packing.builder.PackingPlanBuilder.initContainers(PackingPlanBuilder.java:259)
at com.twitter.heron.packing.builder.PackingPlanBuilder.addInstance(PackingPlanBuilder.java:153)
at com.twitter.heron.packing.builder.PackingPlanBuilder.addInstance(PackingPlanBuilder.java:141)
at com.twitter.heron.packing.binpacking.FirstFitDecreasingPacking.placeFFDInstance(FirstFitDecreasingPacking.java:312)
at com.twitter.heron.packing.binpacking.FirstFitDecreasingPacking.assignInstancesToContainers(FirstFitDecreasingPacking.java:265)
at com.twitter.heron.packing.binpacking.FirstFitDecreasingPacking.getFFDAllocation(FirstFitDecreasingPacking.java:246)
at com.twitter.heron.packing.binpacking.FirstFitDecreasingPacking.repack(FirstFitDecreasingPacking.java:180)
at com.twitter.heron.scheduler.RuntimeManagerRunner.buildNewPackingPlan(RuntimeManagerRunner.java:304)
at com.twitter.heron.scheduler.RuntimeManagerRunner.updateTopologyHandler(RuntimeManagerRunner.java:183)
at com.twitter.heron.scheduler.RuntimeManagerRunner.call(RuntimeManagerRunner.java:81)
at com.twitter.heron.scheduler.RuntimeManagerMain.callRuntimeManagerRunner(RuntimeManagerMain.java:448)
at com.twitter.heron.scheduler.RuntimeManagerMain.manageTopology(RuntimeManagerMain.java:396)
at com.twitter.heron.scheduler.RuntimeManagerMain.main(RuntimeManagerMain.java:317)
Caused by: com.twitter.heron.packing.ResourceExceededException: Insufficient container resources to add instancePlan {component-name: split, task-id: 1, component-index: 0, instance-resource: {cpu: 1.000000, ram: ByteAmount{1 GB (536870912 bytes)}, disk: ByteAmount{1 GB (1073741824 bytes)}}} to container {containerId=1, instances=[{component-name: spout, task-id: 3, component-index: 0, instance-resource: {cpu: 1.000000, ram: ByteAmount{1 GB (536870912 bytes)}, disk: ByteAmount{1 GB (1073741824 bytes)}}}, {component-name: count, task-id: 5, component-index: 1, instance-resource: {cpu: 1.000000, ram: ByteAmount{1 GB (536870912 bytes)}, disk: ByteAmount{1 GB (1073741824 bytes)}}}], capacity={cpu: 2.000000, ram: ByteAmount{4 GB (3758096384 bytes)}, disk: ByteAmount{3 GB (3221225472 bytes)}}, paddingPercentage=10}
at com.twitter.heron.packing.builder.PackingPlanBuilder.getContainers(PackingPlanBuilder.java:392)
at com.twitter.heron.packing.builder.PackingPlanBuilder.initContainers(PackingPlanBuilder.java:256)
... 12 more
Caused by: com.twitter.heron.packing.ResourceExceededException: Adding 1.0 cores to existing 2.0 cores with 10 percent padding would exceed capacity 2.0
at com.twitter.heron.packing.builder.Container.assertHasSpace(Container.java:165)
at com.twitter.heron.packing.builder.Container.add(Container.java:77)
at com.twitter.heron.packing.builder.PackingPlanBuilder.addToContainer(PackingPlanBuilder.java:417)
at com.twitter.heron.packing.builder.PackingPlanBuilder.getContainers(PackingPlanBuilder.java:390)
... 13 more
[2018-07-03 12:07:28 +0000] [ERROR]: Could not initialize containers using existing packing plan
This topology is running normally, I don't know what reasons causes this problem.
The error implies that your config doesn't set the repacking class correctly: Creating Packing Class in Heron
To solve this problem, you will need to add the corresponding config into your packing.yaml
config file. Here is an Example