[SOLVED] How can I define a JSR 352 chunk step to checkpoint every 10 seconds (and use built-in checkpointing in general)?

How can I define a JSR 352 chunk step to checkpoint every 10 seconds (and use built-in checkpointing in general)?

Is there a simple way using <chunk time-limit="10"...>?

Can I combine a time-limit with my custom CheckpointAlgorithm?

Solution

Short Answer:

Probably the simplest way is something like

<chunk time-limit="10" item-count="999999">

where 999,999 is just some big number that will never be hit.

Background

To explain why, here's a more general answer to "how can I use the JSR 352 built-in checkpoints?"

There are two ways to configure checkpoints in JSR 352, the two checkpoint policies.

In JSL (XML)

This is the default, "built-in" behavior.

In Java

Controlled by application's CheckpointAlgorithm impl, and enabled via checkpoint-policy="custom" in JSL.

Built-in policy is based on item OR time, whichever "hits" first

The fundamental point to understand about the built-in checkpoint policy is that it is based on item count OR time limit, whichever comes first.

Examples

<chunk>

After 10 items (default)
<chunk item-count="25">

After 25 items
<chunk time-limit="10">

After 10 seconds or 10 items (again, the item-count default), whichever comes first.
<chunk time-limit="10" item-count="25">

After 25 items, or 10 seconds, whichever comes first.
<chunk time-limit="10" item-count="999999">

After 999,999 items, or 10 seconds, whichever comes first (So in all but the simplest processing, this effectively means after 10 seconds, and you could make this number even bigger if necessary).
<chunk checkpoint-policy="custom">

Implemented in your own application code via CheckpointAlgorithm, using method isReadyToCheckpoint(), and optionally the timeouts as well), and referenced like:
```
 <chunk checkpoint-policy="custom">
     <checkpoint-algorithm ref="myCustomCheckpointAlgorithm">
```

Discussion

So the time-limit defaults to '0' which is defined to mean "unlimited time" or "don't checkpoint on time". On the other hand, the item-count defaults to '10', and an analogous behavior for item-count of '0' is undefined.

So the best way to checkpoint based on, some number of seconds, is just to set the item-count high enough that it doesn't matter, which would typically not be hard in real-world applications.

This is example #5 above.

You can not combine built-in controls with your "custom" algorithm !

<chunk checkpoint-policy="custom" time-limit="5"> The time-limit is IGNORED !

You can't combine a custom algorithm with any of the "item" checkpoint policy attributes (these attributes are just ignored). You can't say checkpoint based on my custom algorithm OR at 5 seconds, whichever comes first.

<chunk checkpoint-policy="custom" item-count="500"> The item-count is IGNORED !

Same as previous example.

Caveat

There are some older, obsolete examples floating around, e.g. this article that also incorporate ""commit-interval" and a "time" checkpoint policy. These were NOT incorporated into the final 1.0 specification.

Closing Thoughts

The above solution has a less-elegant, "hack" quality to it.

Perhaps the specification should define a behavior where item-count="0" means, "never checkpoint based on item count", which would allow a simpler solution. Perhaps this should be considered for a possible 1.1 update.