Compatibility :	3.5.0
Editions :
Platforms :

This wrapper.check.deadlock property, in combination with the other properties,

• wrapper.check.deadlock.interval,
• wrapper.check.deadlock.action, and
• wrapper.check.deadlock.output

are used to configure how the Wrapper monitors a JVM for deadlocked threads. This can be very useful to detect and then work around potentially fatal problems which would otherwise be difficult if not impossible to work around.

Thread deadlock checks require that at least Java version 1.5 is used. Only JVMs will ignore the checks.

Checking for Deadlocks is fairly quick, but it does involve briefly locking and taking a snapshot of all threads so this property is FALSE by default.

wrapper.check.deadlock=FALSE

Simple Example:

Please read over the property details below, but the following simple example will configure the Wrapper to log the location of a deadlock and then immediately restart the JVM.

wrapper.check.deadlock=TRUE
wrapper.check.deadlock.interval=60
wrapper.check.deadlock.action=RESTART
wrapper.check.deadlock.output=FULL

Deadlock?:

Deadlocks can occur when two or more threads are locking resources in an order which results in all threads waiting indefinitely.

The simplest example is where Thread A locks Object A and then attempts to lock Object B, while another Thread B has Object B locked and is waiting to lock Object A. In this case, Thread A will never release Object A because it is waiting for Object B. This will never happen because Thread B will keep Object B locked indefinitely as it waits for Object A to become available.

Compatibility :	3.5.0
Editions :
Platforms :

The wrapper.check.deadlock.interval property makes it possible to control the interval at which the Wrapper looks for deadlocks in an application. It can be set to any interval starting with once per second, but it defaults to "60" seconds (once per minute). In general, for applications which are known to be stable, it may be fine to greatly decrease the frequency of these deadlock checks.

wrapper.check.deadlock.interval=60

Compatibility :	3.5.0
Editions :
Platforms :

The wrapper.check.deadlock.action property makes it possible to control what the Wrapper does when a deadlock is detected. The default action is to RESTART.

wrapper.check.deadlock.action=RESTART

Possible actions are:

• DEBUG -

  will cause a debug message to be logged. This is only really useful in helping to understand when the action is fired.

• DUMP -

  will invoke a thread dump.

• GC (Since ver. 3.5.7) -

  will invoke a full garbage collection sweep in the JVM. Be aware that doing this frequently can affect performance of the JVM as a full sweep will often cause all threads to freeze for the duration of the GC.

• RESTART -

  will stop the current JVM and then restart a new invocation.

• SHUTDOWN -

  will stop the JVM as well as the Wrapper.

• USER_<n> (Professional Edition) -

  will cause a user defined event to be fired. This can be either the sending of an email, or the execution of an external system command. The command could be anything from performing clean up operations to raising an SNMP trap.

• PAUSE -

  will pause the Java application if pausing is enabled and the JVM is running. See the wrapper.pausable property for details.

• RESUME -

  will resume the Java application if it is in a paused state. This could be used if the JVM is not stopped when paused. See the wrapper.pausable property for details.

• SUCCESS (Since ver. 3.5.5) -

  will tell the Wrapper to reset its internal failed invocation count and count the current JVM invocation as successful. This is probably not useful in this context, but here for consistency with other properties.

• NONE -

  will tell the Wrapper to log the fact that a deadlock was detected, but it will not actually do anything.

Note that actions, like NONE, which do not restart the JVM will repeatedly be fired each time the deadlock check is made.

Chaining Multiple Actions:

It is also possible to list multiple actions so they each happen in the order specified.

wrapper.check.deadlock.action=DUMP,RESTART

Compatibility :	3.5.0
Editions :
Platforms :

The wrapper.check.deadlock.output property makes it possible to control what information the Wrapper causes to be logged when a deadlock is detected. The default output level is FULL.

wrapper.check.deadlock.output=FULL

Possible output levels are:

• FULL -

  will cause the WrapperManager class within the JVM to output a report which includes full stack traces for the threads involved in the deadlock.

  Output Example of "FULL":

  INFO | jvm 1 | WrapperManager Error: Found 2 deadlocked threads! INFO | jvm 1 | WrapperManager Error: ============================= INFO | jvm 1 | WrapperManager Error: "Locker-2" tid=18 INFO | jvm 1 | WrapperManager Error: java.lang.Thread.State: BLOCKED INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.lockSecond(DeadLock.java:64) INFO | jvm 1 | WrapperManager Error: - waiting on <0x000000002fcac6db> (a java.lang.Object) owned by "Locker-1" tid=17 INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.lockFirst(DeadLock.java:83) INFO | jvm 1 | WrapperManager Error: - locked <0x0000000029c56c60> (a java.lang.Object) INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.access$100(DeadLock.java:22) INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock$1.run(DeadLock.java:42) INFO | jvm 1 | WrapperManager Error: INFO | jvm 1 | WrapperManager Error: "Locker-1" tid=17 INFO | jvm 1 | WrapperManager Error: java.lang.Thread.State: BLOCKED INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.lockSecond(DeadLock.java:64) INFO | jvm 1 | WrapperManager Error: - waiting on <0x0000000029c56c60> (a java.lang.Object) owned by "Locker-2" tid=18 INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.lockFirst(DeadLock.java:83) INFO | jvm 1 | WrapperManager Error: - locked <0x000000002fcac6db> (a java.lang.Object) INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock.access$100(DeadLock.java:22) INFO | jvm 1 | WrapperManager Error: at org.tanukisoftware.wrapper.test.DeadLock$1.run(DeadLock.java:42) INFO | jvm 1 | WrapperManager Error: INFO | jvm 1 | WrapperManager Error: ============================= STATUS | wrapper | A Thread Deadlock was detected in the JVM. Restarting JVM.

• SIMPLE -

  will cause the WrapperManager class within the JVM to output a report which includes only a brief summary naming the threads and objects involved in the deadlock. In many cases, this is enough especially for known problems.

  Output Example of "SIMPLE":

  INFO | jvm 1 | WrapperManager Error: Found 2 deadlocked threads! INFO | jvm 1 | WrapperManager Error: ============================= INFO | jvm 1 | WrapperManager Error: "Locker-2" BLOCKED waiting on a java.lang.Object owned by "Locker-1" INFO | jvm 1 | WrapperManager Error: "Locker-1" BLOCKED waiting on a java.lang.Object owned by "Locker-2" INFO | jvm 1 | WrapperManager Error: ============================= STATUS | wrapper | A Thread Deadlock was detected in the JVM. Restarting JVM.

• NONE -

  will cause the WrapperManager class within the JVM to suppress all output. This may be desired for production systems when a problem is known but you don't want or need to maintain too much information in the logs. The Wrapper process, as in the other options, will always log a single entry to provide a reason why the triggered action takes place.

  Output Example of "NONE":

  STATUS | wrapper | A Thread Deadlock was detected in the JVM. Restarting JVM.