Optimistic and Pessimistic Locking Recommendations

This is one of the data gathering methodologies and recommendations for functional performance improvement. The full list is defined in the Performance Recommendations topic.

STEP uses locks to prevent data corruption due to concurrent updates by multiple processes. For example, consider a bank account where one person deposits $100 and another person concurrently withdraws $100. If the two (2) processes read the balance at the same time, then add or subtract $100 and update the balance, the balance is either increased or decreased by $100. But the correct result is for the balance to remain the same.

In STEP, when concurrent threads attempt to generate new node revisions, locks protect the node's revision history to avoid data corruption. Contention for row locks indicate a thread is locked (often by business logic) while waiting for another thread to commit data. To avoid row lock contention, business logic that runs updates (like bulk updates, approvals, workflow transitions, imports, etc.) should be fast to execute and limit the updates to a small number of objects. For example, updating a field on child products when approving a product may work reasonably well when updating a product with 10 children. However, it is not uncommon for some products to have thousands of children. With the 'update child field' business logic, the approval would likely run for a very long time, and as a result, the product being approved would be locked, blocking other attempts to update that product until the approval operation is complete.

The available locking options include:

  • Optimistic - This is the default option for STEP and involves no locking but increases the version number of an object in the database. When a process attempts to update a value on a node, it reads the version number, updates the value, and then re-reads the version number. If another process has updated the value in the meantime, then the first process retries up to 10 times (including 2000 ms waiting time) until successful.

    Optimistic locking exceptions indicate that additional attempts were required before the update was successful. The exceptions prevent two (2) identical updates from starting concurrently which would result in an infinite loop. However, even with a successful update, the update runs slowly.

  • Success / Failure: If the version number is the same as it was before the update, the version number is incremented. If, in the meantime, another process has updated the value, an optimistic locking exception is thrown and the update is not committed.

  • User Interface / Automatic Process: If an optimistic locking error occurs in the user interface, the user is immediately notified that another process has already updated the same data. If an optimistic locking error occurs in an automatic process, the lock is reported after the process retries 10 times. The result is that optimistic locking errors can slow down processes without reporting any errors. The longer a transaction takes, the higher the probability of optimistic locking issues when other processes run simultaneously.

  • Pessimistic - Locking occurs when two or more processes attempt to update a value in the database at exactly the same time. Other processes must wait for the lock to be released before proceeding which can degrade performance. STEP uses pessimistic locking when creating a new revision of a node and when creating a link to a node unless the target node type is set to a relaxed locking policy.

    The disadvantage to pessimistic locking is that until the lock is released, the locked data cannot be edited even if the user that originally locked the row has completed their task. There is also a higher risk for deadlocks.

Deadlocks

Database deadlocks occur when two or more processes have locked a resource, and each process requests a lock on a resource that another process has already locked. One or more transactions are waiting for one another to release a lock on their process.

For example, Process A has resource 1 locked and is requesting a lock on resource 2. Process B has resource 2 locked and is requesting a lock on resource 1. A deadlock occurs and is not resolved until one of the processes releases a lock. Although this example is simplistic, it shows how deadlocks work and can be resolved.

This topic includes the following sections:

  • 'Reference Target Lock Policy' Parameter

  • Optimistic Locking - Log Level Setting

  • Optimistic Locking - Analyze Failures

  • Optimistic and Pessimistic Locking Recommendations

'Reference Target Lock Policy' Parameter

Both optimistic and pessimistic locking are impacted by the 'Reference Target Lock Policy' parameter on the object types of the objects being handled. For automated processes that create references to target objects, the 'Relaxed' setting is beneficial on the Reference Target Lock Policy parameter of the target object types.

The 'Reference Target Lock Policy' parameter for an asset, entity, classification, and product object type manages how objects are locked while they are being referenced.

When long-running transactions cannot be simplified easily, consider setting the 'Reference Target Lock Policy' parameter to 'Relaxed' on the object types to which the long transaction applies.

Strict or Relaxed

The 'Strict' setting is default for the 'Reference Target Lock Policy' parameter. When a reference between two objects is being created, the target object is locked to ensure that it is not being deleted while the reference is being created. This means that only one process or user can edit it at a time. Often, deletion of the object is not a deletion from the database, but a deletion due to revision control when the insert of a new history entry occurs.

When using the Optimistic Locking option, a 'Strict' setting can result in an issue running parallel imports where the first import locks the object type being referenced, and the second import eventually stops running because it cannot access the locked object. Since STEP continues to retry the import, this can negatively impact inbound feeds.

The 'Relaxed' setting uses a less restrictive lock (a shared 'user lock') on the reference target objects being edited so that concurrent updates by multiple processes and/or users are allowed. A full lock is used only when a deletion is attempted. The Relaxed setting allows faster parallel creation of references to the same target object. This setting, when used with object types that are frequently referenced but rarely deleted, improves the performance and stability of parallel inbound imports, bulk updates, and users concurrently creating references to the same objects.

Important: Do not use the 'Relaxed' setting on an object type for objects that are often deleted as this can result in poor performance, and risks locks and deadlocks.

For example, the image below shows the 'Reference Target Lock Policy' is set to 'Relaxed' for object type 'Product Image (ProductImage)' since the many of the optimistic locking errors in the asset push apply to this object type.

Optimistic Locking - Log Level Setting

STEP puts a transaction on hold when optimistic locking occurs and after some time, tries to process the transaction again. This illustrates how optimistic locking errors can degrade the performance of imports, exports, asset push, business rules, etc. and can degrade the performance of the STEP system.

The message Optimistic lock exception occurred, retrying occurs for optimistic locking errors with fewer than 10 retries. When the Log.Level.com.stibo.core.domain.impl.ManagerImpl property is set to FINER or FINEST in the sharedconfig.properties file on the application server, these errors are written to the step.log files. For more information, refer to the Logs topic in the Administration Portal documentation here.

Optimistic Locking - Analyzing Failures

Use the STEP Admin Portal to analyze optimistic locking errors.

  1. From the Start Page, click the System Administration button and supply the login credentials.
  2. On the Logs tab, click the Fetch data button to load the data.
  3. Select the Main STEP Log File and click either the View link or the Download link to review the log.

  1. Search the log for the instance of 'optimistic locking' text.

  1. Determine the cause of the optimistic locking error as shown in the Examples section below.
  2. Follow the steps in the Optimistic Locking Recommendations section to reduce optimistic locking errors.

Examples

Optimistic locking errors are caused by long transactions and may be an import, export, business rule, asset push, etc. The following examples illustrate how to identify what caused the optimistic locking errors.

  • Warning for optimistic locking caused by Asset Push

    2017/10/25-16:34:07 9e8e com.stibo.services.assetpush.beans.AbstractServiceBean wrapUnexpectedException 
WARNING Caught unexpected: kodo.util.OptimisticVerificationException: Optimistic locking errors were 
detected when flushing to the data store. This indicates that some objects were concurrently modified 
in another transaction. Failed objects: [AssetPO@7caa505: 3615004291612_1, AssetPO@526cb477: 
3615004302998_3, AssetPO@3c0fccc3: 3615004302998_pantone, AssetPO@3bd3211d: 3615004291612_3, 
AssetPO@696359ef: 3615004302998_1, AssetPO@5e9b87b8: 3615004291612_pantone, AssetPO@1cb0217c: 
3615004302998_2, AssetPO@3e49203e: 3615004291612_2] [java.util.ArrayList]

  • Severe optimistic locking error caused by a Web UI transaction in Web UI 'C4userportal'

    2017/10/17-12:34:59 795|L3022356|PRT|C4userportal com.stibo.portal.engine.server.util.ExceptionConverter 
convertExceptionSerializeAndLocalize SEVERE Type: Error no esperado, Message: Optimistic locking errors 
were detected when flushing to the data store. This indicates that some objects were concurrently 
modified in another transaction. Failed objects: [ProductPO@21d744cc: C4A-305195, Value7PO@5b8b4f21, 
valno: 231765818, qualifier: -5, rev: 0, Value7PO@344dc391, valno: 231765817, qualifier: -5, rev: 0, 
Value7PO@3f2b8947, valno: 231765815, qualifier: -5, rev: 0, Value7PO@5478ad23, valno: 231765816, 
qualifier: -5, rev: 0, com.stibo.core.persistence.NodeStatePO@36bc02a7] [java.util.ArrayList]

  • Warning for optimistic locking caused by business condition transaction in state 'ModeCommerce' in Web UI 'C4userprotal'

    2017/10/23-09:52:10 6d7b|R3404828|PRT|c4userportal com.stibo.core.domain.impl.state.scxmlimpl.StateFlowImpl 
evaluateConditionNoThrow WARNING Business Condition failed for item "C4A-1957171" in state 
"ModeCommerce" : Wrapped kodo.util.OptimisticVerificationException: Optimistic locking errors were 
detected when flushing to the data store. This indicates that some objects were concurrently modified 
in another transaction. Failed objects: [ProductPO@3c0b71a7: C4V-1957172, ProductPO@2f9030b2: 
C4V-1038657, ProductProductReferencePO@284a0999, edgeid: 23130110, rev: 0] [java.util.ArrayList] 
(Script#305) in Script at line number 25 at column number 0

Optimistic and Pessimistic Locking Recommendations

For both locking options:

  • Modify the reference target lock policy - set the parameter to 'Relaxed' when all of these points are true:

  • optimistic locking failures occur frequently in the log file

  • transactions cannot be optimized

  • the object types that are frequently referenced are rarely deleted

For more information, refer to the 'Reference Target Lock Policy' Parameter section.

For the optimistic locking option:

  • Optimize large transactions - keep transactions small to limit the probability of introducing an optimistic locking failure by optimizing the business rules and functions used at e.g., imports, exports, bulk updates.

  • Temporarily set Log.Level.com.stibo.core.domain.impl.ManagerImpl = FINER or FINEST to analyze all optimistic locking issues as defined in the Optimistic Locking - Log Level Setting section.

  • Analyze optimistic locking problems - defined in the Optimistic Locking - Analyzing Failures section.