Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

Raw Rows

The use of Raw Rows can dramatically decrease your memory usage (and increase your application's performance) at the cost of convenience. This uses less memory than normal EO's, allows better support for fetch limits, and you can still promote them to full-blown EO's when need be. Check out the Raw Rows topic for more information.

Qualify your Fetches

If you are not careful when you fetch, you may bring in huge datasets. When you use an EOFetchSpecification, be mindful to specify the most restrictive EOQualifier that still returns the data you need. Otherwise, you will be potentially faulting in much more data than your application will use, and you decrease your overall application performance because of the data that must be transferred from your database.

Relationships from Enumerated Types to other EO's

It is very common to have EO's that represent "Enumerated Types". For instance, you might have a bug tracking system that has a Severity entity (with individual instances of "Crasher", "Minor", etc) that are referenced by all bugs in the system. It is tempting to always create relationship back-references (i.e. Bug=>Severity, Severity=>>Bug). However, be aware that if you create the reference from Severity back to Bug, any time a Bug is added and you properly use the addObjectToBothSidesOfRelationshipWithKey method, EOF will fault in the ENTIRE "bugs" relationship on Severity prior to adding your new bug to the end of the list. The consequences of this can be crippling to your application. As the number of bugs increases, it will take longer and longer to add a new bug. The recommended best practice for this is to only create Bug=>Severity and NOT create the reverse relationship. In the event that you need to perform that fetch, you can manually construct the EOFetchSpecification rather than use the automatic approach.

Note that this is a filed bug against EOF. One approach that EOF could use is to only add the object to Severity's bugs relationship if the bugs relationship has already been faulted into memory. If the array has not yet been faulted, then causing the fault on insert is potentially very expensive. The reasoning for the current behavior is to maintain the ability to access the newly committed Bug via the Severity bugs relationship prior to committing your transaction (i.e. faulting the relationship from the database wouldn't yet return the newly inserted Bug). This is very important to maintaining object graph consistency pre-saveChanges, but does come at a high performance cost.

Dispose()ing your EOEditingContext

Calling dispose() on your EOEditingContext is not strictly required. When the EOEditingContext is no longer referenced, it will be automatically disposed. However, the question is how long it takes the GC to get to your object. If you call dispose() manually, you won't hurt anything, and you immediately decrement the snapshot reference counts. If your EC is falling out of scope or you set it to null, then it's probably a tossup as to who gets to it first - GC finalizing your EC or you manually calling dispose(). It is recommended that you dispose() your EOEditingContext when you are in a long running process in a single method where the EOEditingContext will not go out-of-scope until the end of the method.

Calling System.gc()

Calling System.gc() is usually a bad idea and can cause performance problems in your application. Additionally, System.gc() is not an order, it is a request, so if your application is relying on this command to execute, you should re-evaluate your application's design, because you may run into problems with this.

Anecdotes on editingContext.invalidateAllObjects()

It can be tempting to resolve memory problems by calling editingContext.invalidateObject(..) or editingContext.invalidateAllObjects(). Avoid this. You will only hurt yourself in the long run. Invalidating is very aggressive and WILL cause exceptions to be thrown if someone in another EOEditingContext is modifying an EO that you invalidate in your EOEditingContext (the shared snapshost will be destroyed, resulting in an exception on .saveChanges in the other context). It should only be used under very controlled situations where you understand the implications of the method.

Chuck Hill

As for invalidating objects, this is something to be avoided. I admit that I have been driven to desperation a couple of times and used this, but only with regrets and reservations.

Art Isbell

I just have a bias against using invalidateAllObjects() because, for me, it's caused more problems than it's solved.

Mike Schrag

The couple of times I resorted to this, I regretted it later, because it ended up screwing me. The biggest thing is you toss snapshots that people might be in the middle of editing, and that's going to cause really funky problems. It's a really heavy-handed way to go.

NSUndoManager

One of the most common memory problems people run into with WebObjects is the NSUndoManager. WebObjects has a particularly cool feature in that it supports an undo stack for EO transactions. Each time you saveChanges, you push an undoable set of changes onto the stack. The downside of this behavior is that you end up with strong references to old EO states, which can be deadly in a large EOF operation.

There are several fixes for this. One is to simply disable the NSUndoManager on your editing context by calling:

Code Block

editingContext.setUndoManager(null);

It has been referenced that setting the undo manager to null may cause issues with deletes under certain circumstances. In testing this with WebObjects 5.3, the situation could not be reproduced, but if you use WebObjects 5.2 or lower, you should be aware that you might run into problems.

...

  • VisualVM is a tool to monitor and troubleshoot Java applications. It runs on Oracle/Sun JDK 6, but is able to monitor applications running on JDK 1.4 and higher. It utilizes various available technologies like jvmstat, JMX, the Serviceability Agent (SA), and the Attach API to get the data and automatically uses the fastest and most lightweight technology to impose minimal overhead on monitored applications.

Raw Rows

The use of Raw Rows can dramatically decrease your memory usage (and increase your application's performance) at the cost of convenience. This uses less memory than normal EO's, allows better support for fetch limits, and you can still promote them to full-blown EO's when need be. Check out the Raw Rows topic for more information.

Qualify your Fetches

If you are not careful when you fetch, you may bring in huge datasets. When you use an EOFetchSpecification, be mindful to specify the most restrictive EOQualifier that still returns the data you need. Otherwise, you will be potentially faulting in much more data than your application will use, and you decrease your overall application performance because of the data that must be transferred from your database.

Relationships from Enumerated Types to other EO's

It is very common to have EO's that represent "Enumerated Types". For instance, you might have a bug tracking system that has a Severity entity (with individual instances of "Crasher", "Minor", etc) that are referenced by all bugs in the system. It is tempting to always create relationship back-references (i.e. Bug=>Severity, Severity=>>Bug). However, be aware that if you create the reference from Severity back to Bug, any time a Bug is added and you properly use the addObjectToBothSidesOfRelationshipWithKey method, EOF will fault in the ENTIRE "bugs" relationship on Severity prior to adding your new bug to the end of the list. The consequences of this can be crippling to your application. As the number of bugs increases, it will take longer and longer to add a new bug. The recommended best practice for this is to only create Bug=>Severity and NOT create the reverse relationship. In the event that you need to perform that fetch, you can manually construct the EOFetchSpecification rather than use the automatic approach.

Note that this is a filed bug against EOF. One approach that EOF could use is to only add the object to Severity's bugs relationship if the bugs relationship has already been faulted into memory. If the array has not yet been faulted, then causing the fault on insert is potentially very expensive. The reasoning for the current behavior is to maintain the ability to access the newly committed Bug via the Severity bugs relationship prior to committing your transaction (i.e. faulting the relationship from the database wouldn't yet return the newly inserted Bug). This is very important to maintaining object graph consistency pre-saveChanges, but does come at a high performance cost.

Dispose()ing your EOEditingContext

Calling dispose() on your EOEditingContext is not strictly required. When the EOEditingContext is no longer referenced, it will be automatically disposed. However, the question is how long it takes the GC to get to your object. If you call dispose() manually, you won't hurt anything, and you immediately decrement the snapshot reference counts. If your EC is falling out of scope or you set it to null, then it's probably a tossup as to who gets to it first - GC finalizing your EC or you manually calling dispose(). It is recommended that you dispose() your EOEditingContext when you are in a long running process in a single method where the EOEditingContext will not go out-of-scope until the end of the method.

Calling System.gc()

Calling System.gc() is usually a bad idea and can cause performance problems in your application. Additionally, System.gc() is not an order, it is a request, so if your application is relying on this command to execute, you should re-evaluate your application's design, because you may run into problems with this.

Anecdotes on editingContext.invalidateAllObjects()

It can be tempting to resolve memory problems by calling editingContext.invalidateObject(..) or editingContext.invalidateAllObjects(). Avoid this. You will only hurt yourself in the long run. Invalidating is very aggressive and WILL cause exceptions to be thrown if someone in another EOEditingContext is modifying an EO that you invalidate in your EOEditingContext (the shared snapshost will be destroyed, resulting in an exception on .saveChanges in the other context). It should only be used under very controlled situations where you understand the implications of the method.

Chuck Hill

As for invalidating objects, this is something to be avoided. I admit that I have been driven to desperation a couple of times and used this, but only with regrets and reservations.

Art Isbell

I just have a bias against using invalidateAllObjects() because, for me, it's caused more problems than it's solved.

Mike Schrag

The couple of times I resorted to this, I regretted it later, because it ended up screwing me. The biggest thing is you toss snapshots that people might be in the middle of editing, and that's going to cause really funky problems. It's a really heavy-handed way to go.

NSUndoManager

One of the most common memory problems people run into with WebObjects is the NSUndoManager. WebObjects has a particularly cool feature in that it supports an undo stack for EO transactions. Each time you saveChanges, you push an undoable set of changes onto the stack. The downside of this behavior is that you end up with strong references to old EO states, which can be deadly in a large EOF operation.

There are several fixes for this. One is to simply disable the NSUndoManager on your editing context by calling:

Code Block
editingContext.undoManagersetUndoManager(null).disableUndoRegistration();

Note It has been referenced that setting the undo levels to 0 using setLevelsOfUndo() does not work, as 0 means no limit!
Prior to 5.2, the NSUndoManager had effectively infinite undo levels by default. The javadoc of WO 5.3 still mentions the following:EOEditingContext's undo support is arbitrarily deep; you manager to null may cause issues with deletes under certain circumstances. In testing this with WebObjects 5.3, the situation could not be reproduced, but if you use WebObjects 5.2 or lower, you should be aware that you might run into problems.

The recommended alternative in this case is to disable the registration of undo events by calling:

Code Block

editingContext.undoManager().disableUndoRegistration();

Note that setting the undo levels to 0 using setLevelsOfUndo() does not work, as 0 means no limit!
Prior to 5.2, the NSUndoManager had effectively infinite undo levels by default. The javadoc of WO 5.3 still mentions the following:

EOEditingContext's undo support is arbitrarily deep; you can undo an object repeatedly until you restore it to the state it was in when it was first created or fetched into its editing context. Even after saving, you can undo a change. To support this feature, the NSUndoManager can keep a lot of data in memory.

However, as of 5.2, if you are using the session defaultEditingContext, WOSession sets the default undo stack size to 10 if there is no default size specified with the WODEFAULTUNDOSTACKLIMIT parameter.
Lastly you can manually clear your NSUndoManager when you are done with an operation by calling:

Code Block

editingContext.undoManager().removeAllActions();

If you use an NSUndoManager, it is recommended that you call this after performing very large saveChanges that involved large amounts of datacreated or fetched into its editing context. Even after saving, you can undo a change. To support this feature, the NSUndoManager can keep a lot of data in memory.

However, as of 5.2, if you are using the session defaultEditingContext, WOSession sets the default undo stack size to 10 if there is no default size specified with the WODEFAULTUNDOSTACKLIMIT parameter.
Lastly you can manually clear your NSUndoManager when you are done with an operation by calling:

Code Block

editingContext.undoManager().removeAllActions();

If you use an NSUndoManager, it is recommended that you call this after performing very large saveChanges that involved large amounts of data.

Aaron Rosenzweig

In our experience, the NSUndoManager should be considered an integral part of an EOEditingContext and neither be crippled nor removed. Calling

Code Block
removeAllActions()

is safe and can be done in your project's descendent of EOEditingContext after a successful call to

Code Block
super.saveChanges()

In WO 5.3 and above, setting the NSUndoManager to null will create the following error if you try to delete and save an EO that has a delete rule of "deny" on an existing relationship:

Code Block

java.lang.RuntimeException: java.lang.IllegalStateException:
Editing context needs an undo manager to recover from delete propagation problems.
Do not set the undo manager of this editing context to null.

Additionally, if you simply remove undo registration, you'll find that little undo / redo pieces are still allocated in memory... which defeats the purpose of removing registration. This can be verified with a tool such as Jprofiler, etc. Any tool which allows you to view the java heap and memory allocations.

If what you are really after is processing many EOs in a long running process you should consider making an array of primary keys and materializing them onesy-twosy in a temporary EOEditingContext. Before exiting the loop call

Code Block
ec.dispose()

then

Code Block
ec = null

Another way to say it, make an EOEditingContext for each pass through the loop then clear it out before the next pass.

WebObjects 5.2+

This is a description of the current behavior in 5.3 that one might be able to gather if one were to, say, decompile and review the entire process - not that i would ever do this or condone it, of course - but if you DID, you would probably gather exactly this info, which is pretty well documented (and seems to behave to-spec) in the 5.2 release notes:

...