Wiki source code of EOF-Using EOF-Bulk Operations
Last modified by Ray Kiddy on 2007/09/03 13:56
Hide last authors
| author | version | line-number | content |
|---|---|---|---|
 |
9.1 | 1 | == Overview == |
| |
3.1 | 2 | |
| |
10.1 | 3 | The standard techniques for using EOF can potentially break down when you are dealing with extremely large datasets. The techniques described here are useful for taking advantage of EOF's benefits while not at the same time running out of memory. |
| |
3.1 | 4 | |
| |
12.1 | 5 | There is quite a bit of overlap between this topic and the [[Memory Management>>doc:documentation.Home.Development Architecture.EOF Architecture.EOF.EOF-Using EOF-Memory Management.WebHome]] section. It is recommended that you review the general memory management methodologies prior to reading this. |
| |
3.1 | 6 | |
| |
9.1 | 7 | == Moving Large Amounts of Data Between Databases == |
| |
3.1 | 8 | |
| |
9.1 | 9 | === Chuck Hill === |
| |
3.1 | 10 | |
| 11 | Call unlock() then dispose() on the ec, then set the reference to null, then force garbage collection with: | ||
| |
10.1 | 12 | [[http:~~/~~/java.sun.com/docs/books/tutorial/essential/system/garbage.html>>url:http://java.sun.com/docs/books/tutorial/essential/system/garbage.html||shape="rect"]] |
| |
3.1 | 13 | |
| |
10.1 | 14 | You will need to do this often with a large database, maybe more than once per entity. I don't know if that would work while maintaining the object graph. |
| |
3.1 | 15 | |
| |
9.1 | 16 | === Pierce T Wetter III === |
| |
3.1 | 17 | |
| 18 | There used to be an EOUtil that could do something like this. Does that still exists? | ||
| 19 | |||
| 20 | Otherwise, it depends on how the ER relationship chart works. In our case, we have a set of objects that would more or less "own" all the other objects in the ER tree. So the code would look something like this: | ||
| 21 | |||
| |
9.1 | 22 | {{noformat}} |
| |
3.1 | 23 | |
| |
9.1 | 24 | get master editing context |
| |
3.1 | 25 | |
| |
9.1 | 26 | fetch copies of all the master objects into the master ec (not that many, so ok) |
| |
3.1 | 27 | |
| |
9.1 | 28 | for each master object: |
| |
3.1 | 29 | |
| |
9.1 | 30 | make a local ec |
| |
3.1 | 31 | |
| |
9.1 | 32 | copy the master object into the local ec (localInstanceOfObject()) |
| 33 | do stuff with master object (i.e. copy them into the sink model) | ||
| 34 | toss local ec | ||
| |
3.1 | 35 | |
| |
9.1 | 36 | {{/noformat}} |
| |
3.1 | 37 | |
| 38 | Using the local EC is the key. | ||
| 39 | |||
| 40 | If you have a large number of master objects enough that you can't load all of them into memory at once, then it gets trickier. One way is to just fetch a list of primary keys using raw rows, then use objectWithPrimaryKeyValue() to build the objects in the local editing context. | ||
| 41 | |||
| 42 | If your ER tree is more complex (there is no "owner" object, then YMMV). | ||
| 43 | |||
| |
9.1 | 44 | === EOImportExportApplication from Cassini === |
| |
3.1 | 45 | |
| |
10.1 | 46 | Cassini is available at [[Sourceforge>>url:http://sourceforge.net/projects/oce||shape="rect"]]. |
| |
3.1 | 47 | |
| |
9.1 | 48 | === Anjo Krank === |
| |
3.1 | 49 | |
| 50 | There is some stuff in Wonder's ERXJDBCUtilities that should be able to copy from one DB to another. I never tried it, but maybe it gives you some hints. There is no real need to use EOs anyway as you just want to copy the data. | ||
| 51 | |||
| |
9.1 | 52 | === David Teran === |
| |
3.1 | 53 | |
| |
10.1 | 54 | Hmpf! 'that should be able to copy from one DB to another.' NoNo, Anjo, that is not fair |
| 55 | In fact it works perfectly (% style="text-decoration: line-through;" %)if(%%) the JDBC driver is not buggy. Here is a quick documentation: | ||
| 56 | OK, you need to use ERExtensions and ERJars. Not a big deal, drop me a line and i'll send a demo app to you. | ||
| 57 | Then you need this code here, the example takes data from frontbase and saves it to a postgres database: | ||
| |
3.1 | 58 | |
| |
9.1 | 59 | {{code}} |
| |
3.1 | 60 | |
| |
9.1 | 61 | NSMutableDictionary sourceDict = new NSMutableDictionary(); |
| 62 | sourceDict.setObjectForKey("", "password"); | ||
| 63 | sourceDict.setObjectForKey("_SYSTEM", "username"); | ||
| 64 | sourceDict.setObjectForKey("jdbc:FrontBase://195.135.143.205/merces/user=_system", "url"); | ||
| 65 | sourceDict.setObjectForKey("com.frontbase.jdbc.FBJDriver", "driver"); | ||
| 66 | sourceDict.setObjectForKey(Boolean.FALSE, "autoCommit"); | ||
| 67 | sourceDict.setObjectForKey(Boolean.TRUE, "readOnly"); | ||
| |
3.1 | 68 | |
| |
9.1 | 69 | NSMutableDictionary destDict = sourceDict.mutableClone(); |
| 70 | destDict.setObjectForKey("jdbc:postgresql://195.135.143.205/merces", "url"); | ||
| 71 | destDict.setObjectForKey("dev", "password"); | ||
| 72 | destDict.setObjectForKey("dev", "username"); | ||
| 73 | destDict.setObjectForKey("org.postgresql.Driver", "driver"); | ||
| 74 | destDict.setObjectForKey(Boolean.FALSE, "autoCommit"); | ||
| 75 | destDict.setObjectForKey(Boolean.FALSE, "readOnly"); | ||
| |
3.1 | 76 | |
| |
9.1 | 77 | EOModel yourModel = EOModelGroup.defaultGroup().modelNamed("YourEOModelName"); |
| 78 | ERXJDBCUtilities.copyDatabaseDefinedByEOModelAndConnectionDictionaryToDatabaseWithConnectionDictionary(yourModel, sourceDict, destDict); | ||
| |
7.1 | 79 | |
| |
9.1 | 80 | {{/code}} |
| |
7.1 | 81 | |
| |
10.1 | 82 | Thas_all! |
| |
7.1 | 83 | |
| |
9.1 | 84 | === BulkMover from 360works === |
| |
3.1 | 85 | |
| |
10.1 | 86 | [[BulkMover>>url:http://omnivore.360works.com/360Site-3.0/dev_tools/BulkMover.tgz||shape="rect"]] provides these capabilities, though it is a ProjectBuilder project. |