eLifeMagz.Com » Berkeley DB Java Edition 4.1 Improvements

Home > Business > Berkeley DB Java Edition 4.1 Improvements

Berkeley DB Java Edition 4.1 Improvements

October 30th, 2010 elifemagz Leave a comment Go to comments

The new let go of Berkeley DB Java Edition (JE), Release 4.1, includes a few new features that drastically upgrade out-of-cache performance. When a JE focus has a information set that does not fit wholly in cache, and there is no specific working set of information that fits in cache, the focus will see the most appropriate opening when as ample of the inner btree nodes (the index) are kept in cache. 4.1 includes improvements to make JE’s cache administration more efficient, and provides census data to help establish cache efficiency.

The initial critical new underline is Concurrent Eviction. In past releases, cache eviction has been carried out by JE daemon threads, focus threads that call JE operations, and an elective singular evictor thread. The real eviction operation was serialized, and could emanate a bottleneck where many threads could be seen watchful on the way com.sleepycat.je.evictor.Evictor.doEvict().

In 4.1.6, cache eviction is no longer serialized and may be executed concurrently. In addition, JE right away has a dedicated configurable thread pool that will do cache eviction when mental recall boundary are reached. Eviction is completed by this dedicated pool, by other JE daemon threads, and by focus threads. JE attempts to askance the eviction effort toward the pool and daemon threads, to be able to unload focus threads.

The second critical underline is in-memory btree Internal Node (IN) compression that is targeted at shortening the mental recall indispensable to grip these nodes in cache. One of the optimizations reduces the in-memory footprint of an IN when usually a tiny part of it has been referenced, as would be the box when information archives are accessed in a pointless order, or when a subset of information is accessed. It does not help if the focus is carrying out (e.g.) a database-wide cursor traversal. A second optimization in this area occurs when a key’s length is reduction than or next to to 16 bytes, that may be loyal possibly when the key is of course small, or when key prefixing is enabled by Database.setKeyPrefix().

A user ran tests comparing JE 4.0.103 and JE 4.1 in a read-only effort and common the results with us. When the database fits completely in the cache (4 GB of memory), opening is about the same. Dropping the cache to 2 GB (all INs still fit in to memory), opening (throughput and latency) improves 5%. When the cache is serve marked down to assorted values between 1 GB and 512 MB (only some of the INs fit in memory), opening improves more than 3x.

One other engaging note about these tests is that the assessment setup has sufficient mental recall to grip the database in the record network cache (even even though they did not set aside sufficient mental recall to the JE cache to grip all of the database). The net of this is that there is no “true” IO occurring, but rsther than all IO is usually to the record network cache. By putting the information in the record network cache rsther than than on the Java pile (and thus the JE cache), GC up above is marked down whilst still progressing “in-memory” opening (since the information was in the record network cache).

What is moreover engaging about this is that the existing JE-tuning saying that out-of-cache scenarios should arrange the je.evictor.lruOnly and je.evictor.nodesPerScan parameters is changing. By varying these values in 4.1 from the endorsed norms (false and 100, respectively), the user is able to accomplish even improved performance. We will of course be updating the FAQ entries to state the new endorsed values.

Naturally we’re really vehement about these results and wish to share them with you. Stay tuned for more headlines when you have the results of read/write workloads.