1 Release 1.6.7 on 02/07/2014
3 * Updating the ivy file to refer to tusk-0.0.2 (build fix for nuage)
5 Release 1.6.6 on 02/07/2014
7 * Quota management in Voldemort
8 * Creation of read only stores in RO2 format
9 * Updation of query key feature in admin tool
11 Release 1.6.5 on 02/04/2014
13 * Adding AdminClient functionality to update store definitions
15 Release 1.6.4 on 02/03/2014
17 * Fix bug when RO server NPE when rolled back
18 * RO bug fix caused by RW optimization code
20 Release 1.6.3 on 01/24/2014
22 * Fix KeyVersionFetcher to not depend on nodeid 0
24 Release 1.6.2 on 01/10/2014
26 * Slops stored for nodes no longer in cluster, will die automatically
27 * Admin command to selectively purge slops destined to nodes/zones/stores
28 * Admin command to atomically update cluster.xml, stores.xml in one go
29 * Added admin command show-routing-plan to trace routing table for a key
30 * ZoneClipperCLI also adjusts store definitions based on dropped zone
31 * Numerous tests around non contiguous node ids/zone ids
33 Release 1.6.1 on 12/13/2013
35 * Avro support for fetch keys/entries admin command
36 * Store Clients are now cached by default, within the same factory
37 * Rigoruous tests around non contiguous node ids/zone ids
38 * ZenStoreClient efficient resource managerment
41 Release 1.6.0 on 11/26/2013
43 NOTE: This is an open source release! This release can be downloaded here:
44 https://github.com/voldemort/voldemort/releases
46 Changes made since 1.5.9:
48 * Fixes to remove dependency on non contiguous node ids
49 * Stats for store client
52 Release 1.5.9 on 11/05/2013
54 * Bug fix in AsyncMetadataVersionManager (related to store version tracking)
56 Release 1.5.8 on 10/29/2013
58 * Replace resthdfs jar with tusk.jar
59 * Incorporate retries into rest hdfs fetcher
61 Release 1.5.7 on 10/28/2013
63 * Voldemort now uses BDB 5.0.88
64 * Support for RO fetches via RestHdfs
65 * Fix NPE in Streaming Client
66 * Retrying fetches on all exceptions w/ delay
68 Release 1.5.6 on 10/18/2013
70 * Removed getStoreClientFactoryStats from StoreClientFactory interface (fix)
72 Release 1.5.5 on 10/18/2013
74 * Client shell with avro support
76 * Fix ObsoleteVersion Exceptions
77 * Build once and push to multiple clusters
78 * Improvements to failure detector
80 * Prevent NPE in worker thread when shutdown
81 * Bug fixes in repair and prune job
83 Release 1.5.3 on 09/19/2013
85 * Bug fixes (NPE during shutdown
86 * Client side visibility changes (Histogram, Connect exception)
87 * Adding PruneJob and timebased resolving mechanism to streaming client
88 * R2Store cleanup required for the rest client
89 * Monitoring improvements (aggregated BDB stats, streaming stats)
90 * Monitoring and Config improvements for REST service
91 * Removed donor based rebalancing code
92 * Removed replica type from the code base
93 * Improvements to Rebalance controller
95 Release 1.4.7 on 09/04/13
98 * Added the Zone affinity feature on the client side
99 * Refactoring the rest package (Rest server and Coordinator)
100 * Using standard Netty pipeline in the Coordinator
101 * Added monitoring hooks for the Rest Service
103 Release 1.4.6 on 07/24/2013
106 * Cleanup of R2Store to work with the REST server
107 * Modified ZenStoreClient to avoid creating SystemStores during each
109 * Added slop streaming functionality
111 Release 1.4.4 on 07/08/2013
113 Changes made since 1.4.3
114 * Change Repartioner to swap among nodes from all specified zones
115 * Added rebalance shuffle, cluster expansion scripts
117 Release 1.4.3 on 07/03/2013
119 Note: Server changes are not backwards compatible. To use new
120 rebalancing tooling, servers must be upgraded before hand.
123 - Proxying / abortable rebalance
124 - added proxy puts which rebalance safely abortable
125 - improved proxy gets to check locally first and only fetch remote
127 - Repartitioning: improved algorithms to optimize partition layout
128 - Plan: avoids cross-zone data movement in most rebalance cases. Only
129 expanding into a new zone requires data movement across zones.
131 - better / more accurate progress monitoring.
132 - 'proxy pause' before rebalancing. Ensures clients pick up new
133 metadata and allows performance with proxy'ing to be observed
134 before servers start rebalancing.
135 - NOTE: Donor-based rebalance is currently broken by these changes.
137 - PartitionBalanceCLI: more nuanced and verbose analysis of cluster
139 - RepartitionerCLI: Stand alone tool to determine repartitioning
140 - Stand alone 'repartitioning' scripts for key use cases: new cluster,
141 shuffle (existing cluster), cluster expansion, and zone expansion.
142 - RebalancePlanCLI: Stand alone plan tool to determine cost of a
144 - RebalanceControllerCLI: Stand-alone tool for executing a specific
148 - moved to Guava from Google Collections
149 - better unit test coverage of zoned clusters
150 - added example prod configuration
151 - added tools directory
152 - .gitignore improvements
154 - fixed concurrency bugs in metadata store
155 - fixed vector clock comparisons
157 - write slops in more cases when they are needed
158 - KeySampler/Fetcher handle larger working sets with OOME
160 Changes made since 1.4.2
162 * Faster creation of BaseStoreRoutingPlan objects in the fast rebalancing path
164 Release 1.4.2 on 06/27/2013
165 * Fixing costly StoreRoutingPlan object construction
167 Release 1.4.0 on 06/21/2013
168 * Zone expansion release
170 Release 1.3.4 on 06/19/2013
171 * Read Write store bug fixes
172 - Rewrite of InMemoryStorageEngine + config to control multiVersionPuts
173 - Fixed a bug in the read-repair logic which was causing unnecessary puts.
174 - Fixed put operation in PipelineRoutedStore to ensure slop is submitted
175 - Fixed VectorClock bug triggered during versioned puts
176 * Rebalance improvement
177 - Made it safe to abort rebalance
178 - Proxy puts are established so that 'old' partition is updated and so an aborted rebalance can "roll back" without any data loss
179 - Proxy gets & puts are established within the zone making improving performance during rebalance
181 - Added monitoring capability in the Coordinator
183 Release 1.3.3 on 04/24/2013
184 * VoldemortBuildandPush
185 - Fixed bug with schema check
187 - Fixed issue with redundant callback invocation
188 Release 1.3.1 on 03/25/2013
190 - Fixed the bug in calculating checksums when we entere a retry loop
191 - refactored per file checksums
192 - added junit test case to simulate intermittent IO exceptions
193 * voldemort.client.protocol.admin.AdminClient
194 - Added AdminStoreClient so that AdminClient can do store operations
195 against specific store on a specific node.
196 - Added helper methods for diong put & get for specific node/store
197 - Added voldemort.client.protocol.admin.QueryKeyResult type to
198 simplify QueryKey interface
199 * Improved FetchStreamRequestHandler and sub-classes
200 - Renamed all sub-classes: 'FullScan' and 'PartitionScan' prefixes
201 for pertinent stream request handler implementations.
202 - Removed unused skipRecords parameter.
203 - Added recordsPerPartition parameter to fetch limited portion of
205 - All logic about how many keys to fetch (return to invoker) is
208 - Added many options to help improve the balance of (zoned) clusters.
209 - Analysis of the balance of a cluster is significantly more detailed.
210 - Fixed a bug that reduced the balance of a cluster each time it was
212 - Many different algorithms for improving cluster balance are
213 implemented in voldemort.utils.RebalanceClusterUtils
214 * ConsistencyCheck & ConsistencyFixCLI
215 - New tools for ensuring data durability. These tools are necessary
216 because slop creation can fail during put operations.
217 - ConsistencyCheck determines which keys, if any, lack
218 "consistency". I.e., are present on only a subset of the expected
220 - ConsistencyFix takes a list of bad (inconsistent) keys and makes
221 sure they are present on all expected partitions.
222 - ConsistencyFix also has an interface for repairing "orphaned" keys
223 that could result from an aborted rebalance.
224 * KeySamplerCLI & KeyVersionFetcherCLI
225 - KeySamplerCLI is a new tool that reads some number of keys for
226 specified partitions/stores.
227 - KeyVersionFetcherCLI is a new tool that, given a key and a store,
228 fetches the version from all nodes that host a partition that
229 ought to store a replica of the key's value.
230 - Together, KeySamplerCLI and KeyVersionFetcherCLI correctly
231 implement the intended functionality of the Entropy tool (for
232 servers that implement either FullScan and PartitionScan fetches).
233 - Entropy tool had been used in the past to verify a sample of keys
234 before and after a rebalance. Entropy tool does not work as
235 intended/expected. This is exacerbated by the partition aware
236 layouts. Instead of trying to fix the Entropy tool, these two new
237 tools were developed. Entropy is deprecated and will eventually be
238 removed from the code base.
239 * Substantial refactoring of helper & util methods:
240 - voldemort.cluster.Cluster : added helper methods
241 - voldemort.utils.ClusterInstance : wraps up one Cluster &
242 List<StoreDefinition>
243 - voldemort.utils.Cluster : utils for single Cluster object.
244 - voldemort.utils.NodeUtils : utils for Node object.
245 - voldemort.utils.RebalanceUtils : Many methods moved to more
246 appropriate helper classes
247 - voldemort.utils.StoreDefinitionUtils : utils for StoreDefinition
249 - voldemort.utils.StoreInstance : wraps up one Cluster & one
252 - ByteUtils toHexString & from HexString now rely on standard
254 - voldemort.client.AdminFetchTest now tests FullScan and
255 PartitionScan fetches
256 - voldemort.store.routed.ReadRepairerTest annotated all tests with
260 Release 1.3.0 on 03/08/2013
262 NOTE: This is an open source release! This release can be downloaded here:
263 http://github.com/voldemort/voldemort/downloads.
265 Changes made since 1.2.3
266 * VoldemortConfig and ClientConfig now contain detailed documentation
267 * BDB-JE defaults set to ones in prod@linkedin
268 * Bug fixes on kerberos support for Hadoop
271 Release 1.2.3 on 02/20/2013
273 Changes made since 1.2.2
274 * Added a retry loop and synchronized block while getting Hadoop FS
275 * Code cleanup in HdfsFetcher to make it more readable.
276 * Throwing explicit exceptions in HdfsFetcher instead of
277 returning null to be more precise in the Azkaban logs.
280 Release 1.2.2 on 02/19/2013
282 Changes made since 1.2.1
283 * Synchronized the streaming API
284 * Fixed some of the streaming API tests.
287 Release 1.2.1 on 0/30/2013
289 Changes made since 1.2.0
290 * Added a Streaming API and related tests.
291 * Refactoring of the admin client apis into functional inner classes
294 Release 1.2.0 on 01/21/2013
296 Changes made since 1.1.9
297 * Added an Admin API to fetch orphaned key / entries
298 * Improved some tests related to streaming API.
299 * Correcting commons-codec version in ivy file (1.4)
302 Release 1.1.9 on 01/15/2013
304 Changes made since 1.1.8
305 * Asynchronous socket checkout improvements
306 * Changed checkout behavior of KeyedResourcePool to only create new
307 connections when there are no resources available (rather than
308 creating new connections until the pool is full)
309 * Changed QueuedKeyedResourcePool.reset behavior to better match
310 KeyedResourcePool (i.e., to not cancel queued asynchronous
311 requests unnecessarily)
312 * Removed (unnecessary) synchronization primitives from keyed resource pool
313 * Reduce granularity of failure detector locking within ThresholdFailureDetector
314 * Minor features/improvements
315 * Less verbose logging in the face of expected exceptions and errors
316 * Refactored (Queued)KeyedResourcePoolTest
318 * Fixed possible race condition for resource creation in KeyedResourcePool
319 * More efficient (time & space) and simpler Histogram implementation
323 Release 1.1.8 on 01/14/2013
325 Changes made since release 1.1.7
326 * Enhanced Server Monitoring
328 -- Streaming operations to the server
329 -- BDB storage exception counts
330 * Ability to turn off BDB Checkpointing during batch modifications
331 * Added ability to delete old checksum files in Build and Push reducer
332 * Upgrade Hadoop jar to 1.0.4-p2
335 Release 1.1.7 on 01/03/2013
337 NOTE: This release is based off of release 1.1.4
339 Changes made since release 1.1.4
340 * Upgrading Hadoop jar to 1.0.2
341 * Added support for Kerberos authentication in HdfsFetcher
342 * Extra config parameters for Kerberos config and keytab file
345 NOTE: Release 1.1.5 and 1.1.6 are special client side releases
346 not based off of master. 1.1.5 was rolled back to to a weird bug.
347 1.1.6 is a special client side release including Auto-
348 bootstrapper and Versioned Avro support.
351 Release 1.1.4 on 11/29/2012
353 Changes made since release 1.1.3
354 * Added BDB parameters to control LRU behavior in cache & proactive cleaner migration
355 * Added a mlock fix for pinning the indexes of RO stores in memory
358 Release 1.1.3 on 11/28/2012
360 Changes made since release 1.1.2
361 * Fixed a bug in the build and push job, specifically the Mapper
362 that caused collisions
363 * Added retry mechanism with the HDFS fetcher for hftp
366 Release 1.1.2 on 10/31/2012
368 Changes made since release 1.1.1
369 * Reverted a change to voldemort.versioning.Versioned.getVersion() so
370 that a Version is returned as our clients expect.
373 Release 1.1.1 on 10/30/2012
375 Changes made since release 1.1.0
376 * Fixed connection leak in ClientRequestExecutorFactory
377 * Changed client to default to DefaultStoreClient
380 Release 1.1.0 on 10/19/2012
382 Changes made since release 1.0.0
384 IMPORTANT NOTE : This release has significant changes to the BDB storage layer.
385 Users are required to read the bin/PREUPGRADE_FOR_1_1_X_README file
386 thoroughly before attempting to upgrade to 1.1.0. The necessary data
387 conversion will be done through bin/voldemort-convert-bdb.sh
389 * Upgrading to JE 4.1.17
390 * New data format that handles conflicting updates in Voldemort more
392 * Move data off heap and only use it for Index
393 * When scanning, evict whatever you bring in right away.
394 * Partition based scan api to dramatically speed up rebalancing & restore
395 using Partition aware scans (you exactly scan whatever you want to fetch)
396 * Flexible knobs to control scheduling of DataCleanupJob
399 Release 1.0.0 on 10/17/2012
401 NOTE: The large version number jump from 0.96 to 1.0.0 is to
402 standardize on a version number of the sort MAJOR.MINOR.PATCH. This
403 change is part of our effort to treat internal and open source
404 releases in a much more similar manner. Along these lines, release
405 notes for internal releases (like this one) are committed on the
406 master branch. We hope this improves transparency as we work towards
407 the next open source release.
409 Changes made since release 0.96
411 * Auto bootstrapping: ZenStoreClient and System stores
412 * Added server side system stores for managing metadata
413 * ZenStoreClient interacts with system stores
414 * ZenStoreClient auto bootstraps whenever cluster.xml or stores.xml changes
415 * Added a new routing strategy to route to all with local preference
416 * Added a client-registry for publishing client info and config values
417 * Updated LazyClientStore to try to bootstrap during Init
418 * Modified Failure Detector to work on a shared cluster object reference
419 * Avro: schema evolution and read only support
420 * Added new Avro serializer type that supports schema evolution
421 * Added Avro support to read only stores
422 * Added LinkedIn build-and-push Azkaban jobs to build read only stores to contrib
423 * Added a schema backwards compatibility check to VoldemortAdminTool and on server startup to prevent mishaps due to bad schemas
424 * Non-blocking IO: Fixed design flaw that blocked in the face of slow servers
425 * Asynchronous operations no longer do a blocking checkout to get a SocketDestination
426 * Added additional stats collection for better visibility into request queues
428 * Enhanced VoldemortAdminTool to update store metadata version
429 * Enhanced VoldemortAdminTool to work with the new system stores
430 * Added feature to voldemort-shell.sh to dump byte & object arrays
431 * Added a SlowStorageEngine for testing degraded mode performance
432 * Added mechanism to isolate BDB cache usage among stores
433 * Enhanced debug logging (for traffic analysis).
434 * Python client bug fixes (from pull request)
435 * Improved messages in request tracing
436 * Cleaned up help/usage messages within the client shell
437 * Added server config to control socket backlog
438 * Added "--query-keys" option to query multiple keys of multiple stores from specific node
439 * Added control to DataCleanupJob Frequency
440 * Unified jmxid as the factory across the board
442 * bin/generate_cluster_xml.py to generate cluster.xml
443 * bin/repeat-junit.sh and bin/repeat-junit-test.sh to repeatedly run tests
445 * Changed getall return behavior to comply with javadoc
446 * Fixed a bug that caused unnecessary serial requests in getall
447 * HFTP performance issue bug fix (fix in byte buffer and copy process)
448 * Fixed a bug that prevented "--fetch-keys" and "--fetch-entries" in admin tool from showing multiple store results
449 * Fixed problem in sample config that prevented the server from starting
450 * Fixed some intermittent BindException failures across many unit tests
451 * Fixed some intermittent rebalance test failures
452 * Wrapped long running tests with timeouts
455 Release 0.96 on 09/05/2012
457 Changes made since 0.90.1
460 * Append cluster name to various mbeans for better stats display
461 * Implement average throughput in bytes
463 * Add 95th and 99th latency tracking
464 * Add stats for ClientRequestExecutorPool
465 * Add error/exception count and max getall count
466 * BDB+ Data cleanup Monitoring changes
468 * Donor-based rebalancing and post cleanup (see https://github.com/voldemort/voldemort/wiki/Voldemort-Donor-Based-Rebalancing for more details)
469 * Rebalancing integration testing framework (under test/integration/voldemort/rebalance/)
470 * Generate multiple cluster.xml files based on the number specified when running the tool and choose the cluster with the smallest std dev as the final-cluster.xml
471 * Add status output to log for updateEntries (used by rebalancing)
472 * Read-only pipeline:
473 * Add hftp and webhdfs support
474 * Read-only bandwidth dynamic throttler
475 * Add minimum throttle limit per store
476 * Add rollback capability to the Admin tool
477 * Voldemort-backed stack and index linked list impl
478 * Change client requests to not process responses after timeout
479 * Modified client request executor timeout to not factor in the NIO selector timeout
480 * Added BDB native backup capabalities, checksum verification and incremental backups (well tested, but not yet used in production)
481 * Add additional client-side tracing for debugging and consistency analytics
482 * Clean up logging during exception at client-side
483 * Security exception handling
484 * Add snappy to CompressionStrategyFactory
485 * Add configurable option to interrupt service being unscheduled
486 * Add logging support for tracking ScanPermit owners (for debugging purposes)
487 * Add a jmx terminate operation for async jobs
488 * Add zone option for restore from replicas
489 * Changing the enable.nio.connector to true by default
490 * Better disconnection handling for python client
491 * Split junit tests into a long and a short test suites
492 * Add separate timeouts for different operations (put, get, delete, and getAll
493 * Allow getAll to return partial results upon timeout
494 * Improved cluster generation tool
495 * Added log4j properties folder for junit test
497 * httpclient 3.x to httpclient 4.x
498 * Fix NPE in listing read-only store versions
499 * Fixed 2 failure detector bugs during rebalancing or node swapping
500 * Fixed a thread leak issue in StreamingSlopPusher
502 * Fixed a bug in TimeBasedInconsistency resolver.
503 * Fixed race condition in client socket close
504 * Fixed a potential deadlock issue in ScanPermitWrapper
505 * Fixed a bug where a read returns null (on rare occations) when being concurrent with a write
506 * Fixed a performance bug in HdfsFetcher when hftp is used
509 Changes made since 0.90
511 * Updated the documentation for Voldemort shell tool in NOTES
512 * Added Admin API to perform Bdb data cleanup (repairJob)
513 and corresponding unit tests
514 * Fixes in restore from replication, store creation code
515 * Improved failure detector configuration. ThresholdFailureDetector
516 is now the default option
517 * Multiple Fixes to the Voldemort ruby client
518 * Added additional Jmx metrics to expose Bdb environment statistics,
519 caching statistics and Voldemort batch operation statistics
520 * Updated default timeout for restore 'from replica' to 365 days
521 * New feature in the performance tool: '--use-sample' option enables
522 'read, write back unmodified' transactions in place of writes in
523 the workload (allows for testing read-write transactions on stores
524 with complex schemas)
525 * Added the ability to dynamically update cluster.xml and
526 reinitialize the scheduler
528 Release 0.90 on 7/10/2011
530 Changes made since 0.81
531 * All upgrading instructions can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
533 - Single consolidated administrative tool - Got rid of the admin client
534 shell and have a better Voldemort admin tool. More documentation -
535 https://github.com/voldemort/voldemort/wiki/Voldemort-Admin-tool
538 - Basic GUI in JRuby ( and Sinatra ), capable of querying for store metadata.
539 - Read contrib/web-manager/README.md for more details
542 - New better rebalancing support with (a) checkpointing (b) progress bar (c) support for RO store rebalancing
543 - Documentation - https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing
544 - In the process solves Issue 203, 288, 305, 307, 311
546 * Client side changes
547 - Pipeline routed store ( with support for topology awareness ) - Changed the default client side routing
548 to use a state machine like system. More documentation - https://github.com/voldemort/voldemort/wiki/RoutedStore-redesign
549 - Lazy store client - Some of our clients are very inactive and don't really do many requests. For such
550 clients it doesn't make sense to bootstrap the metadata at startup. Hence from now onwards clients instantiated
551 from SocketStoreClientFactory give a LazyStoreClient which will not bootstrap till the first request is made
552 - Caching store client factory - We have also checked-in a new store client factory which we use internally
553 at LinkedIn to cache store clients for stores which we repeatedly.
554 - Failure detector - We have fixed some bugs in the ThresholdFailureDetector. This is an improvement over the naive
555 BannageFailureDetector which would aggressively mark nodes down after a single failure. More details -
556 https://github.com/voldemort/voldemort/wiki/Client-side-failure-detector-implementations
557 - Issue 219: StoreClient::put returns a Version - You may have to upgrade your clients since now the StoreClient returns
558 a version to be used in successive requests.
560 * Read-only store changes
561 - Admin based store swapper - Till 0.81 we relied on a servlet based fetcher / swapper.
562 Now we support an admin based store swapper which reports progress.
563 - Other changes - (a) New directory format (b) Two new data format. Support for iterating over read-only data
564 (c) Checksum of data in .metadata file (d) Some monitoring changes of RO stores
567 - Introduction of JNA 3.2.7 - We have introduced JNA 3.2.7 in this release for supporting symbolic links in read-only stores.
568 More details about the new RO format and changes can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
569 - Protocol Buffers 2.3.0 - Upgrading protocol buffers from 2.2.0 to 2.3.0
570 - Avro 1.4.0 - Upgraded Avro from 1.3.0 to 1.4.0
573 - Added support for Krati 0.3.6 as a storage engine ( http://sna-projects.com/krati/ ) -
574 https://github.com/voldemort/voldemort/tree/release-090/contrib/krati
577 - Hinted handoff - https://github.com/voldemort/voldemort/wiki/Hinted-Handoff
578 - Experimental support for server side transforms - https://github.com/voldemort/voldemort/wiki/Server-side-transforms-in-Voldemort
579 - Topology awareness ( i.e. datacenter / rack ) - https://github.com/voldemort/voldemort/wiki/Topology-awareness-capability
580 - Repair Job ( Job which will delete data if the node is not responsible for it )
583 - Tons of JMX changes ( average bytes, etc ) - More details ( https://github.com/voldemort/voldemort/wiki/JMX-Monitoring )
584 - Key distribution generator - Ability to estimate skew of your cluster ( ./bin/run-class.sh voldemort.utils.KeyDistributionGenerator )
587 - Python - Updated Python client with support for Binary JSON serialized stores ( ./clients/python/README )
588 - Ruby - Updated Ruby client ( ./clients/ruby/README.md )
590 Release 0.81 on 6/15/2010
592 Changes made since 0.80.2:
594 * IMPORTANT: we have upgraded the Hadoop Core jar to v0.20.2. Since
595 this version of Hadoop requires Java 6, in order to retain backwards
596 compatibility with Java 5, you will need to replace this jar with
597 the Hadoop 0.18.* core jar.
598 * Multiple donors for Voldemort Rebalancing: speeds up rebalancing, by
599 allowing a single stealer node to transfer partitions from multiple
601 * Features for EC2 Testing
602 * Read-only Stores: backwards compatibility with 0.70, bug fix in
604 * Hadoop InputFormat, Pig LoadFunc in Contrib: ability to perform
605 Map/Reduce (either directly via Hadoop or using Pig) over data in
606 Voldemort stores. See:
607 < contrib/hadoop/test/voldemort/hadoop/VoldemortWordCount.java > for
609 * Voldemort Performance Tool: performance measurement tool based on
610 YCSB (Yahoo Cloud Serving Benchmark) code. Run
611 < bin/voldemort-performance-tool.sh --help > for more information.
612 * Reverted default failure detector implementation back to
613 BannagedPeriodFailureDetector due to potential bugs in the
614 ThresholdFailureDetector
616 Release 0.80.2 on 4/27/2010
618 Changes made since 0.80.1:
620 * Batched NIO writes (improves performance for AdminClient/Streaming
621 operations when the NIO connector is enabled)
622 * VoldemortAdminTool: Added support to specify stores, support to
623 fetch all keys to a binary file, support to save keys in ASCII
624 (JSON) format [experimental], capability to add stores. Run
625 < bin/voldemort-admin-tool.sh --help > for more information.
626 * Minor bug fixes for Rebalancing
627 * Issue 240: Voldemort fetcher should use different temp directories for
629 * Checksum capability during construction of Voldemort read-only stores
631 Release 0.80.1 on 3/23/2010
633 Changes made since 0.80:
635 * Issue 133: Support for Apache Avro as a serialization format
636 * Issue 223: Changed the default client to use
637 TresholdFailureDetector
638 * Fixed issue 222: Revised KeyedResourcePool.close(K key) to fix
640 * Fixed issue 198: Revised ReadRepairer to use a separate copy of the
641 vector clock, fixing a situation where NoSuchElementException would
643 * Miscellaneous enhancement: support for TCP keep-alives, improved
644 read only store utilities, command line interface to AdminClient,
645 improved load testing tools
647 Release 0.80 on 2/18/2010
649 Changes made since 0.70.1:
651 * IMPORTANT: backwards compatibility between the client and server has
652 changed. A backwards incompatibility in the wire protocol was found
653 between releases 0.60 to 0.70.1 and releases prior to 0.60. We chose
654 to make 0.80 compatible with 0.57.1 and earlier versions, while
655 introducing an incompatibility with 0.60-0.70.1. What this means is
656 that if you're presently running 0.60 and higher, you would need to
657 upgrade the Voldemort jar files on *all* servers and clients.
658 * Upgraded the BDB storage engine to use BerkeleyDB-JE 4.0.92,
659 retaining ability to use BerkeleyDB-JE 3.3.* if desired. IMPORTANT:
660 if one switches from BerkeleyDB-JE 3.3.* to 4.0.92 they will be able
661 to access all of existing data. Once a switch has been made to
662 4.0.92 the data will not be readable by earlier versions of BDB. If
663 there's a chance that a roll back to 3.3.* might be needed, the best
664 course of action will be to make a backup of existing data
666 Switching between 3.3.* and 4.0.* would also require rebuilding the
667 class files (e.g., by running "ant clean && ant release" after
668 replacing the BDB jar files).
669 * Compression support for read-only stores
670 * Increased the socket buffer size for transferring read-only
671 stores from Hadoop for improved performance over high-latency links
672 * NIO support for the Admin Service, including Streaming
674 * Support for adding stores on the fly via the Admin
676 * Fixed issue 209: Incorrect object passed to List.contains in
677 RebalanceUtils.getLatestCluster()
678 * Fixed issue 211: Unnecessary read repairs during getAll() with more
680 * Other enhancements: better CLI for rebalancing, throttling in
681 Admin Service is now based on all disk activity
683 Release 0.70.1 on 2/1/2010
685 Changes made since 0.70:
687 * Fixed issue 205: if no keys passed to getAll() were in partitions
688 undergoing rebalancing, proxyGetAll() would be called with an
689 empty list even if rebalancing wasn't happening
691 Release 0.70 on 1/27/2010
693 Changes made since 0.60.1:
695 * A beta of rebalancing (dynamic cluster expansion) support merged
696 into the main branch. See the project's wiki for more information:
697 http://wiki.github.com/voldemort/voldemort/voldemort-rebalancing
698 * New failure detector merged into the main branch:
699 http://wiki.github.com/voldemort/voldemort/failure-detection
700 * Beta mechanism for restoring all of node's data from replicas on
701 demand. This is an alternative to a more gradual mechanism provided
702 by read-repair: useful when a machine is down for a prolonged period
703 and is then re-inserted into the cluster.
704 Invoked via JMX: the operation is restoreDataFromReplication in the
705 voldemort.server.VoldemortServer MBean, with a mandatory parameter
706 (integer >= 1) indicating the number of transfers to do in parallel.
707 * Simple gossip protocol (for cluster metadata) merged
708 into the main branch. Disabled by default: use "enable.gossip=true"
709 to enable, use "gossip.interval.ms" to set an interval at which gossip
710 occurs (default: 30000 i.e., 30 seconds).
711 * Fixed issue 190: add a way of aggregating performance data over
713 * Fixed issue 181: stack trackes shouldn't be filled for Obsolete
716 Release 0.60.1 on 12/18/2009
718 Minor changes made since 0.60:
720 * Better logging in the exception thrown if config/.temp and
721 config/.version are copied
722 * Bumping up the version to 0.60.1 in order to release updated
723 archives, fixing an error in the stores.xml for single_node_cluster
726 Release 0.60 on 12/15/2009
728 Changes made since 0.57.1:
730 * Admin Client/Server API: adds support for streaming-based transfer
731 of entries between nodes, deleting entries on remote nodes,
732 remotely deleting and updating metadata
733 * EC2 testing: a way to periodically run integration and performance
734 tests which involve Voldemort instances on different machines
735 * Experimental support for views
736 * Interpolation search for read-only stores
737 * Support for large lists and strings in the JSON serializer
738 * LZF compression support
739 * Ruby client contributed
740 * Fixed issue 170: hanging if a port is used by another process
741 * Fixed issue 122: suspicious integer division in
742 RequestCounter.getThroughput
743 * Miscellaneous improvements and bug fixes for read-only stores
744 * Fixed issue 168: added StorageEngine.keys()
746 Release 0.57.1 on 11/27/2009
748 Minor change made since 0.57:
750 * Modified build.xml to exclude .git directory from release tarballs/zipfiles
752 Release 0.57 on 11/16/2009
754 The following changes were made since 0.56:
756 * Fixed an issue in ReadOnlyEngine's close() method
757 * Fixed hidden logging in StorageService
758 * Fix for issue 163 (lock mode during get)
759 * Exposed bdb environment stats with setFast(false)
761 Release 0.56 on 10/26/2009
763 The following changes were made since 0.55:
765 * Fix for issue 164: Changed default bdb.max.logfile.size to 60MB
766 * Make file deletes asynchronous in read only store swap
767 * Added better debug logs for bdb stats
768 * Fixed race condition in AbstractSocketPoolTest
769 * Added improved monitoring for bdb stats
770 * Added backoff and retry logic in bootstrap code
771 * Not logging obosleteVersionException(s) or counting it in JMX exception count
773 * C++ client building on OS X
774 * Rely only on the number of versions returned to decide whether to retrieve
775 the value for put(K,V)
776 * Implemented issue 152: getVersion() API in Store
778 Release 0.55 on 10/7/2009
780 The following changes were made since 0.52 (in summary):
782 * Add an event throttler
783 * Added DataCleanupJob
784 * Protocol buffers at version 2.2.0
785 * BDB JE upgraded to 3.3.87
786 * Added data compression support (enable by adding
790 to value-serializer or key-serializer sections in stores.xml)
791 * Added a resource pool/socket pool implementation (no longer using
793 * Added server-side NIO support (enabled by setting
794 enable.nio.connector=true
795 in server.properties)
796 * Improved the efficiency of the protocol buffers network protocol by using
797 CodedOutputStream/CodedInputStream (zero copy transfers)
798 * Fix for issue #21: incorrectness in vector clock inconsistency resolver
799 * Upgrade google-collections
800 * Support for building read only stores in Hadoop