1 Release 1.8.4 on 06/03/2014
3 * Fix RESTClientFactory for a clean shutdown
4 * Add check for SocketAndStreams in sendAndReceive
5 * Added the ability to query a single store using AdminClient / admin tool
7 Release 1.8.3 on 05/27/2014
9 * Add debug log messages for CoordinatorService
10 * Fix NetworkClassLoader bug when running in Windows OS (Carlos Tasada)
11 * Add junit support for gradle
12 * Fix bind failures while running tests
13 * Add overwrite option to fork lift tool
14 * Use standard logger in the StreamingClient constructor
15 * [python client] Fixed reconnect in _execute_request (Søren Holbech)
16 * Fix the admin metadata check command and did same change for old admin command.
18 Release 1.8.2 on 05/09/2014
20 * Bug fix in MetadataStore regarding updation of routing strategy
22 Release 1.8.1 on 05/01/2014
24 * Fix AbstractStoreClientFactory for backwards compatibility
25 * Cleaned up logging messages
26 * Unit tests for big Binary data
27 * Refactored voldemort admin tool - Phase 1 and Phase 2
29 Release 1.8.0 on 04/21/2014
31 * Split the stores.xml into individual stores on Voldemort server
32 * Maintaing backwards compatibility of MetadataStore
33 * DefaultStoreClient bootstraps using store name (instead of stores.xml)
34 * New Admin tool functionality to update stores (instead of replace)
36 Release 1.7.3 on 04/16/2014
38 * Fix RESTClientFactory to return a LazyStoreClient
40 Release 1.7.2 on 04/15/2014
42 * Quota limiting related fix
43 * Coordinator related fix
45 Release 1.7.1 on 04/10/2014
49 Release 1.7.0 on 04/09/2014
51 * Improves BDB coversion process and documentations
52 * Adds scala build support and shell (Saurabh Bhatia)
53 * Add removeStorageEngine call to StorageConfiguration
55 Release 1.6.9 on 03/19/2014
57 * Fix a big where StreamingClient will throw NPE if not initialized
58 * Fix multiple broken tests
59 * Modified RO Rebalance Test to cover one more case
60 * Adding thin client shell for Voldemort Coordinator
63 Release 1.6.8 on 02/21/2014
65 * Add a tool to dump BDB data to Text and the reverse
66 * Fix racing condition problem in CachingStoreClientFactory
67 * Testing, client shell and admin tool enhancements from suletm,
68 ahimta, bhsaurabh and Holden Caufield
69 * ConsistencyCheckTool enhancement
70 * Fix bug in Coordinator for enabling different startup ports
72 Release 1.6.7 on 02/07/2014
74 * Updating the ivy file to refer to tusk-0.0.2 (build fix for nuage)
76 Release 1.6.6 on 02/07/2014
78 * Quota management in Voldemort
79 * Creation of read only stores in RO2 format
80 * Updation of query key feature in admin tool
82 Release 1.6.5 on 02/04/2014
84 * Adding AdminClient functionality to update store definitions
86 Release 1.6.4 on 02/03/2014
88 * Fix bug when RO server NPE when rolled back
89 * RO bug fix caused by RW optimization code
91 Release 1.6.3 on 01/24/2014
93 * Fix KeyVersionFetcher to not depend on nodeid 0
95 Release 1.6.2 on 01/10/2014
97 * Slops stored for nodes no longer in cluster, will die automatically
98 * Admin command to selectively purge slops destined to nodes/zones/stores
99 * Admin command to atomically update cluster.xml, stores.xml in one go
100 * Added admin command show-routing-plan to trace routing table for a key
101 * ZoneClipperCLI also adjusts store definitions based on dropped zone
102 * Numerous tests around non contiguous node ids/zone ids
104 Release 1.6.1 on 12/13/2013
106 * Avro support for fetch keys/entries admin command
107 * Store Clients are now cached by default, within the same factory
108 * Rigoruous tests around non contiguous node ids/zone ids
109 * ZenStoreClient efficient resource managerment
112 Release 1.6.0 on 11/26/2013
114 NOTE: This is an open source release! This release can be downloaded here:
115 https://github.com/voldemort/voldemort/releases
117 Changes made since 1.5.9:
119 * Fixes to remove dependency on non contiguous node ids
120 * Stats for store client
123 Release 1.5.9 on 11/05/2013
125 * Bug fix in AsyncMetadataVersionManager (related to store version tracking)
127 Release 1.5.8 on 10/29/2013
129 * Replace resthdfs jar with tusk.jar
130 * Incorporate retries into rest hdfs fetcher
132 Release 1.5.7 on 10/28/2013
134 * Voldemort now uses BDB 5.0.88
135 * Support for RO fetches via RestHdfs
136 * Fix NPE in Streaming Client
137 * Retrying fetches on all exceptions w/ delay
139 Release 1.5.6 on 10/18/2013
141 * Removed getStoreClientFactoryStats from StoreClientFactory interface (fix)
143 Release 1.5.5 on 10/18/2013
145 * Client shell with avro support
146 * Fix bind exceptions
147 * Fix ObsoleteVersion Exceptions
148 * Build once and push to multiple clusters
149 * Improvements to failure detector
151 * Prevent NPE in worker thread when shutdown
152 * Bug fixes in repair and prune job
154 Release 1.5.3 on 09/19/2013
156 * Bug fixes (NPE during shutdown
157 * Client side visibility changes (Histogram, Connect exception)
158 * Adding PruneJob and timebased resolving mechanism to streaming client
159 * R2Store cleanup required for the rest client
160 * Monitoring improvements (aggregated BDB stats, streaming stats)
161 * Monitoring and Config improvements for REST service
162 * Removed donor based rebalancing code
163 * Removed replica type from the code base
164 * Improvements to Rebalance controller
166 Release 1.4.7 on 09/04/13
169 * Added the Zone affinity feature on the client side
170 * Refactoring the rest package (Rest server and Coordinator)
171 * Using standard Netty pipeline in the Coordinator
172 * Added monitoring hooks for the Rest Service
174 Release 1.4.6 on 07/24/2013
177 * Cleanup of R2Store to work with the REST server
178 * Modified ZenStoreClient to avoid creating SystemStores during each
180 * Added slop streaming functionality
182 Release 1.4.4 on 07/08/2013
184 Changes made since 1.4.3
185 * Change Repartioner to swap among nodes from all specified zones
186 * Added rebalance shuffle, cluster expansion scripts
188 Release 1.4.3 on 07/03/2013
190 Note: Server changes are not backwards compatible. To use new
191 rebalancing tooling, servers must be upgraded before hand.
194 - Proxying / abortable rebalance
195 - added proxy puts which rebalance safely abortable
196 - improved proxy gets to check locally first and only fetch remote
198 - Repartitioning: improved algorithms to optimize partition layout
199 - Plan: avoids cross-zone data movement in most rebalance cases. Only
200 expanding into a new zone requires data movement across zones.
202 - better / more accurate progress monitoring.
203 - 'proxy pause' before rebalancing. Ensures clients pick up new
204 metadata and allows performance with proxy'ing to be observed
205 before servers start rebalancing.
206 - NOTE: Donor-based rebalance is currently broken by these changes.
208 - PartitionBalanceCLI: more nuanced and verbose analysis of cluster
210 - RepartitionerCLI: Stand alone tool to determine repartitioning
211 - Stand alone 'repartitioning' scripts for key use cases: new cluster,
212 shuffle (existing cluster), cluster expansion, and zone expansion.
213 - RebalancePlanCLI: Stand alone plan tool to determine cost of a
215 - RebalanceControllerCLI: Stand-alone tool for executing a specific
219 - moved to Guava from Google Collections
220 - better unit test coverage of zoned clusters
221 - added example prod configuration
222 - added tools directory
223 - .gitignore improvements
225 - fixed concurrency bugs in metadata store
226 - fixed vector clock comparisons
228 - write slops in more cases when they are needed
229 - KeySampler/Fetcher handle larger working sets with OOME
231 Changes made since 1.4.2
233 * Faster creation of BaseStoreRoutingPlan objects in the fast rebalancing path
235 Release 1.4.2 on 06/27/2013
236 * Fixing costly StoreRoutingPlan object construction
238 Release 1.4.0 on 06/21/2013
239 * Zone expansion release
241 Release 1.3.4 on 06/19/2013
242 * Read Write store bug fixes
243 - Rewrite of InMemoryStorageEngine + config to control multiVersionPuts
244 - Fixed a bug in the read-repair logic which was causing unnecessary puts.
245 - Fixed put operation in PipelineRoutedStore to ensure slop is submitted
246 - Fixed VectorClock bug triggered during versioned puts
247 * Rebalance improvement
248 - Made it safe to abort rebalance
249 - Proxy puts are established so that 'old' partition is updated and so an aborted rebalance can "roll back" without any data loss
250 - Proxy gets & puts are established within the zone making improving performance during rebalance
252 - Added monitoring capability in the Coordinator
254 Release 1.3.3 on 04/24/2013
255 * VoldemortBuildandPush
256 - Fixed bug with schema check
258 - Fixed issue with redundant callback invocation
259 Release 1.3.1 on 03/25/2013
261 - Fixed the bug in calculating checksums when we entere a retry loop
262 - refactored per file checksums
263 - added junit test case to simulate intermittent IO exceptions
264 * voldemort.client.protocol.admin.AdminClient
265 - Added AdminStoreClient so that AdminClient can do store operations
266 against specific store on a specific node.
267 - Added helper methods for diong put & get for specific node/store
268 - Added voldemort.client.protocol.admin.QueryKeyResult type to
269 simplify QueryKey interface
270 * Improved FetchStreamRequestHandler and sub-classes
271 - Renamed all sub-classes: 'FullScan' and 'PartitionScan' prefixes
272 for pertinent stream request handler implementations.
273 - Removed unused skipRecords parameter.
274 - Added recordsPerPartition parameter to fetch limited portion of
276 - All logic about how many keys to fetch (return to invoker) is
279 - Added many options to help improve the balance of (zoned) clusters.
280 - Analysis of the balance of a cluster is significantly more detailed.
281 - Fixed a bug that reduced the balance of a cluster each time it was
283 - Many different algorithms for improving cluster balance are
284 implemented in voldemort.utils.RebalanceClusterUtils
285 * ConsistencyCheck & ConsistencyFixCLI
286 - New tools for ensuring data durability. These tools are necessary
287 because slop creation can fail during put operations.
288 - ConsistencyCheck determines which keys, if any, lack
289 "consistency". I.e., are present on only a subset of the expected
291 - ConsistencyFix takes a list of bad (inconsistent) keys and makes
292 sure they are present on all expected partitions.
293 - ConsistencyFix also has an interface for repairing "orphaned" keys
294 that could result from an aborted rebalance.
295 * KeySamplerCLI & KeyVersionFetcherCLI
296 - KeySamplerCLI is a new tool that reads some number of keys for
297 specified partitions/stores.
298 - KeyVersionFetcherCLI is a new tool that, given a key and a store,
299 fetches the version from all nodes that host a partition that
300 ought to store a replica of the key's value.
301 - Together, KeySamplerCLI and KeyVersionFetcherCLI correctly
302 implement the intended functionality of the Entropy tool (for
303 servers that implement either FullScan and PartitionScan fetches).
304 - Entropy tool had been used in the past to verify a sample of keys
305 before and after a rebalance. Entropy tool does not work as
306 intended/expected. This is exacerbated by the partition aware
307 layouts. Instead of trying to fix the Entropy tool, these two new
308 tools were developed. Entropy is deprecated and will eventually be
309 removed from the code base.
310 * Substantial refactoring of helper & util methods:
311 - voldemort.cluster.Cluster : added helper methods
312 - voldemort.utils.ClusterInstance : wraps up one Cluster &
313 List<StoreDefinition>
314 - voldemort.utils.Cluster : utils for single Cluster object.
315 - voldemort.utils.NodeUtils : utils for Node object.
316 - voldemort.utils.RebalanceUtils : Many methods moved to more
317 appropriate helper classes
318 - voldemort.utils.StoreDefinitionUtils : utils for StoreDefinition
320 - voldemort.utils.StoreInstance : wraps up one Cluster & one
323 - ByteUtils toHexString & from HexString now rely on standard
325 - voldemort.client.AdminFetchTest now tests FullScan and
326 PartitionScan fetches
327 - voldemort.store.routed.ReadRepairerTest annotated all tests with
331 Release 1.3.0 on 03/08/2013
333 NOTE: This is an open source release! This release can be downloaded here:
334 http://github.com/voldemort/voldemort/downloads.
336 Changes made since 1.2.3
337 * VoldemortConfig and ClientConfig now contain detailed documentation
338 * BDB-JE defaults set to ones in prod@linkedin
339 * Bug fixes on kerberos support for Hadoop
342 Release 1.2.3 on 02/20/2013
344 Changes made since 1.2.2
345 * Added a retry loop and synchronized block while getting Hadoop FS
346 * Code cleanup in HdfsFetcher to make it more readable.
347 * Throwing explicit exceptions in HdfsFetcher instead of
348 returning null to be more precise in the Azkaban logs.
351 Release 1.2.2 on 02/19/2013
353 Changes made since 1.2.1
354 * Synchronized the streaming API
355 * Fixed some of the streaming API tests.
358 Release 1.2.1 on 0/30/2013
360 Changes made since 1.2.0
361 * Added a Streaming API and related tests.
362 * Refactoring of the admin client apis into functional inner classes
365 Release 1.2.0 on 01/21/2013
367 Changes made since 1.1.9
368 * Added an Admin API to fetch orphaned key / entries
369 * Improved some tests related to streaming API.
370 * Correcting commons-codec version in ivy file (1.4)
373 Release 1.1.9 on 01/15/2013
375 Changes made since 1.1.8
376 * Asynchronous socket checkout improvements
377 * Changed checkout behavior of KeyedResourcePool to only create new
378 connections when there are no resources available (rather than
379 creating new connections until the pool is full)
380 * Changed QueuedKeyedResourcePool.reset behavior to better match
381 KeyedResourcePool (i.e., to not cancel queued asynchronous
382 requests unnecessarily)
383 * Removed (unnecessary) synchronization primitives from keyed resource pool
384 * Reduce granularity of failure detector locking within ThresholdFailureDetector
385 * Minor features/improvements
386 * Less verbose logging in the face of expected exceptions and errors
387 * Refactored (Queued)KeyedResourcePoolTest
389 * Fixed possible race condition for resource creation in KeyedResourcePool
390 * More efficient (time & space) and simpler Histogram implementation
394 Release 1.1.8 on 01/14/2013
396 Changes made since release 1.1.7
397 * Enhanced Server Monitoring
399 -- Streaming operations to the server
400 -- BDB storage exception counts
401 * Ability to turn off BDB Checkpointing during batch modifications
402 * Added ability to delete old checksum files in Build and Push reducer
403 * Upgrade Hadoop jar to 1.0.4-p2
406 Release 1.1.7 on 01/03/2013
408 NOTE: This release is based off of release 1.1.4
410 Changes made since release 1.1.4
411 * Upgrading Hadoop jar to 1.0.2
412 * Added support for Kerberos authentication in HdfsFetcher
413 * Extra config parameters for Kerberos config and keytab file
416 NOTE: Release 1.1.5 and 1.1.6 are special client side releases
417 not based off of master. 1.1.5 was rolled back to to a weird bug.
418 1.1.6 is a special client side release including Auto-
419 bootstrapper and Versioned Avro support.
422 Release 1.1.4 on 11/29/2012
424 Changes made since release 1.1.3
425 * Added BDB parameters to control LRU behavior in cache & proactive cleaner migration
426 * Added a mlock fix for pinning the indexes of RO stores in memory
429 Release 1.1.3 on 11/28/2012
431 Changes made since release 1.1.2
432 * Fixed a bug in the build and push job, specifically the Mapper
433 that caused collisions
434 * Added retry mechanism with the HDFS fetcher for hftp
437 Release 1.1.2 on 10/31/2012
439 Changes made since release 1.1.1
440 * Reverted a change to voldemort.versioning.Versioned.getVersion() so
441 that a Version is returned as our clients expect.
444 Release 1.1.1 on 10/30/2012
446 Changes made since release 1.1.0
447 * Fixed connection leak in ClientRequestExecutorFactory
448 * Changed client to default to DefaultStoreClient
451 Release 1.1.0 on 10/19/2012
453 Changes made since release 1.0.0
455 IMPORTANT NOTE : This release has significant changes to the BDB storage layer.
456 Users are required to read the bin/PREUPGRADE_FOR_1_1_X_README file
457 thoroughly before attempting to upgrade to 1.1.0. The necessary data
458 conversion will be done through bin/voldemort-convert-bdb.sh
460 * Upgrading to JE 4.1.17
461 * New data format that handles conflicting updates in Voldemort more
463 * Move data off heap and only use it for Index
464 * When scanning, evict whatever you bring in right away.
465 * Partition based scan api to dramatically speed up rebalancing & restore
466 using Partition aware scans (you exactly scan whatever you want to fetch)
467 * Flexible knobs to control scheduling of DataCleanupJob
470 Release 1.0.0 on 10/17/2012
472 NOTE: The large version number jump from 0.96 to 1.0.0 is to
473 standardize on a version number of the sort MAJOR.MINOR.PATCH. This
474 change is part of our effort to treat internal and open source
475 releases in a much more similar manner. Along these lines, release
476 notes for internal releases (like this one) are committed on the
477 master branch. We hope this improves transparency as we work towards
478 the next open source release.
480 Changes made since release 0.96
482 * Auto bootstrapping: ZenStoreClient and System stores
483 * Added server side system stores for managing metadata
484 * ZenStoreClient interacts with system stores
485 * ZenStoreClient auto bootstraps whenever cluster.xml or stores.xml changes
486 * Added a new routing strategy to route to all with local preference
487 * Added a client-registry for publishing client info and config values
488 * Updated LazyClientStore to try to bootstrap during Init
489 * Modified Failure Detector to work on a shared cluster object reference
490 * Avro: schema evolution and read only support
491 * Added new Avro serializer type that supports schema evolution
492 * Added Avro support to read only stores
493 * Added LinkedIn build-and-push Azkaban jobs to build read only stores to contrib
494 * Added a schema backwards compatibility check to VoldemortAdminTool and on server startup to prevent mishaps due to bad schemas
495 * Non-blocking IO: Fixed design flaw that blocked in the face of slow servers
496 * Asynchronous operations no longer do a blocking checkout to get a SocketDestination
497 * Added additional stats collection for better visibility into request queues
499 * Enhanced VoldemortAdminTool to update store metadata version
500 * Enhanced VoldemortAdminTool to work with the new system stores
501 * Added feature to voldemort-shell.sh to dump byte & object arrays
502 * Added a SlowStorageEngine for testing degraded mode performance
503 * Added mechanism to isolate BDB cache usage among stores
504 * Enhanced debug logging (for traffic analysis).
505 * Python client bug fixes (from pull request)
506 * Improved messages in request tracing
507 * Cleaned up help/usage messages within the client shell
508 * Added server config to control socket backlog
509 * Added "--query-keys" option to query multiple keys of multiple stores from specific node
510 * Added control to DataCleanupJob Frequency
511 * Unified jmxid as the factory across the board
513 * bin/generate_cluster_xml.py to generate cluster.xml
514 * bin/repeat-junit.sh and bin/repeat-junit-test.sh to repeatedly run tests
516 * Changed getall return behavior to comply with javadoc
517 * Fixed a bug that caused unnecessary serial requests in getall
518 * HFTP performance issue bug fix (fix in byte buffer and copy process)
519 * Fixed a bug that prevented "--fetch-keys" and "--fetch-entries" in admin tool from showing multiple store results
520 * Fixed problem in sample config that prevented the server from starting
521 * Fixed some intermittent BindException failures across many unit tests
522 * Fixed some intermittent rebalance test failures
523 * Wrapped long running tests with timeouts
526 Release 0.96 on 09/05/2012
528 Changes made since 0.90.1
531 * Append cluster name to various mbeans for better stats display
532 * Implement average throughput in bytes
534 * Add 95th and 99th latency tracking
535 * Add stats for ClientRequestExecutorPool
536 * Add error/exception count and max getall count
537 * BDB+ Data cleanup Monitoring changes
539 * Donor-based rebalancing and post cleanup (see https://github.com/voldemort/voldemort/wiki/Voldemort-Donor-Based-Rebalancing for more details)
540 * Rebalancing integration testing framework (under test/integration/voldemort/rebalance/)
541 * Generate multiple cluster.xml files based on the number specified when running the tool and choose the cluster with the smallest std dev as the final-cluster.xml
542 * Add status output to log for updateEntries (used by rebalancing)
543 * Read-only pipeline:
544 * Add hftp and webhdfs support
545 * Read-only bandwidth dynamic throttler
546 * Add minimum throttle limit per store
547 * Add rollback capability to the Admin tool
548 * Voldemort-backed stack and index linked list impl
549 * Change client requests to not process responses after timeout
550 * Modified client request executor timeout to not factor in the NIO selector timeout
551 * Added BDB native backup capabalities, checksum verification and incremental backups (well tested, but not yet used in production)
552 * Add additional client-side tracing for debugging and consistency analytics
553 * Clean up logging during exception at client-side
554 * Security exception handling
555 * Add snappy to CompressionStrategyFactory
556 * Add configurable option to interrupt service being unscheduled
557 * Add logging support for tracking ScanPermit owners (for debugging purposes)
558 * Add a jmx terminate operation for async jobs
559 * Add zone option for restore from replicas
560 * Changing the enable.nio.connector to true by default
561 * Better disconnection handling for python client
562 * Split junit tests into a long and a short test suites
563 * Add separate timeouts for different operations (put, get, delete, and getAll
564 * Allow getAll to return partial results upon timeout
565 * Improved cluster generation tool
566 * Added log4j properties folder for junit test
568 * httpclient 3.x to httpclient 4.x
569 * Fix NPE in listing read-only store versions
570 * Fixed 2 failure detector bugs during rebalancing or node swapping
571 * Fixed a thread leak issue in StreamingSlopPusher
573 * Fixed a bug in TimeBasedInconsistency resolver.
574 * Fixed race condition in client socket close
575 * Fixed a potential deadlock issue in ScanPermitWrapper
576 * Fixed a bug where a read returns null (on rare occations) when being concurrent with a write
577 * Fixed a performance bug in HdfsFetcher when hftp is used
580 Changes made since 0.90
582 * Updated the documentation for Voldemort shell tool in NOTES
583 * Added Admin API to perform Bdb data cleanup (repairJob)
584 and corresponding unit tests
585 * Fixes in restore from replication, store creation code
586 * Improved failure detector configuration. ThresholdFailureDetector
587 is now the default option
588 * Multiple Fixes to the Voldemort ruby client
589 * Added additional Jmx metrics to expose Bdb environment statistics,
590 caching statistics and Voldemort batch operation statistics
591 * Updated default timeout for restore 'from replica' to 365 days
592 * New feature in the performance tool: '--use-sample' option enables
593 'read, write back unmodified' transactions in place of writes in
594 the workload (allows for testing read-write transactions on stores
595 with complex schemas)
596 * Added the ability to dynamically update cluster.xml and
597 reinitialize the scheduler
599 Release 0.90 on 7/10/2011
601 Changes made since 0.81
602 * All upgrading instructions can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
604 - Single consolidated administrative tool - Got rid of the admin client
605 shell and have a better Voldemort admin tool. More documentation -
606 https://github.com/voldemort/voldemort/wiki/Voldemort-Admin-tool
609 - Basic GUI in JRuby ( and Sinatra ), capable of querying for store metadata.
610 - Read contrib/web-manager/README.md for more details
613 - New better rebalancing support with (a) checkpointing (b) progress bar (c) support for RO store rebalancing
614 - Documentation - https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing
615 - In the process solves Issue 203, 288, 305, 307, 311
617 * Client side changes
618 - Pipeline routed store ( with support for topology awareness ) - Changed the default client side routing
619 to use a state machine like system. More documentation - https://github.com/voldemort/voldemort/wiki/RoutedStore-redesign
620 - Lazy store client - Some of our clients are very inactive and don't really do many requests. For such
621 clients it doesn't make sense to bootstrap the metadata at startup. Hence from now onwards clients instantiated
622 from SocketStoreClientFactory give a LazyStoreClient which will not bootstrap till the first request is made
623 - Caching store client factory - We have also checked-in a new store client factory which we use internally
624 at LinkedIn to cache store clients for stores which we repeatedly.
625 - Failure detector - We have fixed some bugs in the ThresholdFailureDetector. This is an improvement over the naive
626 BannageFailureDetector which would aggressively mark nodes down after a single failure. More details -
627 https://github.com/voldemort/voldemort/wiki/Client-side-failure-detector-implementations
628 - Issue 219: StoreClient::put returns a Version - You may have to upgrade your clients since now the StoreClient returns
629 a version to be used in successive requests.
631 * Read-only store changes
632 - Admin based store swapper - Till 0.81 we relied on a servlet based fetcher / swapper.
633 Now we support an admin based store swapper which reports progress.
634 - Other changes - (a) New directory format (b) Two new data format. Support for iterating over read-only data
635 (c) Checksum of data in .metadata file (d) Some monitoring changes of RO stores
638 - Introduction of JNA 3.2.7 - We have introduced JNA 3.2.7 in this release for supporting symbolic links in read-only stores.
639 More details about the new RO format and changes can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
640 - Protocol Buffers 2.3.0 - Upgrading protocol buffers from 2.2.0 to 2.3.0
641 - Avro 1.4.0 - Upgraded Avro from 1.3.0 to 1.4.0
644 - Added support for Krati 0.3.6 as a storage engine ( http://sna-projects.com/krati/ ) -
645 https://github.com/voldemort/voldemort/tree/release-090/contrib/krati
648 - Hinted handoff - https://github.com/voldemort/voldemort/wiki/Hinted-Handoff
649 - Experimental support for server side transforms - https://github.com/voldemort/voldemort/wiki/Server-side-transforms-in-Voldemort
650 - Topology awareness ( i.e. datacenter / rack ) - https://github.com/voldemort/voldemort/wiki/Topology-awareness-capability
651 - Repair Job ( Job which will delete data if the node is not responsible for it )
654 - Tons of JMX changes ( average bytes, etc ) - More details ( https://github.com/voldemort/voldemort/wiki/JMX-Monitoring )
655 - Key distribution generator - Ability to estimate skew of your cluster ( ./bin/run-class.sh voldemort.utils.KeyDistributionGenerator )
658 - Python - Updated Python client with support for Binary JSON serialized stores ( ./clients/python/README )
659 - Ruby - Updated Ruby client ( ./clients/ruby/README.md )
661 Release 0.81 on 6/15/2010
663 Changes made since 0.80.2:
665 * IMPORTANT: we have upgraded the Hadoop Core jar to v0.20.2. Since
666 this version of Hadoop requires Java 6, in order to retain backwards
667 compatibility with Java 5, you will need to replace this jar with
668 the Hadoop 0.18.* core jar.
669 * Multiple donors for Voldemort Rebalancing: speeds up rebalancing, by
670 allowing a single stealer node to transfer partitions from multiple
672 * Features for EC2 Testing
673 * Read-only Stores: backwards compatibility with 0.70, bug fix in
675 * Hadoop InputFormat, Pig LoadFunc in Contrib: ability to perform
676 Map/Reduce (either directly via Hadoop or using Pig) over data in
677 Voldemort stores. See:
678 < contrib/hadoop/test/voldemort/hadoop/VoldemortWordCount.java > for
680 * Voldemort Performance Tool: performance measurement tool based on
681 YCSB (Yahoo Cloud Serving Benchmark) code. Run
682 < bin/voldemort-performance-tool.sh --help > for more information.
683 * Reverted default failure detector implementation back to
684 BannagedPeriodFailureDetector due to potential bugs in the
685 ThresholdFailureDetector
687 Release 0.80.2 on 4/27/2010
689 Changes made since 0.80.1:
691 * Batched NIO writes (improves performance for AdminClient/Streaming
692 operations when the NIO connector is enabled)
693 * VoldemortAdminTool: Added support to specify stores, support to
694 fetch all keys to a binary file, support to save keys in ASCII
695 (JSON) format [experimental], capability to add stores. Run
696 < bin/voldemort-admin-tool.sh --help > for more information.
697 * Minor bug fixes for Rebalancing
698 * Issue 240: Voldemort fetcher should use different temp directories for
700 * Checksum capability during construction of Voldemort read-only stores
702 Release 0.80.1 on 3/23/2010
704 Changes made since 0.80:
706 * Issue 133: Support for Apache Avro as a serialization format
707 * Issue 223: Changed the default client to use
708 TresholdFailureDetector
709 * Fixed issue 222: Revised KeyedResourcePool.close(K key) to fix
711 * Fixed issue 198: Revised ReadRepairer to use a separate copy of the
712 vector clock, fixing a situation where NoSuchElementException would
714 * Miscellaneous enhancement: support for TCP keep-alives, improved
715 read only store utilities, command line interface to AdminClient,
716 improved load testing tools
718 Release 0.80 on 2/18/2010
720 Changes made since 0.70.1:
722 * IMPORTANT: backwards compatibility between the client and server has
723 changed. A backwards incompatibility in the wire protocol was found
724 between releases 0.60 to 0.70.1 and releases prior to 0.60. We chose
725 to make 0.80 compatible with 0.57.1 and earlier versions, while
726 introducing an incompatibility with 0.60-0.70.1. What this means is
727 that if you're presently running 0.60 and higher, you would need to
728 upgrade the Voldemort jar files on *all* servers and clients.
729 * Upgraded the BDB storage engine to use BerkeleyDB-JE 4.0.92,
730 retaining ability to use BerkeleyDB-JE 3.3.* if desired. IMPORTANT:
731 if one switches from BerkeleyDB-JE 3.3.* to 4.0.92 they will be able
732 to access all of existing data. Once a switch has been made to
733 4.0.92 the data will not be readable by earlier versions of BDB. If
734 there's a chance that a roll back to 3.3.* might be needed, the best
735 course of action will be to make a backup of existing data
737 Switching between 3.3.* and 4.0.* would also require rebuilding the
738 class files (e.g., by running "ant clean && ant release" after
739 replacing the BDB jar files).
740 * Compression support for read-only stores
741 * Increased the socket buffer size for transferring read-only
742 stores from Hadoop for improved performance over high-latency links
743 * NIO support for the Admin Service, including Streaming
745 * Support for adding stores on the fly via the Admin
747 * Fixed issue 209: Incorrect object passed to List.contains in
748 RebalanceUtils.getLatestCluster()
749 * Fixed issue 211: Unnecessary read repairs during getAll() with more
751 * Other enhancements: better CLI for rebalancing, throttling in
752 Admin Service is now based on all disk activity
754 Release 0.70.1 on 2/1/2010
756 Changes made since 0.70:
758 * Fixed issue 205: if no keys passed to getAll() were in partitions
759 undergoing rebalancing, proxyGetAll() would be called with an
760 empty list even if rebalancing wasn't happening
762 Release 0.70 on 1/27/2010
764 Changes made since 0.60.1:
766 * A beta of rebalancing (dynamic cluster expansion) support merged
767 into the main branch. See the project's wiki for more information:
768 http://wiki.github.com/voldemort/voldemort/voldemort-rebalancing
769 * New failure detector merged into the main branch:
770 http://wiki.github.com/voldemort/voldemort/failure-detection
771 * Beta mechanism for restoring all of node's data from replicas on
772 demand. This is an alternative to a more gradual mechanism provided
773 by read-repair: useful when a machine is down for a prolonged period
774 and is then re-inserted into the cluster.
775 Invoked via JMX: the operation is restoreDataFromReplication in the
776 voldemort.server.VoldemortServer MBean, with a mandatory parameter
777 (integer >= 1) indicating the number of transfers to do in parallel.
778 * Simple gossip protocol (for cluster metadata) merged
779 into the main branch. Disabled by default: use "enable.gossip=true"
780 to enable, use "gossip.interval.ms" to set an interval at which gossip
781 occurs (default: 30000 i.e., 30 seconds).
782 * Fixed issue 190: add a way of aggregating performance data over
784 * Fixed issue 181: stack trackes shouldn't be filled for Obsolete
787 Release 0.60.1 on 12/18/2009
789 Minor changes made since 0.60:
791 * Better logging in the exception thrown if config/.temp and
792 config/.version are copied
793 * Bumping up the version to 0.60.1 in order to release updated
794 archives, fixing an error in the stores.xml for single_node_cluster
797 Release 0.60 on 12/15/2009
799 Changes made since 0.57.1:
801 * Admin Client/Server API: adds support for streaming-based transfer
802 of entries between nodes, deleting entries on remote nodes,
803 remotely deleting and updating metadata
804 * EC2 testing: a way to periodically run integration and performance
805 tests which involve Voldemort instances on different machines
806 * Experimental support for views
807 * Interpolation search for read-only stores
808 * Support for large lists and strings in the JSON serializer
809 * LZF compression support
810 * Ruby client contributed
811 * Fixed issue 170: hanging if a port is used by another process
812 * Fixed issue 122: suspicious integer division in
813 RequestCounter.getThroughput
814 * Miscellaneous improvements and bug fixes for read-only stores
815 * Fixed issue 168: added StorageEngine.keys()
817 Release 0.57.1 on 11/27/2009
819 Minor change made since 0.57:
821 * Modified build.xml to exclude .git directory from release tarballs/zipfiles
823 Release 0.57 on 11/16/2009
825 The following changes were made since 0.56:
827 * Fixed an issue in ReadOnlyEngine's close() method
828 * Fixed hidden logging in StorageService
829 * Fix for issue 163 (lock mode during get)
830 * Exposed bdb environment stats with setFast(false)
832 Release 0.56 on 10/26/2009
834 The following changes were made since 0.55:
836 * Fix for issue 164: Changed default bdb.max.logfile.size to 60MB
837 * Make file deletes asynchronous in read only store swap
838 * Added better debug logs for bdb stats
839 * Fixed race condition in AbstractSocketPoolTest
840 * Added improved monitoring for bdb stats
841 * Added backoff and retry logic in bootstrap code
842 * Not logging obosleteVersionException(s) or counting it in JMX exception count
844 * C++ client building on OS X
845 * Rely only on the number of versions returned to decide whether to retrieve
846 the value for put(K,V)
847 * Implemented issue 152: getVersion() API in Store
849 Release 0.55 on 10/7/2009
851 The following changes were made since 0.52 (in summary):
853 * Add an event throttler
854 * Added DataCleanupJob
855 * Protocol buffers at version 2.2.0
856 * BDB JE upgraded to 3.3.87
857 * Added data compression support (enable by adding
861 to value-serializer or key-serializer sections in stores.xml)
862 * Added a resource pool/socket pool implementation (no longer using
864 * Added server-side NIO support (enabled by setting
865 enable.nio.connector=true
866 in server.properties)
867 * Improved the efficiency of the protocol buffers network protocol by using
868 CodedOutputStream/CodedInputStream (zero copy transfers)
869 * Fix for issue #21: incorrectness in vector clock inconsistency resolver
870 * Upgrade google-collections
871 * Support for building read only stores in Hadoop