1 Release 1.8.12 on 08/11/2014
3 * Speeds up vector clock serialization.
4 * Adds a clean up task in the Gradle build that fixes an intermittent issue with builds on mac.
5 * Expands test configurations.
6 * Some fixes and clean ups in the C++ client.
7 * Rewrites RequestCounter so that it leverages the Tehuti metrics lib.
9 Release 1.8.11 on 08/01/2014
11 * Expose http decoder parameters as config
12 * Client Parallel request Exceptions are not reported to failure detector
13 * ClientRequest Executor has infinite timeout and log cleanup
14 * Threshold failure detector bug fixes
16 Release 1.8.10 on 07/23/2014
18 * Registering slops for parallel puts that fail on QuotaExceededException
20 Release 1.8.9 on 07/02/2014
22 * Adding backwards compatibility checks in AdminClient
24 Release 1.8.8 on 06/24/2014
26 * Wrap mime multipart into a mime message to ensure headers are updated in
28 * Fix transport client instantiation and HttpClientFactory shutdown
29 * Fix build.gradle for setting java builder property in .project file
31 Release 1.8.6 on 06/11/2014
33 * Fix related to Quota metrics (store level instead of aggregate)
35 Release 1.8.6 on 06/11/2014
37 * Fix related to quotas - checking per store instead of aggregate
39 Release 1.8.5 on 06/06/2014
40 *Gradle related changes
42 - empty voldemort-contrib directory
43 - gradle wrapper to fetch required version
45 Release 1.8.4 on 06/03/2014
47 * Fix RESTClientFactory for a clean shutdown
48 * Add check for SocketAndStreams in sendAndReceive
49 * Added the ability to query a single store using AdminClient / admin tool
51 Release 1.8.3 on 05/27/2014
53 * Add debug log messages for CoordinatorService
54 * Fix NetworkClassLoader bug when running in Windows OS (Carlos Tasada)
55 * Add junit support for gradle
56 * Fix bind failures while running tests
57 * Add overwrite option to fork lift tool
58 * Use standard logger in the StreamingClient constructor
59 * [python client] Fixed reconnect in _execute_request (Søren Holbech)
60 * Fix the admin metadata check command and did same change for old admin command.
62 Release 1.8.2 on 05/09/2014
64 * Bug fix in MetadataStore regarding updation of routing strategy
66 Release 1.8.1 on 05/01/2014
68 * Fix AbstractStoreClientFactory for backwards compatibility
69 * Cleaned up logging messages
70 * Unit tests for big Binary data
71 * Refactored voldemort admin tool - Phase 1 and Phase 2
73 Release 1.8.0 on 04/21/2014
75 * Split the stores.xml into individual stores on Voldemort server
76 * Maintaing backwards compatibility of MetadataStore
77 * DefaultStoreClient bootstraps using store name (instead of stores.xml)
78 * New Admin tool functionality to update stores (instead of replace)
80 Release 1.7.3 on 04/16/2014
82 * Fix RESTClientFactory to return a LazyStoreClient
84 Release 1.7.2 on 04/15/2014
86 * Quota limiting related fix
87 * Coordinator related fix
89 Release 1.7.1 on 04/10/2014
93 Release 1.7.0 on 04/09/2014
95 * Improves BDB coversion process and documentations
96 * Adds scala build support and shell (Saurabh Bhatia)
97 * Add removeStorageEngine call to StorageConfiguration
99 Release 1.6.9 on 03/19/2014
101 * Fix a big where StreamingClient will throw NPE if not initialized
102 * Fix multiple broken tests
103 * Modified RO Rebalance Test to cover one more case
104 * Adding thin client shell for Voldemort Coordinator
107 Release 1.6.8 on 02/21/2014
109 * Add a tool to dump BDB data to Text and the reverse
110 * Fix racing condition problem in CachingStoreClientFactory
111 * Testing, client shell and admin tool enhancements from suletm,
112 ahimta, bhsaurabh and Holden Caufield
113 * ConsistencyCheckTool enhancement
114 * Fix bug in Coordinator for enabling different startup ports
116 Release 1.6.7 on 02/07/2014
118 * Updating the ivy file to refer to tusk-0.0.2 (build fix for nuage)
120 Release 1.6.6 on 02/07/2014
122 * Quota management in Voldemort
123 * Creation of read only stores in RO2 format
124 * Updation of query key feature in admin tool
126 Release 1.6.5 on 02/04/2014
128 * Adding AdminClient functionality to update store definitions
130 Release 1.6.4 on 02/03/2014
132 * Fix bug when RO server NPE when rolled back
133 * RO bug fix caused by RW optimization code
135 Release 1.6.3 on 01/24/2014
137 * Fix KeyVersionFetcher to not depend on nodeid 0
139 Release 1.6.2 on 01/10/2014
141 * Slops stored for nodes no longer in cluster, will die automatically
142 * Admin command to selectively purge slops destined to nodes/zones/stores
143 * Admin command to atomically update cluster.xml, stores.xml in one go
144 * Added admin command show-routing-plan to trace routing table for a key
145 * ZoneClipperCLI also adjusts store definitions based on dropped zone
146 * Numerous tests around non contiguous node ids/zone ids
148 Release 1.6.1 on 12/13/2013
150 * Avro support for fetch keys/entries admin command
151 * Store Clients are now cached by default, within the same factory
152 * Rigoruous tests around non contiguous node ids/zone ids
153 * ZenStoreClient efficient resource managerment
156 Release 1.6.0 on 11/26/2013
158 NOTE: This is an open source release! This release can be downloaded here:
159 https://github.com/voldemort/voldemort/releases
161 Changes made since 1.5.9:
163 * Fixes to remove dependency on non contiguous node ids
164 * Stats for store client
167 Release 1.5.9 on 11/05/2013
169 * Bug fix in AsyncMetadataVersionManager (related to store version tracking)
171 Release 1.5.8 on 10/29/2013
173 * Replace resthdfs jar with tusk.jar
174 * Incorporate retries into rest hdfs fetcher
176 Release 1.5.7 on 10/28/2013
178 * Voldemort now uses BDB 5.0.88
179 * Support for RO fetches via RestHdfs
180 * Fix NPE in Streaming Client
181 * Retrying fetches on all exceptions w/ delay
183 Release 1.5.6 on 10/18/2013
185 * Removed getStoreClientFactoryStats from StoreClientFactory interface (fix)
187 Release 1.5.5 on 10/18/2013
189 * Client shell with avro support
190 * Fix bind exceptions
191 * Fix ObsoleteVersion Exceptions
192 * Build once and push to multiple clusters
193 * Improvements to failure detector
195 * Prevent NPE in worker thread when shutdown
196 * Bug fixes in repair and prune job
198 Release 1.5.3 on 09/19/2013
200 * Bug fixes (NPE during shutdown
201 * Client side visibility changes (Histogram, Connect exception)
202 * Adding PruneJob and timebased resolving mechanism to streaming client
203 * R2Store cleanup required for the rest client
204 * Monitoring improvements (aggregated BDB stats, streaming stats)
205 * Monitoring and Config improvements for REST service
206 * Removed donor based rebalancing code
207 * Removed replica type from the code base
208 * Improvements to Rebalance controller
210 Release 1.4.7 on 09/04/13
213 * Added the Zone affinity feature on the client side
214 * Refactoring the rest package (Rest server and Coordinator)
215 * Using standard Netty pipeline in the Coordinator
216 * Added monitoring hooks for the Rest Service
218 Release 1.4.6 on 07/24/2013
221 * Cleanup of R2Store to work with the REST server
222 * Modified ZenStoreClient to avoid creating SystemStores during each
224 * Added slop streaming functionality
226 Release 1.4.4 on 07/08/2013
228 Changes made since 1.4.3
229 * Change Repartioner to swap among nodes from all specified zones
230 * Added rebalance shuffle, cluster expansion scripts
232 Release 1.4.3 on 07/03/2013
234 Note: Server changes are not backwards compatible. To use new
235 rebalancing tooling, servers must be upgraded before hand.
238 - Proxying / abortable rebalance
239 - added proxy puts which rebalance safely abortable
240 - improved proxy gets to check locally first and only fetch remote
242 - Repartitioning: improved algorithms to optimize partition layout
243 - Plan: avoids cross-zone data movement in most rebalance cases. Only
244 expanding into a new zone requires data movement across zones.
246 - better / more accurate progress monitoring.
247 - 'proxy pause' before rebalancing. Ensures clients pick up new
248 metadata and allows performance with proxy'ing to be observed
249 before servers start rebalancing.
250 - NOTE: Donor-based rebalance is currently broken by these changes.
252 - PartitionBalanceCLI: more nuanced and verbose analysis of cluster
254 - RepartitionerCLI: Stand alone tool to determine repartitioning
255 - Stand alone 'repartitioning' scripts for key use cases: new cluster,
256 shuffle (existing cluster), cluster expansion, and zone expansion.
257 - RebalancePlanCLI: Stand alone plan tool to determine cost of a
259 - RebalanceControllerCLI: Stand-alone tool for executing a specific
263 - moved to Guava from Google Collections
264 - better unit test coverage of zoned clusters
265 - added example prod configuration
266 - added tools directory
267 - .gitignore improvements
269 - fixed concurrency bugs in metadata store
270 - fixed vector clock comparisons
272 - write slops in more cases when they are needed
273 - KeySampler/Fetcher handle larger working sets with OOME
275 Changes made since 1.4.2
277 * Faster creation of BaseStoreRoutingPlan objects in the fast rebalancing path
279 Release 1.4.2 on 06/27/2013
280 * Fixing costly StoreRoutingPlan object construction
282 Release 1.4.0 on 06/21/2013
283 * Zone expansion release
285 Release 1.3.4 on 06/19/2013
286 * Read Write store bug fixes
287 - Rewrite of InMemoryStorageEngine + config to control multiVersionPuts
288 - Fixed a bug in the read-repair logic which was causing unnecessary puts.
289 - Fixed put operation in PipelineRoutedStore to ensure slop is submitted
290 - Fixed VectorClock bug triggered during versioned puts
291 * Rebalance improvement
292 - Made it safe to abort rebalance
293 - Proxy puts are established so that 'old' partition is updated and so an aborted rebalance can "roll back" without any data loss
294 - Proxy gets & puts are established within the zone making improving performance during rebalance
296 - Added monitoring capability in the Coordinator
298 Release 1.3.3 on 04/24/2013
299 * VoldemortBuildandPush
300 - Fixed bug with schema check
302 - Fixed issue with redundant callback invocation
303 Release 1.3.1 on 03/25/2013
305 - Fixed the bug in calculating checksums when we entere a retry loop
306 - refactored per file checksums
307 - added junit test case to simulate intermittent IO exceptions
308 * voldemort.client.protocol.admin.AdminClient
309 - Added AdminStoreClient so that AdminClient can do store operations
310 against specific store on a specific node.
311 - Added helper methods for diong put & get for specific node/store
312 - Added voldemort.client.protocol.admin.QueryKeyResult type to
313 simplify QueryKey interface
314 * Improved FetchStreamRequestHandler and sub-classes
315 - Renamed all sub-classes: 'FullScan' and 'PartitionScan' prefixes
316 for pertinent stream request handler implementations.
317 - Removed unused skipRecords parameter.
318 - Added recordsPerPartition parameter to fetch limited portion of
320 - All logic about how many keys to fetch (return to invoker) is
323 - Added many options to help improve the balance of (zoned) clusters.
324 - Analysis of the balance of a cluster is significantly more detailed.
325 - Fixed a bug that reduced the balance of a cluster each time it was
327 - Many different algorithms for improving cluster balance are
328 implemented in voldemort.utils.RebalanceClusterUtils
329 * ConsistencyCheck & ConsistencyFixCLI
330 - New tools for ensuring data durability. These tools are necessary
331 because slop creation can fail during put operations.
332 - ConsistencyCheck determines which keys, if any, lack
333 "consistency". I.e., are present on only a subset of the expected
335 - ConsistencyFix takes a list of bad (inconsistent) keys and makes
336 sure they are present on all expected partitions.
337 - ConsistencyFix also has an interface for repairing "orphaned" keys
338 that could result from an aborted rebalance.
339 * KeySamplerCLI & KeyVersionFetcherCLI
340 - KeySamplerCLI is a new tool that reads some number of keys for
341 specified partitions/stores.
342 - KeyVersionFetcherCLI is a new tool that, given a key and a store,
343 fetches the version from all nodes that host a partition that
344 ought to store a replica of the key's value.
345 - Together, KeySamplerCLI and KeyVersionFetcherCLI correctly
346 implement the intended functionality of the Entropy tool (for
347 servers that implement either FullScan and PartitionScan fetches).
348 - Entropy tool had been used in the past to verify a sample of keys
349 before and after a rebalance. Entropy tool does not work as
350 intended/expected. This is exacerbated by the partition aware
351 layouts. Instead of trying to fix the Entropy tool, these two new
352 tools were developed. Entropy is deprecated and will eventually be
353 removed from the code base.
354 * Substantial refactoring of helper & util methods:
355 - voldemort.cluster.Cluster : added helper methods
356 - voldemort.utils.ClusterInstance : wraps up one Cluster &
357 List<StoreDefinition>
358 - voldemort.utils.Cluster : utils for single Cluster object.
359 - voldemort.utils.NodeUtils : utils for Node object.
360 - voldemort.utils.RebalanceUtils : Many methods moved to more
361 appropriate helper classes
362 - voldemort.utils.StoreDefinitionUtils : utils for StoreDefinition
364 - voldemort.utils.StoreInstance : wraps up one Cluster & one
367 - ByteUtils toHexString & from HexString now rely on standard
369 - voldemort.client.AdminFetchTest now tests FullScan and
370 PartitionScan fetches
371 - voldemort.store.routed.ReadRepairerTest annotated all tests with
375 Release 1.3.0 on 03/08/2013
377 NOTE: This is an open source release! This release can be downloaded here:
378 http://github.com/voldemort/voldemort/downloads.
380 Changes made since 1.2.3
381 * VoldemortConfig and ClientConfig now contain detailed documentation
382 * BDB-JE defaults set to ones in prod@linkedin
383 * Bug fixes on kerberos support for Hadoop
386 Release 1.2.3 on 02/20/2013
388 Changes made since 1.2.2
389 * Added a retry loop and synchronized block while getting Hadoop FS
390 * Code cleanup in HdfsFetcher to make it more readable.
391 * Throwing explicit exceptions in HdfsFetcher instead of
392 returning null to be more precise in the Azkaban logs.
395 Release 1.2.2 on 02/19/2013
397 Changes made since 1.2.1
398 * Synchronized the streaming API
399 * Fixed some of the streaming API tests.
402 Release 1.2.1 on 0/30/2013
404 Changes made since 1.2.0
405 * Added a Streaming API and related tests.
406 * Refactoring of the admin client apis into functional inner classes
409 Release 1.2.0 on 01/21/2013
411 Changes made since 1.1.9
412 * Added an Admin API to fetch orphaned key / entries
413 * Improved some tests related to streaming API.
414 * Correcting commons-codec version in ivy file (1.4)
417 Release 1.1.9 on 01/15/2013
419 Changes made since 1.1.8
420 * Asynchronous socket checkout improvements
421 * Changed checkout behavior of KeyedResourcePool to only create new
422 connections when there are no resources available (rather than
423 creating new connections until the pool is full)
424 * Changed QueuedKeyedResourcePool.reset behavior to better match
425 KeyedResourcePool (i.e., to not cancel queued asynchronous
426 requests unnecessarily)
427 * Removed (unnecessary) synchronization primitives from keyed resource pool
428 * Reduce granularity of failure detector locking within ThresholdFailureDetector
429 * Minor features/improvements
430 * Less verbose logging in the face of expected exceptions and errors
431 * Refactored (Queued)KeyedResourcePoolTest
433 * Fixed possible race condition for resource creation in KeyedResourcePool
434 * More efficient (time & space) and simpler Histogram implementation
438 Release 1.1.8 on 01/14/2013
440 Changes made since release 1.1.7
441 * Enhanced Server Monitoring
443 -- Streaming operations to the server
444 -- BDB storage exception counts
445 * Ability to turn off BDB Checkpointing during batch modifications
446 * Added ability to delete old checksum files in Build and Push reducer
447 * Upgrade Hadoop jar to 1.0.4-p2
450 Release 1.1.7 on 01/03/2013
452 NOTE: This release is based off of release 1.1.4
454 Changes made since release 1.1.4
455 * Upgrading Hadoop jar to 1.0.2
456 * Added support for Kerberos authentication in HdfsFetcher
457 * Extra config parameters for Kerberos config and keytab file
460 NOTE: Release 1.1.5 and 1.1.6 are special client side releases
461 not based off of master. 1.1.5 was rolled back to to a weird bug.
462 1.1.6 is a special client side release including Auto-
463 bootstrapper and Versioned Avro support.
466 Release 1.1.4 on 11/29/2012
468 Changes made since release 1.1.3
469 * Added BDB parameters to control LRU behavior in cache & proactive cleaner migration
470 * Added a mlock fix for pinning the indexes of RO stores in memory
473 Release 1.1.3 on 11/28/2012
475 Changes made since release 1.1.2
476 * Fixed a bug in the build and push job, specifically the Mapper
477 that caused collisions
478 * Added retry mechanism with the HDFS fetcher for hftp
481 Release 1.1.2 on 10/31/2012
483 Changes made since release 1.1.1
484 * Reverted a change to voldemort.versioning.Versioned.getVersion() so
485 that a Version is returned as our clients expect.
488 Release 1.1.1 on 10/30/2012
490 Changes made since release 1.1.0
491 * Fixed connection leak in ClientRequestExecutorFactory
492 * Changed client to default to DefaultStoreClient
495 Release 1.1.0 on 10/19/2012
497 Changes made since release 1.0.0
499 IMPORTANT NOTE : This release has significant changes to the BDB storage layer.
500 Users are required to read the bin/PREUPGRADE_FOR_1_1_X_README file
501 thoroughly before attempting to upgrade to 1.1.0. The necessary data
502 conversion will be done through bin/voldemort-convert-bdb.sh
504 * Upgrading to JE 4.1.17
505 * New data format that handles conflicting updates in Voldemort more
507 * Move data off heap and only use it for Index
508 * When scanning, evict whatever you bring in right away.
509 * Partition based scan api to dramatically speed up rebalancing & restore
510 using Partition aware scans (you exactly scan whatever you want to fetch)
511 * Flexible knobs to control scheduling of DataCleanupJob
514 Release 1.0.0 on 10/17/2012
516 NOTE: The large version number jump from 0.96 to 1.0.0 is to
517 standardize on a version number of the sort MAJOR.MINOR.PATCH. This
518 change is part of our effort to treat internal and open source
519 releases in a much more similar manner. Along these lines, release
520 notes for internal releases (like this one) are committed on the
521 master branch. We hope this improves transparency as we work towards
522 the next open source release.
524 Changes made since release 0.96
526 * Auto bootstrapping: ZenStoreClient and System stores
527 * Added server side system stores for managing metadata
528 * ZenStoreClient interacts with system stores
529 * ZenStoreClient auto bootstraps whenever cluster.xml or stores.xml changes
530 * Added a new routing strategy to route to all with local preference
531 * Added a client-registry for publishing client info and config values
532 * Updated LazyClientStore to try to bootstrap during Init
533 * Modified Failure Detector to work on a shared cluster object reference
534 * Avro: schema evolution and read only support
535 * Added new Avro serializer type that supports schema evolution
536 * Added Avro support to read only stores
537 * Added LinkedIn build-and-push Azkaban jobs to build read only stores to contrib
538 * Added a schema backwards compatibility check to VoldemortAdminTool and on server startup to prevent mishaps due to bad schemas
539 * Non-blocking IO: Fixed design flaw that blocked in the face of slow servers
540 * Asynchronous operations no longer do a blocking checkout to get a SocketDestination
541 * Added additional stats collection for better visibility into request queues
543 * Enhanced VoldemortAdminTool to update store metadata version
544 * Enhanced VoldemortAdminTool to work with the new system stores
545 * Added feature to voldemort-shell.sh to dump byte & object arrays
546 * Added a SlowStorageEngine for testing degraded mode performance
547 * Added mechanism to isolate BDB cache usage among stores
548 * Enhanced debug logging (for traffic analysis).
549 * Python client bug fixes (from pull request)
550 * Improved messages in request tracing
551 * Cleaned up help/usage messages within the client shell
552 * Added server config to control socket backlog
553 * Added "--query-keys" option to query multiple keys of multiple stores from specific node
554 * Added control to DataCleanupJob Frequency
555 * Unified jmxid as the factory across the board
557 * bin/generate_cluster_xml.py to generate cluster.xml
558 * bin/repeat-junit.sh and bin/repeat-junit-test.sh to repeatedly run tests
560 * Changed getall return behavior to comply with javadoc
561 * Fixed a bug that caused unnecessary serial requests in getall
562 * HFTP performance issue bug fix (fix in byte buffer and copy process)
563 * Fixed a bug that prevented "--fetch-keys" and "--fetch-entries" in admin tool from showing multiple store results
564 * Fixed problem in sample config that prevented the server from starting
565 * Fixed some intermittent BindException failures across many unit tests
566 * Fixed some intermittent rebalance test failures
567 * Wrapped long running tests with timeouts
570 Release 0.96 on 09/05/2012
572 Changes made since 0.90.1
575 * Append cluster name to various mbeans for better stats display
576 * Implement average throughput in bytes
578 * Add 95th and 99th latency tracking
579 * Add stats for ClientRequestExecutorPool
580 * Add error/exception count and max getall count
581 * BDB+ Data cleanup Monitoring changes
583 * Donor-based rebalancing and post cleanup (see https://github.com/voldemort/voldemort/wiki/Voldemort-Donor-Based-Rebalancing for more details)
584 * Rebalancing integration testing framework (under test/integration/voldemort/rebalance/)
585 * Generate multiple cluster.xml files based on the number specified when running the tool and choose the cluster with the smallest std dev as the final-cluster.xml
586 * Add status output to log for updateEntries (used by rebalancing)
587 * Read-only pipeline:
588 * Add hftp and webhdfs support
589 * Read-only bandwidth dynamic throttler
590 * Add minimum throttle limit per store
591 * Add rollback capability to the Admin tool
592 * Voldemort-backed stack and index linked list impl
593 * Change client requests to not process responses after timeout
594 * Modified client request executor timeout to not factor in the NIO selector timeout
595 * Added BDB native backup capabalities, checksum verification and incremental backups (well tested, but not yet used in production)
596 * Add additional client-side tracing for debugging and consistency analytics
597 * Clean up logging during exception at client-side
598 * Security exception handling
599 * Add snappy to CompressionStrategyFactory
600 * Add configurable option to interrupt service being unscheduled
601 * Add logging support for tracking ScanPermit owners (for debugging purposes)
602 * Add a jmx terminate operation for async jobs
603 * Add zone option for restore from replicas
604 * Changing the enable.nio.connector to true by default
605 * Better disconnection handling for python client
606 * Split junit tests into a long and a short test suites
607 * Add separate timeouts for different operations (put, get, delete, and getAll
608 * Allow getAll to return partial results upon timeout
609 * Improved cluster generation tool
610 * Added log4j properties folder for junit test
612 * httpclient 3.x to httpclient 4.x
613 * Fix NPE in listing read-only store versions
614 * Fixed 2 failure detector bugs during rebalancing or node swapping
615 * Fixed a thread leak issue in StreamingSlopPusher
617 * Fixed a bug in TimeBasedInconsistency resolver.
618 * Fixed race condition in client socket close
619 * Fixed a potential deadlock issue in ScanPermitWrapper
620 * Fixed a bug where a read returns null (on rare occations) when being concurrent with a write
621 * Fixed a performance bug in HdfsFetcher when hftp is used
624 Changes made since 0.90
626 * Updated the documentation for Voldemort shell tool in NOTES
627 * Added Admin API to perform Bdb data cleanup (repairJob)
628 and corresponding unit tests
629 * Fixes in restore from replication, store creation code
630 * Improved failure detector configuration. ThresholdFailureDetector
631 is now the default option
632 * Multiple Fixes to the Voldemort ruby client
633 * Added additional Jmx metrics to expose Bdb environment statistics,
634 caching statistics and Voldemort batch operation statistics
635 * Updated default timeout for restore 'from replica' to 365 days
636 * New feature in the performance tool: '--use-sample' option enables
637 'read, write back unmodified' transactions in place of writes in
638 the workload (allows for testing read-write transactions on stores
639 with complex schemas)
640 * Added the ability to dynamically update cluster.xml and
641 reinitialize the scheduler
643 Release 0.90 on 7/10/2011
645 Changes made since 0.81
646 * All upgrading instructions can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
648 - Single consolidated administrative tool - Got rid of the admin client
649 shell and have a better Voldemort admin tool. More documentation -
650 https://github.com/voldemort/voldemort/wiki/Voldemort-Admin-tool
653 - Basic GUI in JRuby ( and Sinatra ), capable of querying for store metadata.
654 - Read contrib/web-manager/README.md for more details
657 - New better rebalancing support with (a) checkpointing (b) progress bar (c) support for RO store rebalancing
658 - Documentation - https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing
659 - In the process solves Issue 203, 288, 305, 307, 311
661 * Client side changes
662 - Pipeline routed store ( with support for topology awareness ) - Changed the default client side routing
663 to use a state machine like system. More documentation - https://github.com/voldemort/voldemort/wiki/RoutedStore-redesign
664 - Lazy store client - Some of our clients are very inactive and don't really do many requests. For such
665 clients it doesn't make sense to bootstrap the metadata at startup. Hence from now onwards clients instantiated
666 from SocketStoreClientFactory give a LazyStoreClient which will not bootstrap till the first request is made
667 - Caching store client factory - We have also checked-in a new store client factory which we use internally
668 at LinkedIn to cache store clients for stores which we repeatedly.
669 - Failure detector - We have fixed some bugs in the ThresholdFailureDetector. This is an improvement over the naive
670 BannageFailureDetector which would aggressively mark nodes down after a single failure. More details -
671 https://github.com/voldemort/voldemort/wiki/Client-side-failure-detector-implementations
672 - Issue 219: StoreClient::put returns a Version - You may have to upgrade your clients since now the StoreClient returns
673 a version to be used in successive requests.
675 * Read-only store changes
676 - Admin based store swapper - Till 0.81 we relied on a servlet based fetcher / swapper.
677 Now we support an admin based store swapper which reports progress.
678 - Other changes - (a) New directory format (b) Two new data format. Support for iterating over read-only data
679 (c) Checksum of data in .metadata file (d) Some monitoring changes of RO stores
682 - Introduction of JNA 3.2.7 - We have introduced JNA 3.2.7 in this release for supporting symbolic links in read-only stores.
683 More details about the new RO format and changes can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
684 - Protocol Buffers 2.3.0 - Upgrading protocol buffers from 2.2.0 to 2.3.0
685 - Avro 1.4.0 - Upgraded Avro from 1.3.0 to 1.4.0
688 - Added support for Krati 0.3.6 as a storage engine ( http://sna-projects.com/krati/ ) -
689 https://github.com/voldemort/voldemort/tree/release-090/contrib/krati
692 - Hinted handoff - https://github.com/voldemort/voldemort/wiki/Hinted-Handoff
693 - Experimental support for server side transforms - https://github.com/voldemort/voldemort/wiki/Server-side-transforms-in-Voldemort
694 - Topology awareness ( i.e. datacenter / rack ) - https://github.com/voldemort/voldemort/wiki/Topology-awareness-capability
695 - Repair Job ( Job which will delete data if the node is not responsible for it )
698 - Tons of JMX changes ( average bytes, etc ) - More details ( https://github.com/voldemort/voldemort/wiki/JMX-Monitoring )
699 - Key distribution generator - Ability to estimate skew of your cluster ( ./bin/run-class.sh voldemort.utils.KeyDistributionGenerator )
702 - Python - Updated Python client with support for Binary JSON serialized stores ( ./clients/python/README )
703 - Ruby - Updated Ruby client ( ./clients/ruby/README.md )
705 Release 0.81 on 6/15/2010
707 Changes made since 0.80.2:
709 * IMPORTANT: we have upgraded the Hadoop Core jar to v0.20.2. Since
710 this version of Hadoop requires Java 6, in order to retain backwards
711 compatibility with Java 5, you will need to replace this jar with
712 the Hadoop 0.18.* core jar.
713 * Multiple donors for Voldemort Rebalancing: speeds up rebalancing, by
714 allowing a single stealer node to transfer partitions from multiple
716 * Features for EC2 Testing
717 * Read-only Stores: backwards compatibility with 0.70, bug fix in
719 * Hadoop InputFormat, Pig LoadFunc in Contrib: ability to perform
720 Map/Reduce (either directly via Hadoop or using Pig) over data in
721 Voldemort stores. See:
722 < contrib/hadoop/test/voldemort/hadoop/VoldemortWordCount.java > for
724 * Voldemort Performance Tool: performance measurement tool based on
725 YCSB (Yahoo Cloud Serving Benchmark) code. Run
726 < bin/voldemort-performance-tool.sh --help > for more information.
727 * Reverted default failure detector implementation back to
728 BannagedPeriodFailureDetector due to potential bugs in the
729 ThresholdFailureDetector
731 Release 0.80.2 on 4/27/2010
733 Changes made since 0.80.1:
735 * Batched NIO writes (improves performance for AdminClient/Streaming
736 operations when the NIO connector is enabled)
737 * VoldemortAdminTool: Added support to specify stores, support to
738 fetch all keys to a binary file, support to save keys in ASCII
739 (JSON) format [experimental], capability to add stores. Run
740 < bin/voldemort-admin-tool.sh --help > for more information.
741 * Minor bug fixes for Rebalancing
742 * Issue 240: Voldemort fetcher should use different temp directories for
744 * Checksum capability during construction of Voldemort read-only stores
746 Release 0.80.1 on 3/23/2010
748 Changes made since 0.80:
750 * Issue 133: Support for Apache Avro as a serialization format
751 * Issue 223: Changed the default client to use
752 TresholdFailureDetector
753 * Fixed issue 222: Revised KeyedResourcePool.close(K key) to fix
755 * Fixed issue 198: Revised ReadRepairer to use a separate copy of the
756 vector clock, fixing a situation where NoSuchElementException would
758 * Miscellaneous enhancement: support for TCP keep-alives, improved
759 read only store utilities, command line interface to AdminClient,
760 improved load testing tools
762 Release 0.80 on 2/18/2010
764 Changes made since 0.70.1:
766 * IMPORTANT: backwards compatibility between the client and server has
767 changed. A backwards incompatibility in the wire protocol was found
768 between releases 0.60 to 0.70.1 and releases prior to 0.60. We chose
769 to make 0.80 compatible with 0.57.1 and earlier versions, while
770 introducing an incompatibility with 0.60-0.70.1. What this means is
771 that if you're presently running 0.60 and higher, you would need to
772 upgrade the Voldemort jar files on *all* servers and clients.
773 * Upgraded the BDB storage engine to use BerkeleyDB-JE 4.0.92,
774 retaining ability to use BerkeleyDB-JE 3.3.* if desired. IMPORTANT:
775 if one switches from BerkeleyDB-JE 3.3.* to 4.0.92 they will be able
776 to access all of existing data. Once a switch has been made to
777 4.0.92 the data will not be readable by earlier versions of BDB. If
778 there's a chance that a roll back to 3.3.* might be needed, the best
779 course of action will be to make a backup of existing data
781 Switching between 3.3.* and 4.0.* would also require rebuilding the
782 class files (e.g., by running "ant clean && ant release" after
783 replacing the BDB jar files).
784 * Compression support for read-only stores
785 * Increased the socket buffer size for transferring read-only
786 stores from Hadoop for improved performance over high-latency links
787 * NIO support for the Admin Service, including Streaming
789 * Support for adding stores on the fly via the Admin
791 * Fixed issue 209: Incorrect object passed to List.contains in
792 RebalanceUtils.getLatestCluster()
793 * Fixed issue 211: Unnecessary read repairs during getAll() with more
795 * Other enhancements: better CLI for rebalancing, throttling in
796 Admin Service is now based on all disk activity
798 Release 0.70.1 on 2/1/2010
800 Changes made since 0.70:
802 * Fixed issue 205: if no keys passed to getAll() were in partitions
803 undergoing rebalancing, proxyGetAll() would be called with an
804 empty list even if rebalancing wasn't happening
806 Release 0.70 on 1/27/2010
808 Changes made since 0.60.1:
810 * A beta of rebalancing (dynamic cluster expansion) support merged
811 into the main branch. See the project's wiki for more information:
812 http://wiki.github.com/voldemort/voldemort/voldemort-rebalancing
813 * New failure detector merged into the main branch:
814 http://wiki.github.com/voldemort/voldemort/failure-detection
815 * Beta mechanism for restoring all of node's data from replicas on
816 demand. This is an alternative to a more gradual mechanism provided
817 by read-repair: useful when a machine is down for a prolonged period
818 and is then re-inserted into the cluster.
819 Invoked via JMX: the operation is restoreDataFromReplication in the
820 voldemort.server.VoldemortServer MBean, with a mandatory parameter
821 (integer >= 1) indicating the number of transfers to do in parallel.
822 * Simple gossip protocol (for cluster metadata) merged
823 into the main branch. Disabled by default: use "enable.gossip=true"
824 to enable, use "gossip.interval.ms" to set an interval at which gossip
825 occurs (default: 30000 i.e., 30 seconds).
826 * Fixed issue 190: add a way of aggregating performance data over
828 * Fixed issue 181: stack trackes shouldn't be filled for Obsolete
831 Release 0.60.1 on 12/18/2009
833 Minor changes made since 0.60:
835 * Better logging in the exception thrown if config/.temp and
836 config/.version are copied
837 * Bumping up the version to 0.60.1 in order to release updated
838 archives, fixing an error in the stores.xml for single_node_cluster
841 Release 0.60 on 12/15/2009
843 Changes made since 0.57.1:
845 * Admin Client/Server API: adds support for streaming-based transfer
846 of entries between nodes, deleting entries on remote nodes,
847 remotely deleting and updating metadata
848 * EC2 testing: a way to periodically run integration and performance
849 tests which involve Voldemort instances on different machines
850 * Experimental support for views
851 * Interpolation search for read-only stores
852 * Support for large lists and strings in the JSON serializer
853 * LZF compression support
854 * Ruby client contributed
855 * Fixed issue 170: hanging if a port is used by another process
856 * Fixed issue 122: suspicious integer division in
857 RequestCounter.getThroughput
858 * Miscellaneous improvements and bug fixes for read-only stores
859 * Fixed issue 168: added StorageEngine.keys()
861 Release 0.57.1 on 11/27/2009
863 Minor change made since 0.57:
865 * Modified build.xml to exclude .git directory from release tarballs/zipfiles
867 Release 0.57 on 11/16/2009
869 The following changes were made since 0.56:
871 * Fixed an issue in ReadOnlyEngine's close() method
872 * Fixed hidden logging in StorageService
873 * Fix for issue 163 (lock mode during get)
874 * Exposed bdb environment stats with setFast(false)
876 Release 0.56 on 10/26/2009
878 The following changes were made since 0.55:
880 * Fix for issue 164: Changed default bdb.max.logfile.size to 60MB
881 * Make file deletes asynchronous in read only store swap
882 * Added better debug logs for bdb stats
883 * Fixed race condition in AbstractSocketPoolTest
884 * Added improved monitoring for bdb stats
885 * Added backoff and retry logic in bootstrap code
886 * Not logging obosleteVersionException(s) or counting it in JMX exception count
888 * C++ client building on OS X
889 * Rely only on the number of versions returned to decide whether to retrieve
890 the value for put(K,V)
891 * Implemented issue 152: getVersion() API in Store
893 Release 0.55 on 10/7/2009
895 The following changes were made since 0.52 (in summary):
897 * Add an event throttler
898 * Added DataCleanupJob
899 * Protocol buffers at version 2.2.0
900 * BDB JE upgraded to 3.3.87
901 * Added data compression support (enable by adding
905 to value-serializer or key-serializer sections in stores.xml)
906 * Added a resource pool/socket pool implementation (no longer using
908 * Added server-side NIO support (enabled by setting
909 enable.nio.connector=true
910 in server.properties)
911 * Improved the efficiency of the protocol buffers network protocol by using
912 CodedOutputStream/CodedInputStream (zero copy transfers)
913 * Fix for issue #21: incorrectness in vector clock inconsistency resolver
914 * Upgrade google-collections
915 * Support for building read only stores in Hadoop