1 Release 1.8.3 on 05/27/2014
3 * Add debug log messages for CoordinatorService
4 * Fix NetworkClassLoader bug when running in Windows OS (Carlos Tasada)
5 * Add junit support for gradle
6 * Fix bind failures while running tests
7 * Add overwrite option to fork lift tool
8 * Use standard logger in the StreamingClient constructor
9 * [python client] Fixed reconnect in _execute_request (Søren Holbech)
10 * Fix the admin metadata check command and did same change for old admin command.
12 Release 1.8.2 on 05/09/2014
14 * Bug fix in MetadataStore regarding updation of routing strategy
16 Release 1.8.1 on 05/01/2014
18 * Fix AbstractStoreClientFactory for backwards compatibility
19 * Cleaned up logging messages
20 * Unit tests for big Binary data
21 * Refactored voldemort admin tool - Phase 1 and Phase 2
23 Release 1.8.0 on 04/21/2014
25 * Split the stores.xml into individual stores on Voldemort server
26 * Maintaing backwards compatibility of MetadataStore
27 * DefaultStoreClient bootstraps using store name (instead of stores.xml)
28 * New Admin tool functionality to update stores (instead of replace)
30 Release 1.7.3 on 04/16/2014
32 * Fix RESTClientFactory to return a LazyStoreClient
34 Release 1.7.2 on 04/15/2014
36 * Quota limiting related fix
37 * Coordinator related fix
39 Release 1.7.1 on 04/10/2014
43 Release 1.7.0 on 04/09/2014
45 * Improves BDB coversion process and documentations
46 * Adds scala build support and shell (Saurabh Bhatia)
47 * Add removeStorageEngine call to StorageConfiguration
49 Release 1.6.9 on 03/19/2014
51 * Fix a big where StreamingClient will throw NPE if not initialized
52 * Fix multiple broken tests
53 * Modified RO Rebalance Test to cover one more case
54 * Adding thin client shell for Voldemort Coordinator
57 Release 1.6.8 on 02/21/2014
59 * Add a tool to dump BDB data to Text and the reverse
60 * Fix racing condition problem in CachingStoreClientFactory
61 * Testing, client shell and admin tool enhancements from suletm,
62 ahimta, bhsaurabh and Holden Caufield
63 * ConsistencyCheckTool enhancement
64 * Fix bug in Coordinator for enabling different startup ports
66 Release 1.6.7 on 02/07/2014
68 * Updating the ivy file to refer to tusk-0.0.2 (build fix for nuage)
70 Release 1.6.6 on 02/07/2014
72 * Quota management in Voldemort
73 * Creation of read only stores in RO2 format
74 * Updation of query key feature in admin tool
76 Release 1.6.5 on 02/04/2014
78 * Adding AdminClient functionality to update store definitions
80 Release 1.6.4 on 02/03/2014
82 * Fix bug when RO server NPE when rolled back
83 * RO bug fix caused by RW optimization code
85 Release 1.6.3 on 01/24/2014
87 * Fix KeyVersionFetcher to not depend on nodeid 0
89 Release 1.6.2 on 01/10/2014
91 * Slops stored for nodes no longer in cluster, will die automatically
92 * Admin command to selectively purge slops destined to nodes/zones/stores
93 * Admin command to atomically update cluster.xml, stores.xml in one go
94 * Added admin command show-routing-plan to trace routing table for a key
95 * ZoneClipperCLI also adjusts store definitions based on dropped zone
96 * Numerous tests around non contiguous node ids/zone ids
98 Release 1.6.1 on 12/13/2013
100 * Avro support for fetch keys/entries admin command
101 * Store Clients are now cached by default, within the same factory
102 * Rigoruous tests around non contiguous node ids/zone ids
103 * ZenStoreClient efficient resource managerment
106 Release 1.6.0 on 11/26/2013
108 NOTE: This is an open source release! This release can be downloaded here:
109 https://github.com/voldemort/voldemort/releases
111 Changes made since 1.5.9:
113 * Fixes to remove dependency on non contiguous node ids
114 * Stats for store client
117 Release 1.5.9 on 11/05/2013
119 * Bug fix in AsyncMetadataVersionManager (related to store version tracking)
121 Release 1.5.8 on 10/29/2013
123 * Replace resthdfs jar with tusk.jar
124 * Incorporate retries into rest hdfs fetcher
126 Release 1.5.7 on 10/28/2013
128 * Voldemort now uses BDB 5.0.88
129 * Support for RO fetches via RestHdfs
130 * Fix NPE in Streaming Client
131 * Retrying fetches on all exceptions w/ delay
133 Release 1.5.6 on 10/18/2013
135 * Removed getStoreClientFactoryStats from StoreClientFactory interface (fix)
137 Release 1.5.5 on 10/18/2013
139 * Client shell with avro support
140 * Fix bind exceptions
141 * Fix ObsoleteVersion Exceptions
142 * Build once and push to multiple clusters
143 * Improvements to failure detector
145 * Prevent NPE in worker thread when shutdown
146 * Bug fixes in repair and prune job
148 Release 1.5.3 on 09/19/2013
150 * Bug fixes (NPE during shutdown
151 * Client side visibility changes (Histogram, Connect exception)
152 * Adding PruneJob and timebased resolving mechanism to streaming client
153 * R2Store cleanup required for the rest client
154 * Monitoring improvements (aggregated BDB stats, streaming stats)
155 * Monitoring and Config improvements for REST service
156 * Removed donor based rebalancing code
157 * Removed replica type from the code base
158 * Improvements to Rebalance controller
160 Release 1.4.7 on 09/04/13
163 * Added the Zone affinity feature on the client side
164 * Refactoring the rest package (Rest server and Coordinator)
165 * Using standard Netty pipeline in the Coordinator
166 * Added monitoring hooks for the Rest Service
168 Release 1.4.6 on 07/24/2013
171 * Cleanup of R2Store to work with the REST server
172 * Modified ZenStoreClient to avoid creating SystemStores during each
174 * Added slop streaming functionality
176 Release 1.4.4 on 07/08/2013
178 Changes made since 1.4.3
179 * Change Repartioner to swap among nodes from all specified zones
180 * Added rebalance shuffle, cluster expansion scripts
182 Release 1.4.3 on 07/03/2013
184 Note: Server changes are not backwards compatible. To use new
185 rebalancing tooling, servers must be upgraded before hand.
188 - Proxying / abortable rebalance
189 - added proxy puts which rebalance safely abortable
190 - improved proxy gets to check locally first and only fetch remote
192 - Repartitioning: improved algorithms to optimize partition layout
193 - Plan: avoids cross-zone data movement in most rebalance cases. Only
194 expanding into a new zone requires data movement across zones.
196 - better / more accurate progress monitoring.
197 - 'proxy pause' before rebalancing. Ensures clients pick up new
198 metadata and allows performance with proxy'ing to be observed
199 before servers start rebalancing.
200 - NOTE: Donor-based rebalance is currently broken by these changes.
202 - PartitionBalanceCLI: more nuanced and verbose analysis of cluster
204 - RepartitionerCLI: Stand alone tool to determine repartitioning
205 - Stand alone 'repartitioning' scripts for key use cases: new cluster,
206 shuffle (existing cluster), cluster expansion, and zone expansion.
207 - RebalancePlanCLI: Stand alone plan tool to determine cost of a
209 - RebalanceControllerCLI: Stand-alone tool for executing a specific
213 - moved to Guava from Google Collections
214 - better unit test coverage of zoned clusters
215 - added example prod configuration
216 - added tools directory
217 - .gitignore improvements
219 - fixed concurrency bugs in metadata store
220 - fixed vector clock comparisons
222 - write slops in more cases when they are needed
223 - KeySampler/Fetcher handle larger working sets with OOME
225 Changes made since 1.4.2
227 * Faster creation of BaseStoreRoutingPlan objects in the fast rebalancing path
229 Release 1.4.2 on 06/27/2013
230 * Fixing costly StoreRoutingPlan object construction
232 Release 1.4.0 on 06/21/2013
233 * Zone expansion release
235 Release 1.3.4 on 06/19/2013
236 * Read Write store bug fixes
237 - Rewrite of InMemoryStorageEngine + config to control multiVersionPuts
238 - Fixed a bug in the read-repair logic which was causing unnecessary puts.
239 - Fixed put operation in PipelineRoutedStore to ensure slop is submitted
240 - Fixed VectorClock bug triggered during versioned puts
241 * Rebalance improvement
242 - Made it safe to abort rebalance
243 - Proxy puts are established so that 'old' partition is updated and so an aborted rebalance can "roll back" without any data loss
244 - Proxy gets & puts are established within the zone making improving performance during rebalance
246 - Added monitoring capability in the Coordinator
248 Release 1.3.3 on 04/24/2013
249 * VoldemortBuildandPush
250 - Fixed bug with schema check
252 - Fixed issue with redundant callback invocation
253 Release 1.3.1 on 03/25/2013
255 - Fixed the bug in calculating checksums when we entere a retry loop
256 - refactored per file checksums
257 - added junit test case to simulate intermittent IO exceptions
258 * voldemort.client.protocol.admin.AdminClient
259 - Added AdminStoreClient so that AdminClient can do store operations
260 against specific store on a specific node.
261 - Added helper methods for diong put & get for specific node/store
262 - Added voldemort.client.protocol.admin.QueryKeyResult type to
263 simplify QueryKey interface
264 * Improved FetchStreamRequestHandler and sub-classes
265 - Renamed all sub-classes: 'FullScan' and 'PartitionScan' prefixes
266 for pertinent stream request handler implementations.
267 - Removed unused skipRecords parameter.
268 - Added recordsPerPartition parameter to fetch limited portion of
270 - All logic about how many keys to fetch (return to invoker) is
273 - Added many options to help improve the balance of (zoned) clusters.
274 - Analysis of the balance of a cluster is significantly more detailed.
275 - Fixed a bug that reduced the balance of a cluster each time it was
277 - Many different algorithms for improving cluster balance are
278 implemented in voldemort.utils.RebalanceClusterUtils
279 * ConsistencyCheck & ConsistencyFixCLI
280 - New tools for ensuring data durability. These tools are necessary
281 because slop creation can fail during put operations.
282 - ConsistencyCheck determines which keys, if any, lack
283 "consistency". I.e., are present on only a subset of the expected
285 - ConsistencyFix takes a list of bad (inconsistent) keys and makes
286 sure they are present on all expected partitions.
287 - ConsistencyFix also has an interface for repairing "orphaned" keys
288 that could result from an aborted rebalance.
289 * KeySamplerCLI & KeyVersionFetcherCLI
290 - KeySamplerCLI is a new tool that reads some number of keys for
291 specified partitions/stores.
292 - KeyVersionFetcherCLI is a new tool that, given a key and a store,
293 fetches the version from all nodes that host a partition that
294 ought to store a replica of the key's value.
295 - Together, KeySamplerCLI and KeyVersionFetcherCLI correctly
296 implement the intended functionality of the Entropy tool (for
297 servers that implement either FullScan and PartitionScan fetches).
298 - Entropy tool had been used in the past to verify a sample of keys
299 before and after a rebalance. Entropy tool does not work as
300 intended/expected. This is exacerbated by the partition aware
301 layouts. Instead of trying to fix the Entropy tool, these two new
302 tools were developed. Entropy is deprecated and will eventually be
303 removed from the code base.
304 * Substantial refactoring of helper & util methods:
305 - voldemort.cluster.Cluster : added helper methods
306 - voldemort.utils.ClusterInstance : wraps up one Cluster &
307 List<StoreDefinition>
308 - voldemort.utils.Cluster : utils for single Cluster object.
309 - voldemort.utils.NodeUtils : utils for Node object.
310 - voldemort.utils.RebalanceUtils : Many methods moved to more
311 appropriate helper classes
312 - voldemort.utils.StoreDefinitionUtils : utils for StoreDefinition
314 - voldemort.utils.StoreInstance : wraps up one Cluster & one
317 - ByteUtils toHexString & from HexString now rely on standard
319 - voldemort.client.AdminFetchTest now tests FullScan and
320 PartitionScan fetches
321 - voldemort.store.routed.ReadRepairerTest annotated all tests with
325 Release 1.3.0 on 03/08/2013
327 NOTE: This is an open source release! This release can be downloaded here:
328 http://github.com/voldemort/voldemort/downloads.
330 Changes made since 1.2.3
331 * VoldemortConfig and ClientConfig now contain detailed documentation
332 * BDB-JE defaults set to ones in prod@linkedin
333 * Bug fixes on kerberos support for Hadoop
336 Release 1.2.3 on 02/20/2013
338 Changes made since 1.2.2
339 * Added a retry loop and synchronized block while getting Hadoop FS
340 * Code cleanup in HdfsFetcher to make it more readable.
341 * Throwing explicit exceptions in HdfsFetcher instead of
342 returning null to be more precise in the Azkaban logs.
345 Release 1.2.2 on 02/19/2013
347 Changes made since 1.2.1
348 * Synchronized the streaming API
349 * Fixed some of the streaming API tests.
352 Release 1.2.1 on 0/30/2013
354 Changes made since 1.2.0
355 * Added a Streaming API and related tests.
356 * Refactoring of the admin client apis into functional inner classes
359 Release 1.2.0 on 01/21/2013
361 Changes made since 1.1.9
362 * Added an Admin API to fetch orphaned key / entries
363 * Improved some tests related to streaming API.
364 * Correcting commons-codec version in ivy file (1.4)
367 Release 1.1.9 on 01/15/2013
369 Changes made since 1.1.8
370 * Asynchronous socket checkout improvements
371 * Changed checkout behavior of KeyedResourcePool to only create new
372 connections when there are no resources available (rather than
373 creating new connections until the pool is full)
374 * Changed QueuedKeyedResourcePool.reset behavior to better match
375 KeyedResourcePool (i.e., to not cancel queued asynchronous
376 requests unnecessarily)
377 * Removed (unnecessary) synchronization primitives from keyed resource pool
378 * Reduce granularity of failure detector locking within ThresholdFailureDetector
379 * Minor features/improvements
380 * Less verbose logging in the face of expected exceptions and errors
381 * Refactored (Queued)KeyedResourcePoolTest
383 * Fixed possible race condition for resource creation in KeyedResourcePool
384 * More efficient (time & space) and simpler Histogram implementation
388 Release 1.1.8 on 01/14/2013
390 Changes made since release 1.1.7
391 * Enhanced Server Monitoring
393 -- Streaming operations to the server
394 -- BDB storage exception counts
395 * Ability to turn off BDB Checkpointing during batch modifications
396 * Added ability to delete old checksum files in Build and Push reducer
397 * Upgrade Hadoop jar to 1.0.4-p2
400 Release 1.1.7 on 01/03/2013
402 NOTE: This release is based off of release 1.1.4
404 Changes made since release 1.1.4
405 * Upgrading Hadoop jar to 1.0.2
406 * Added support for Kerberos authentication in HdfsFetcher
407 * Extra config parameters for Kerberos config and keytab file
410 NOTE: Release 1.1.5 and 1.1.6 are special client side releases
411 not based off of master. 1.1.5 was rolled back to to a weird bug.
412 1.1.6 is a special client side release including Auto-
413 bootstrapper and Versioned Avro support.
416 Release 1.1.4 on 11/29/2012
418 Changes made since release 1.1.3
419 * Added BDB parameters to control LRU behavior in cache & proactive cleaner migration
420 * Added a mlock fix for pinning the indexes of RO stores in memory
423 Release 1.1.3 on 11/28/2012
425 Changes made since release 1.1.2
426 * Fixed a bug in the build and push job, specifically the Mapper
427 that caused collisions
428 * Added retry mechanism with the HDFS fetcher for hftp
431 Release 1.1.2 on 10/31/2012
433 Changes made since release 1.1.1
434 * Reverted a change to voldemort.versioning.Versioned.getVersion() so
435 that a Version is returned as our clients expect.
438 Release 1.1.1 on 10/30/2012
440 Changes made since release 1.1.0
441 * Fixed connection leak in ClientRequestExecutorFactory
442 * Changed client to default to DefaultStoreClient
445 Release 1.1.0 on 10/19/2012
447 Changes made since release 1.0.0
449 IMPORTANT NOTE : This release has significant changes to the BDB storage layer.
450 Users are required to read the bin/PREUPGRADE_FOR_1_1_X_README file
451 thoroughly before attempting to upgrade to 1.1.0. The necessary data
452 conversion will be done through bin/voldemort-convert-bdb.sh
454 * Upgrading to JE 4.1.17
455 * New data format that handles conflicting updates in Voldemort more
457 * Move data off heap and only use it for Index
458 * When scanning, evict whatever you bring in right away.
459 * Partition based scan api to dramatically speed up rebalancing & restore
460 using Partition aware scans (you exactly scan whatever you want to fetch)
461 * Flexible knobs to control scheduling of DataCleanupJob
464 Release 1.0.0 on 10/17/2012
466 NOTE: The large version number jump from 0.96 to 1.0.0 is to
467 standardize on a version number of the sort MAJOR.MINOR.PATCH. This
468 change is part of our effort to treat internal and open source
469 releases in a much more similar manner. Along these lines, release
470 notes for internal releases (like this one) are committed on the
471 master branch. We hope this improves transparency as we work towards
472 the next open source release.
474 Changes made since release 0.96
476 * Auto bootstrapping: ZenStoreClient and System stores
477 * Added server side system stores for managing metadata
478 * ZenStoreClient interacts with system stores
479 * ZenStoreClient auto bootstraps whenever cluster.xml or stores.xml changes
480 * Added a new routing strategy to route to all with local preference
481 * Added a client-registry for publishing client info and config values
482 * Updated LazyClientStore to try to bootstrap during Init
483 * Modified Failure Detector to work on a shared cluster object reference
484 * Avro: schema evolution and read only support
485 * Added new Avro serializer type that supports schema evolution
486 * Added Avro support to read only stores
487 * Added LinkedIn build-and-push Azkaban jobs to build read only stores to contrib
488 * Added a schema backwards compatibility check to VoldemortAdminTool and on server startup to prevent mishaps due to bad schemas
489 * Non-blocking IO: Fixed design flaw that blocked in the face of slow servers
490 * Asynchronous operations no longer do a blocking checkout to get a SocketDestination
491 * Added additional stats collection for better visibility into request queues
493 * Enhanced VoldemortAdminTool to update store metadata version
494 * Enhanced VoldemortAdminTool to work with the new system stores
495 * Added feature to voldemort-shell.sh to dump byte & object arrays
496 * Added a SlowStorageEngine for testing degraded mode performance
497 * Added mechanism to isolate BDB cache usage among stores
498 * Enhanced debug logging (for traffic analysis).
499 * Python client bug fixes (from pull request)
500 * Improved messages in request tracing
501 * Cleaned up help/usage messages within the client shell
502 * Added server config to control socket backlog
503 * Added "--query-keys" option to query multiple keys of multiple stores from specific node
504 * Added control to DataCleanupJob Frequency
505 * Unified jmxid as the factory across the board
507 * bin/generate_cluster_xml.py to generate cluster.xml
508 * bin/repeat-junit.sh and bin/repeat-junit-test.sh to repeatedly run tests
510 * Changed getall return behavior to comply with javadoc
511 * Fixed a bug that caused unnecessary serial requests in getall
512 * HFTP performance issue bug fix (fix in byte buffer and copy process)
513 * Fixed a bug that prevented "--fetch-keys" and "--fetch-entries" in admin tool from showing multiple store results
514 * Fixed problem in sample config that prevented the server from starting
515 * Fixed some intermittent BindException failures across many unit tests
516 * Fixed some intermittent rebalance test failures
517 * Wrapped long running tests with timeouts
520 Release 0.96 on 09/05/2012
522 Changes made since 0.90.1
525 * Append cluster name to various mbeans for better stats display
526 * Implement average throughput in bytes
528 * Add 95th and 99th latency tracking
529 * Add stats for ClientRequestExecutorPool
530 * Add error/exception count and max getall count
531 * BDB+ Data cleanup Monitoring changes
533 * Donor-based rebalancing and post cleanup (see https://github.com/voldemort/voldemort/wiki/Voldemort-Donor-Based-Rebalancing for more details)
534 * Rebalancing integration testing framework (under test/integration/voldemort/rebalance/)
535 * Generate multiple cluster.xml files based on the number specified when running the tool and choose the cluster with the smallest std dev as the final-cluster.xml
536 * Add status output to log for updateEntries (used by rebalancing)
537 * Read-only pipeline:
538 * Add hftp and webhdfs support
539 * Read-only bandwidth dynamic throttler
540 * Add minimum throttle limit per store
541 * Add rollback capability to the Admin tool
542 * Voldemort-backed stack and index linked list impl
543 * Change client requests to not process responses after timeout
544 * Modified client request executor timeout to not factor in the NIO selector timeout
545 * Added BDB native backup capabalities, checksum verification and incremental backups (well tested, but not yet used in production)
546 * Add additional client-side tracing for debugging and consistency analytics
547 * Clean up logging during exception at client-side
548 * Security exception handling
549 * Add snappy to CompressionStrategyFactory
550 * Add configurable option to interrupt service being unscheduled
551 * Add logging support for tracking ScanPermit owners (for debugging purposes)
552 * Add a jmx terminate operation for async jobs
553 * Add zone option for restore from replicas
554 * Changing the enable.nio.connector to true by default
555 * Better disconnection handling for python client
556 * Split junit tests into a long and a short test suites
557 * Add separate timeouts for different operations (put, get, delete, and getAll
558 * Allow getAll to return partial results upon timeout
559 * Improved cluster generation tool
560 * Added log4j properties folder for junit test
562 * httpclient 3.x to httpclient 4.x
563 * Fix NPE in listing read-only store versions
564 * Fixed 2 failure detector bugs during rebalancing or node swapping
565 * Fixed a thread leak issue in StreamingSlopPusher
567 * Fixed a bug in TimeBasedInconsistency resolver.
568 * Fixed race condition in client socket close
569 * Fixed a potential deadlock issue in ScanPermitWrapper
570 * Fixed a bug where a read returns null (on rare occations) when being concurrent with a write
571 * Fixed a performance bug in HdfsFetcher when hftp is used
574 Changes made since 0.90
576 * Updated the documentation for Voldemort shell tool in NOTES
577 * Added Admin API to perform Bdb data cleanup (repairJob)
578 and corresponding unit tests
579 * Fixes in restore from replication, store creation code
580 * Improved failure detector configuration. ThresholdFailureDetector
581 is now the default option
582 * Multiple Fixes to the Voldemort ruby client
583 * Added additional Jmx metrics to expose Bdb environment statistics,
584 caching statistics and Voldemort batch operation statistics
585 * Updated default timeout for restore 'from replica' to 365 days
586 * New feature in the performance tool: '--use-sample' option enables
587 'read, write back unmodified' transactions in place of writes in
588 the workload (allows for testing read-write transactions on stores
589 with complex schemas)
590 * Added the ability to dynamically update cluster.xml and
591 reinitialize the scheduler
593 Release 0.90 on 7/10/2011
595 Changes made since 0.81
596 * All upgrading instructions can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
598 - Single consolidated administrative tool - Got rid of the admin client
599 shell and have a better Voldemort admin tool. More documentation -
600 https://github.com/voldemort/voldemort/wiki/Voldemort-Admin-tool
603 - Basic GUI in JRuby ( and Sinatra ), capable of querying for store metadata.
604 - Read contrib/web-manager/README.md for more details
607 - New better rebalancing support with (a) checkpointing (b) progress bar (c) support for RO store rebalancing
608 - Documentation - https://github.com/voldemort/voldemort/wiki/Voldemort-Rebalancing
609 - In the process solves Issue 203, 288, 305, 307, 311
611 * Client side changes
612 - Pipeline routed store ( with support for topology awareness ) - Changed the default client side routing
613 to use a state machine like system. More documentation - https://github.com/voldemort/voldemort/wiki/RoutedStore-redesign
614 - Lazy store client - Some of our clients are very inactive and don't really do many requests. For such
615 clients it doesn't make sense to bootstrap the metadata at startup. Hence from now onwards clients instantiated
616 from SocketStoreClientFactory give a LazyStoreClient which will not bootstrap till the first request is made
617 - Caching store client factory - We have also checked-in a new store client factory which we use internally
618 at LinkedIn to cache store clients for stores which we repeatedly.
619 - Failure detector - We have fixed some bugs in the ThresholdFailureDetector. This is an improvement over the naive
620 BannageFailureDetector which would aggressively mark nodes down after a single failure. More details -
621 https://github.com/voldemort/voldemort/wiki/Client-side-failure-detector-implementations
622 - Issue 219: StoreClient::put returns a Version - You may have to upgrade your clients since now the StoreClient returns
623 a version to be used in successive requests.
625 * Read-only store changes
626 - Admin based store swapper - Till 0.81 we relied on a servlet based fetcher / swapper.
627 Now we support an admin based store swapper which reports progress.
628 - Other changes - (a) New directory format (b) Two new data format. Support for iterating over read-only data
629 (c) Checksum of data in .metadata file (d) Some monitoring changes of RO stores
632 - Introduction of JNA 3.2.7 - We have introduced JNA 3.2.7 in this release for supporting symbolic links in read-only stores.
633 More details about the new RO format and changes can be found here - https://github.com/voldemort/voldemort/wiki/Upgrading-from-0.81
634 - Protocol Buffers 2.3.0 - Upgrading protocol buffers from 2.2.0 to 2.3.0
635 - Avro 1.4.0 - Upgraded Avro from 1.3.0 to 1.4.0
638 - Added support for Krati 0.3.6 as a storage engine ( http://sna-projects.com/krati/ ) -
639 https://github.com/voldemort/voldemort/tree/release-090/contrib/krati
642 - Hinted handoff - https://github.com/voldemort/voldemort/wiki/Hinted-Handoff
643 - Experimental support for server side transforms - https://github.com/voldemort/voldemort/wiki/Server-side-transforms-in-Voldemort
644 - Topology awareness ( i.e. datacenter / rack ) - https://github.com/voldemort/voldemort/wiki/Topology-awareness-capability
645 - Repair Job ( Job which will delete data if the node is not responsible for it )
648 - Tons of JMX changes ( average bytes, etc ) - More details ( https://github.com/voldemort/voldemort/wiki/JMX-Monitoring )
649 - Key distribution generator - Ability to estimate skew of your cluster ( ./bin/run-class.sh voldemort.utils.KeyDistributionGenerator )
652 - Python - Updated Python client with support for Binary JSON serialized stores ( ./clients/python/README )
653 - Ruby - Updated Ruby client ( ./clients/ruby/README.md )
655 Release 0.81 on 6/15/2010
657 Changes made since 0.80.2:
659 * IMPORTANT: we have upgraded the Hadoop Core jar to v0.20.2. Since
660 this version of Hadoop requires Java 6, in order to retain backwards
661 compatibility with Java 5, you will need to replace this jar with
662 the Hadoop 0.18.* core jar.
663 * Multiple donors for Voldemort Rebalancing: speeds up rebalancing, by
664 allowing a single stealer node to transfer partitions from multiple
666 * Features for EC2 Testing
667 * Read-only Stores: backwards compatibility with 0.70, bug fix in
669 * Hadoop InputFormat, Pig LoadFunc in Contrib: ability to perform
670 Map/Reduce (either directly via Hadoop or using Pig) over data in
671 Voldemort stores. See:
672 < contrib/hadoop/test/voldemort/hadoop/VoldemortWordCount.java > for
674 * Voldemort Performance Tool: performance measurement tool based on
675 YCSB (Yahoo Cloud Serving Benchmark) code. Run
676 < bin/voldemort-performance-tool.sh --help > for more information.
677 * Reverted default failure detector implementation back to
678 BannagedPeriodFailureDetector due to potential bugs in the
679 ThresholdFailureDetector
681 Release 0.80.2 on 4/27/2010
683 Changes made since 0.80.1:
685 * Batched NIO writes (improves performance for AdminClient/Streaming
686 operations when the NIO connector is enabled)
687 * VoldemortAdminTool: Added support to specify stores, support to
688 fetch all keys to a binary file, support to save keys in ASCII
689 (JSON) format [experimental], capability to add stores. Run
690 < bin/voldemort-admin-tool.sh --help > for more information.
691 * Minor bug fixes for Rebalancing
692 * Issue 240: Voldemort fetcher should use different temp directories for
694 * Checksum capability during construction of Voldemort read-only stores
696 Release 0.80.1 on 3/23/2010
698 Changes made since 0.80:
700 * Issue 133: Support for Apache Avro as a serialization format
701 * Issue 223: Changed the default client to use
702 TresholdFailureDetector
703 * Fixed issue 222: Revised KeyedResourcePool.close(K key) to fix
705 * Fixed issue 198: Revised ReadRepairer to use a separate copy of the
706 vector clock, fixing a situation where NoSuchElementException would
708 * Miscellaneous enhancement: support for TCP keep-alives, improved
709 read only store utilities, command line interface to AdminClient,
710 improved load testing tools
712 Release 0.80 on 2/18/2010
714 Changes made since 0.70.1:
716 * IMPORTANT: backwards compatibility between the client and server has
717 changed. A backwards incompatibility in the wire protocol was found
718 between releases 0.60 to 0.70.1 and releases prior to 0.60. We chose
719 to make 0.80 compatible with 0.57.1 and earlier versions, while
720 introducing an incompatibility with 0.60-0.70.1. What this means is
721 that if you're presently running 0.60 and higher, you would need to
722 upgrade the Voldemort jar files on *all* servers and clients.
723 * Upgraded the BDB storage engine to use BerkeleyDB-JE 4.0.92,
724 retaining ability to use BerkeleyDB-JE 3.3.* if desired. IMPORTANT:
725 if one switches from BerkeleyDB-JE 3.3.* to 4.0.92 they will be able
726 to access all of existing data. Once a switch has been made to
727 4.0.92 the data will not be readable by earlier versions of BDB. If
728 there's a chance that a roll back to 3.3.* might be needed, the best
729 course of action will be to make a backup of existing data
731 Switching between 3.3.* and 4.0.* would also require rebuilding the
732 class files (e.g., by running "ant clean && ant release" after
733 replacing the BDB jar files).
734 * Compression support for read-only stores
735 * Increased the socket buffer size for transferring read-only
736 stores from Hadoop for improved performance over high-latency links
737 * NIO support for the Admin Service, including Streaming
739 * Support for adding stores on the fly via the Admin
741 * Fixed issue 209: Incorrect object passed to List.contains in
742 RebalanceUtils.getLatestCluster()
743 * Fixed issue 211: Unnecessary read repairs during getAll() with more
745 * Other enhancements: better CLI for rebalancing, throttling in
746 Admin Service is now based on all disk activity
748 Release 0.70.1 on 2/1/2010
750 Changes made since 0.70:
752 * Fixed issue 205: if no keys passed to getAll() were in partitions
753 undergoing rebalancing, proxyGetAll() would be called with an
754 empty list even if rebalancing wasn't happening
756 Release 0.70 on 1/27/2010
758 Changes made since 0.60.1:
760 * A beta of rebalancing (dynamic cluster expansion) support merged
761 into the main branch. See the project's wiki for more information:
762 http://wiki.github.com/voldemort/voldemort/voldemort-rebalancing
763 * New failure detector merged into the main branch:
764 http://wiki.github.com/voldemort/voldemort/failure-detection
765 * Beta mechanism for restoring all of node's data from replicas on
766 demand. This is an alternative to a more gradual mechanism provided
767 by read-repair: useful when a machine is down for a prolonged period
768 and is then re-inserted into the cluster.
769 Invoked via JMX: the operation is restoreDataFromReplication in the
770 voldemort.server.VoldemortServer MBean, with a mandatory parameter
771 (integer >= 1) indicating the number of transfers to do in parallel.
772 * Simple gossip protocol (for cluster metadata) merged
773 into the main branch. Disabled by default: use "enable.gossip=true"
774 to enable, use "gossip.interval.ms" to set an interval at which gossip
775 occurs (default: 30000 i.e., 30 seconds).
776 * Fixed issue 190: add a way of aggregating performance data over
778 * Fixed issue 181: stack trackes shouldn't be filled for Obsolete
781 Release 0.60.1 on 12/18/2009
783 Minor changes made since 0.60:
785 * Better logging in the exception thrown if config/.temp and
786 config/.version are copied
787 * Bumping up the version to 0.60.1 in order to release updated
788 archives, fixing an error in the stores.xml for single_node_cluster
791 Release 0.60 on 12/15/2009
793 Changes made since 0.57.1:
795 * Admin Client/Server API: adds support for streaming-based transfer
796 of entries between nodes, deleting entries on remote nodes,
797 remotely deleting and updating metadata
798 * EC2 testing: a way to periodically run integration and performance
799 tests which involve Voldemort instances on different machines
800 * Experimental support for views
801 * Interpolation search for read-only stores
802 * Support for large lists and strings in the JSON serializer
803 * LZF compression support
804 * Ruby client contributed
805 * Fixed issue 170: hanging if a port is used by another process
806 * Fixed issue 122: suspicious integer division in
807 RequestCounter.getThroughput
808 * Miscellaneous improvements and bug fixes for read-only stores
809 * Fixed issue 168: added StorageEngine.keys()
811 Release 0.57.1 on 11/27/2009
813 Minor change made since 0.57:
815 * Modified build.xml to exclude .git directory from release tarballs/zipfiles
817 Release 0.57 on 11/16/2009
819 The following changes were made since 0.56:
821 * Fixed an issue in ReadOnlyEngine's close() method
822 * Fixed hidden logging in StorageService
823 * Fix for issue 163 (lock mode during get)
824 * Exposed bdb environment stats with setFast(false)
826 Release 0.56 on 10/26/2009
828 The following changes were made since 0.55:
830 * Fix for issue 164: Changed default bdb.max.logfile.size to 60MB
831 * Make file deletes asynchronous in read only store swap
832 * Added better debug logs for bdb stats
833 * Fixed race condition in AbstractSocketPoolTest
834 * Added improved monitoring for bdb stats
835 * Added backoff and retry logic in bootstrap code
836 * Not logging obosleteVersionException(s) or counting it in JMX exception count
838 * C++ client building on OS X
839 * Rely only on the number of versions returned to decide whether to retrieve
840 the value for put(K,V)
841 * Implemented issue 152: getVersion() API in Store
843 Release 0.55 on 10/7/2009
845 The following changes were made since 0.52 (in summary):
847 * Add an event throttler
848 * Added DataCleanupJob
849 * Protocol buffers at version 2.2.0
850 * BDB JE upgraded to 3.3.87
851 * Added data compression support (enable by adding
855 to value-serializer or key-serializer sections in stores.xml)
856 * Added a resource pool/socket pool implementation (no longer using
858 * Added server-side NIO support (enabled by setting
859 enable.nio.connector=true
860 in server.properties)
861 * Improved the efficiency of the protocol buffers network protocol by using
862 CodedOutputStream/CodedInputStream (zero copy transfers)
863 * Fix for issue #21: incorrectness in vector clock inconsistency resolver
864 * Upgrade google-collections
865 * Support for building read only stores in Hadoop