1 Summary: low
-level IO library which stores data
in huge blob files appending records one after another
7 Group: System Environment
/Libraries
8 URL: http
://www.ioremap.net
/projects
/eblob
9 Source0: %{name}-%{version}.tar.bz2
10 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n
)
12 %if %{defined rhel} && 0%{?rhel} < 6
13 BuildRequires
: boost141
-devel
, snappy
-devel
, boost141
-iostreams
, boost141
-thread
, boost141
-system
15 BuildRequires
: snappy
-devel
, boost
-devel
, boost
-thread
, boost
-system
, boost
-iostreams
17 BuildRequires
: automake autoconf libtool
20 libeblob is a low
-level IO library which stores data
in huge blob files
21 appending records one after another.
23 * fast append
-only updates which
do not require disk seeks
24 * compact index to populate lookup information from disk
25 * multi
-threaded index reading during starup
26 * O
(1) data location lookup
time
27 * ability to lock
in-memory lookup index
(hash table
) to eliminate
29 * readahead games with data and index blobs
for maximum performance
30 * multiple blob files support
(tested with blob
-file
-as
-block
-device too
)
31 * optional sha256 on
-disk checksumming
32 * 2-stage write
: prepare
(which reserves the space
) and commit
33 (which calculates checksum and update
in-memory and on
-disk indexes
).
34 One can
(re
)write data using pwrite
() in between without locks
35 * usuall
1-stage write interface
36 * flexible configuration of
hash table size
, flags
, alignment
37 * defragmentation tool
: entries to be deleted are only marked as removed
,
38 eblob_check will iterate over specified blob files and actually
40 * off
-line blob consistency checker
: eblob_check can verify checksums
41 for all records which have them
42 * run
-time sync support
- dedicated thread runs fsync on all files
44 * in-memory index lives
in memory mapped file
47 Summary: Development files
for %{name}
48 Group: Development
/Libraries
49 Requires: %{name} = %{version}-%{release}
53 libeblob is a low
-level IO library which stores data
in huge blob files
54 appending records one after another.
56 This package contains libraries
, header files and developer documentation
57 needed
for developing software which uses the eblob library.
63 export LDFLAGS
="
-Wl
,-z
,defs"
65 %if %{defined rhel} && 0%{?rhel} < 6
66 CXXFLAGS
="
-pthread
-I
/usr
/include
/boost141" LDFLAGS
="
-L
/usr
/lib64
/boost141"
%configure --with
-boost
-libdir
=/usr
/lib64
/boost141
71 %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
72 %{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
80 make install DESTDIR
=%{buildroot}
81 rm -f
%{buildroot}%{_libdir}/*.a
82 rm -f
%{buildroot}%{_libdir}/*.la
84 %post -p
/sbin
/ldconfig
85 %postun -p
/sbin
/ldconfig
91 %defattr(-,root
,root
,-)
92 %doc AUTHORS AUTHORS COPYING README
95 %{python_sitelib}/eblob
*
99 %defattr(-,root
,root
,-)
104 * Mon Jun
25 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.5
107 * Mon Jun
18 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.4
108 - Use libsnappy1 or snappy
(debian
-only build
)
110 * Mon Jun
18 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.3
111 - Use libsnappy1 instead of snappy
(debian
-only build
)
113 * Mon Jun
18 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.2
114 - Added flag which prevents free
-space check during write
116 * Sat Jun
9 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.1
117 - Enable defragmentation
119 * Thu Jun
7 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.16.0
120 - Use ioremap
::eblob namespace
122 * Thu May
17 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.10
123 - Fixed Precise build
125 * Sat Apr
28 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.9
126 - Write 'removed' record
in ->prepare
(), so it can not be
read in parallel. commit
() will
set correct bit. eblob_write
() will
do it too.
128 * Fri Apr
27 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.8
129 - Use correct F_SETFD
/FD_CLOEXEC
131 * Fri Apr
27 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.7
132 - added O_CLOEXEC flags
133 - Do not
return -ENOSPACE when EBLOB_RESERVE_10_PERCENTS is
set and size is less than blob
-size
, but more than
0.1 of total size
135 * Wed Apr
18 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.6
136 - disable defrag
for now
138 * Tue Apr
17 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.5
139 - Index and data files should be
644 by default to allow others like nginx to
read data
142 * Fri Apr
6 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.4
143 - Do not sync
in separate thread
if sync interfal is zero
- we will sync on every write anyway
144 - Added no
-footer blob flag
145 - Added overwrite
-commits flags
146 - Removed posix_fallocate
() call
147 - Added hole detection
148 - eblob_py_iterator cleanup
149 - Drop macroses which are not accessible
in centos
150 - Do not include blob.h into namespace
152 * Fri Mar
23 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.3
153 - Only reserve
10% of free space when EBLOB_RESERVE_10_PERCENTS blob
-config
-flag is
set
154 - Added prepare
/commit methods to cpp bindings
155 - Unify blob names
- rename existing
if found
156 - Generate bloom filter
in case of generation sorted index during startup
157 - Use sleep_time
-- instead of
--sleep_time to honor
1 second timeouts
158 - Sync iteration must be protected by b
->lock
160 * Mon Mar
19 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.2
161 - Fixed
read-modify
-write column update
162 - Fixed missed column during
type remove
163 - Use correct python path
164 - Added eblob
::truncate
()
165 - Protect bctl
->sort update
- fixed defragmentation race
166 - If user does not provide config
->log
then use stdout logger
in cpp binding
167 - Added eblob.py setup.py extra dist files
169 * Sun Mar
11 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.1
171 - Rename defragmented blobs just after they are created
173 - Do not use openat
()
175 * Wed Feb
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.0
176 - Added 'automatic'
test/check tool
177 - New automatic defragmentation implementation
178 - Added
::key
() method
181 * Sun Feb
19 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.4
182 - Added raw python blob class
183 - eblob_remove_all
(): when RAM lookup fails check disk
184 - Deleted eblob_gen_index.cpp
186 * Thu Feb
16 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.3
187 - Added python eblob constructor that accepts eblob_config structure on input
189 * Wed Feb
8 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.2
190 - Use allocate
/read for checksum calculation when reading small objects
191 - Skip .tmp files when scanning base names
193 * Wed Feb
1 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.1
194 - Added start
/num parameters to iterate over selected number of blobs
196 * Mon Jan
30 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.13
197 - Use ssize_t instead of int
in eblob_copy_data
()
199 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.12
200 - Use chunked splice
/copy
, since we may have really big objects written
202 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.11
203 - Allow to rollback preallocation
if prepare fails
205 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.10
206 - Merge helpers and fixes
208 * Tue Dec
20 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.9
209 - If we are doing prepare
, and there is some old data
- reserve
2 times as much as requested
211 * Tue Dec
20 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.8
212 - Added free space check before trying to reserve space on disk
213 - Truncate stat file precisely to the number of bytes written
214 - Fixed cached keys removal
216 * Mon Dec
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.7
217 - Added actual index size checks.
218 - Update index early
in eblob_write_prepare_disk
().
219 - eblob_write
() does not need full eblob_write_commit_nolock
(), only low
-level on
-disk part.
221 * Sun Dec
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.6
222 - Fixed cache entry stall list head processing
224 * Sun Dec
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.5
225 - Fixed cache entry removal
227 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.4
228 - If commit provides size
, use it as object size. If prepare reuses existing region
, do not change its sizes.
229 - Changed eblob_write_prepare
() first to try to check existing region
if it is big enough.
231 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.3
232 - Initialize allocated memory
for ecache to
0
234 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.2
235 - Optimize prepare
(for eblob_write_prepare
() only
) call to reuse existing prepared area
if its size is big enough
237 * Thu Dec
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.1
238 - Fixed cache balancing
240 * Thu Dec
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.0
241 - Added configurable two
-queue cache
for read data by Anton Kortunov
<toshic.toshic@gmail.com
>
243 * Fri Dec
2 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.23
244 - Added eblob_read_nocsum
() - read data without csum check
246 * Sat Nov
26 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.22
247 - Added eblob_csum_ok
()
249 * Thu Nov
24 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.21
250 - Skip malformed zero entries during iteration
251 - eblob_fill_write_control_from_ram
() should not check whether aligned size is enough when doing
read command
253 * Wed Nov
23 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.20
254 - Initialize eblob_write_control to zeroes
in eblob_plain_write
()
256 * Sat Nov
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.19
257 - Multiple prepare
/commit fixes
for POHMELFS. Better write debug.
259 * Mon Nov
14 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.18
260 - Added array of IO locks to protect state update during IO
261 - Simplified eblob_write
()
262 - Dropped openssl bits
264 * Mon Oct
31 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.17
265 - Fill
in-memory rbtree and bloom filter
for online generated sorted index
266 - Do not update
in-memory structures before pwrite
267 - Added eblob_get_types
function
269 * Tue Oct
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.16
270 - Don't compare flags
in bsearch callback
272 * Mon Oct
24 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.15
273 - Do not open index and data files again
in eblob_iterate
274 - Update unsorted indexes of removed records with BLOB_DISK_CTL_REMOVE flag
275 - Use
50 millions of records
in blob by default
276 - Added eblob_merge
(instead of old eblob_defrag
) which defragments and
277 merges
(multiple
) blobs into larger one
279 * Wed Oct
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.14
280 - Added bsearch statistics and on
-disk lookup debugs
281 - Use
40 indexes per range block. Use
128 bits per index
for bloom.
283 * Tue Oct
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.13
284 - Fixed sorted index bsearch range calculation
286 * Sat Oct
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.12
287 - Added bloom filter
in sorted index blocks
288 - Added tree
for range index search
290 * Wed Oct
12 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.11
291 - Only
read mapped data blob
if EBLOB_ITERATE_FLAGS_ALL flag is
set
293 * Tue Oct
11 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.10
294 - Returned offline defrag tool
296 - Check
if record is removed
in blob
298 * Wed Jun
29 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.6.0
299 - Added snappy compression support
300 - Added start
/max types into iterator. Exoport nice interface outside.
(0.5.2)
302 * Sat Jun
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.5.1
303 - Create new files
, if eblob dir is empty
305 * Sat Jun
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.5.0
306 - Added multiple columns support
308 * Tue Jun
22 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.4.4
309 - Added eblob_remove_hashed
() and eblob
::remove_hashed
()
311 * Tue Jun
21 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.4.3
312 - Return real data offset
, do not force clients to mess with sizeof
(struct
315 - Renames write methods
316 - Switched to fixed
-size eblob_key interface
317 - Added namespace zbr
318 - Added eblob range query
319 - Added remove
() method
322 * Tue Jun
7 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.13
323 - Overwrite
hash entry
if we asked to replace it and sizes match
324 - Use
(part of
) provided key as
hash table index
325 - Preallocate more space
for hash entries
in a bucket
326 - Save references to open data
/index failes
in iterator
327 - Initial implementation of startup defragmentation
328 - Extended eblob_iterator class to support index and data iterators
330 * Fri Jun
3 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.12
331 - Use the same mmap file
for subsequent starts by default
332 - Grow and start map file by
1 Gb
334 * Thu Jun
2 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.11
335 - Do not move entries around
, since we are closing down whole mapping anyway
336 - Use posix_fallocate
() to preallocate sufficiently large mmap files
337 - Added completion callback and matched callback counters into iterator
338 - Allocate
/free mmap_file and use eblob file by default as a base path
for mmap file
339 - truncate map file to
0 at cleanup
341 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.10
342 - Remap new portions of backend allocation file at different locations.
344 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.7
345 - Use mremap
(MREMAP_FIXED | MREMAP_MAYMOVE
)
347 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.5
348 - Fixed ostringstream and its std
::string dereference
350 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.4
351 - Added
read() methods
352 - Use ostringstream
for mmap file name
354 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.3
357 * Tue May
31 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.2
358 - Moved
in-memory index into memory mapped file
359 - Fair number of other changes
361 * Tue Mar
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.7-1
364 * Mon Feb
28 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.6-1
365 - Correctly
set index's disk_size to sizeof
(struct eblob_disk_control
)
367 * Mon Feb
28 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.5-1
368 - Added index generating defragmenter.
370 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.2-1
371 - Do not lock entries
in eblob_hash_insert_raw
() since they should be locked via mlockall
().
373 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.2-1
374 - Set memory locking limits to infinity
if EBLOB_HASH_MLOCK is
set in config.
376 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.1-1
377 - Use mlockall
(MCL_CURRENT | MCL_FUTURE
) to lock all current and future
378 allocations when EBLOB_HASH_MLOCK is
set.
380 * Mon Nov
29 2010 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.0-1
381 - Switched to sha512 and
64-byte IDs
383 * Sun Jul
28 2010 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.0.1-1
384 - Initial build
for Fedora.