1 Summary: low
-level IO library which stores data
in huge blob files appending records one after another
7 Group: System Environment
/Libraries
8 URL: http
://www.ioremap.net
/projects
/eblob
9 Source0: %{name}-%{version}.tar.bz2
10 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n
)
12 %if %{defined rhel} && 0%{?rhel} < 6
13 BuildRequires
: python
-devel
, boost141
-python
, boost141
-devel
14 BuildRequires
: boost141
-iostreams
, boost141
-filesystem
, boost141
-thread
, boost141
-python
, boost141
-system
, boost141
-regex
16 BuildRequires
: snappy
-devel python
-devel
17 BuildRequires
: boost
-python
, boost
-devel
, boost
-filesystem
, boost
-thread
, boost
-python
, boost
-system
, boost
-regex
, boost
-iostreams
19 BuildRequires
: automake autoconf libtool
22 libeblob is a low
-level IO library which stores data
in huge blob files
23 appending records one after another.
25 * fast append
-only updates which
do not require disk seeks
26 * compact index to populate lookup information from disk
27 * multi
-threaded index reading during starup
28 * O
(1) data location lookup
time
29 * ability to lock
in-memory lookup index
(hash table
) to eliminate
31 * readahead games with data and index blobs
for maximum performance
32 * multiple blob files support
(tested with blob
-file
-as
-block
-device too
)
33 * optional sha256 on
-disk checksumming
34 * 2-stage write
: prepare
(which reserves the space
) and commit
35 (which calculates checksum and update
in-memory and on
-disk indexes
).
36 One can
(re
)write data using pwrite
() in between without locks
37 * usuall
1-stage write interface
38 * flexible configuration of
hash table size
, flags
, alignment
39 * defragmentation tool
: entries to be deleted are only marked as removed
,
40 eblob_check will iterate over specified blob files and actually
42 * off
-line blob consistency checker
: eblob_check can verify checksums
43 for all records which have them
44 * run
-time sync support
- dedicated thread runs fsync on all files
46 * in-memory index lives
in memory mapped file
49 Summary: Development files
for %{name}
50 Group: Development
/Libraries
51 Requires: %{name} = %{version}-%{release}
55 libeblob is a low
-level IO library which stores data
in huge blob files
56 appending records one after another.
58 This package contains libraries
, header files and developer documentation
59 needed
for developing software which uses the eblob library.
65 export LDFLAGS
="
-Wl
,-z
,defs"
67 %if %{defined rhel} && 0%{?rhel} < 6
68 CXXFLAGS
="
-pthread
-I
/usr
/include
/boost141" LDFLAGS
="
-L
/usr
/lib64
/boost141"
%configure --with
-boost
-libdir
=/usr
/lib64
/boost141
73 %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
74 %{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
82 make install DESTDIR
=%{buildroot}
83 rm -f
%{buildroot}%{_libdir}/*.a
84 rm -f
%{buildroot}%{_libdir}/*.la
86 %post -p
/sbin
/ldconfig
87 %postun -p
/sbin
/ldconfig
93 %defattr(-,root
,root
,-)
94 %doc AUTHORS AUTHORS COPYING README
97 %{python_sitelib}/eblob
*
101 %defattr(-,root
,root
,-)
106 * Sat Apr
28 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.9
107 - Write 'removed' record
in ->prepare
(), so it can not be
read in parallel. commit
() will
set correct bit. eblob_write
() will
do it too.
109 * Fri Apr
27 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.8
110 - Use correct F_SETFD
/FD_CLOEXEC
112 * Fri Apr
27 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.7
113 - added O_CLOEXEC flags
114 - Do not
return -ENOSPACE when EBLOB_RESERVE_10_PERCENTS is
set and size is less than blob
-size
, but more than
0.1 of total size
116 * Wed Apr
18 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.6
117 - disable defrag
for now
119 * Tue Apr
17 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.5
120 - Index and data files should be
644 by default to allow others like nginx to
read data
123 * Fri Apr
6 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.4
124 - Do not sync
in separate thread
if sync interfal is zero
- we will sync on every write anyway
125 - Added no
-footer blob flag
126 - Added overwrite
-commits flags
127 - Removed posix_fallocate
() call
128 - Added hole detection
129 - eblob_py_iterator cleanup
130 - Drop macroses which are not accessible
in centos
131 - Do not include blob.h into namespace
133 * Fri Mar
23 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.3
134 - Only reserve
10% of free space when EBLOB_RESERVE_10_PERCENTS blob
-config
-flag is
set
135 - Added prepare
/commit methods to cpp bindings
136 - Unify blob names
- rename existing
if found
137 - Generate bloom filter
in case of generation sorted index during startup
138 - Use sleep_time
-- instead of
--sleep_time to honor
1 second timeouts
139 - Sync iteration must be protected by b
->lock
141 * Mon Mar
19 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.2
142 - Fixed
read-modify
-write column update
143 - Fixed missed column during
type remove
144 - Use correct python path
145 - Added eblob
::truncate
()
146 - Protect bctl
->sort update
- fixed defragmentation race
147 - If user does not provide config
->log
then use stdout logger
in cpp binding
148 - Added eblob.py setup.py extra dist files
150 * Sun Mar
11 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.1
152 - Rename defragmented blobs just after they are created
154 - Do not use openat
()
156 * Wed Feb
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.15.0
157 - Added 'automatic'
test/check tool
158 - New automatic defragmentation implementation
159 - Added
::key
() method
162 * Sun Feb
19 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.4
163 - Added raw python blob class
164 - eblob_remove_all
(): when RAM lookup fails check disk
165 - Deleted eblob_gen_index.cpp
167 * Thu Feb
16 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.3
168 - Added python eblob constructor that accepts eblob_config structure on input
170 * Wed Feb
8 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.2
171 - Use allocate
/read for checksum calculation when reading small objects
172 - Skip .tmp files when scanning base names
174 * Wed Feb
1 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.14.1
175 - Added start
/num parameters to iterate over selected number of blobs
177 * Mon Jan
30 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.13
178 - Use ssize_t instead of int
in eblob_copy_data
()
180 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.12
181 - Use chunked splice
/copy
, since we may have really big objects written
183 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.11
184 - Allow to rollback preallocation
if prepare fails
186 * Sun Jan
29 2012 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.10
187 - Merge helpers and fixes
189 * Tue Dec
20 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.9
190 - If we are doing prepare
, and there is some old data
- reserve
2 times as much as requested
192 * Tue Dec
20 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.8
193 - Added free space check before trying to reserve space on disk
194 - Truncate stat file precisely to the number of bytes written
195 - Fixed cached keys removal
197 * Mon Dec
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.7
198 - Added actual index size checks.
199 - Update index early
in eblob_write_prepare_disk
().
200 - eblob_write
() does not need full eblob_write_commit_nolock
(), only low
-level on
-disk part.
202 * Sun Dec
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.6
203 - Fixed cache entry stall list head processing
205 * Sun Dec
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.5
206 - Fixed cache entry removal
208 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.4
209 - If commit provides size
, use it as object size. If prepare reuses existing region
, do not change its sizes.
210 - Changed eblob_write_prepare
() first to try to check existing region
if it is big enough.
212 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.3
213 - Initialize allocated memory
for ecache to
0
215 * Sat Dec
17 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.2
216 - Optimize prepare
(for eblob_write_prepare
() only
) call to reuse existing prepared area
if its size is big enough
218 * Thu Dec
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.1
219 - Fixed cache balancing
221 * Thu Dec
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.13.0
222 - Added configurable two
-queue cache
for read data by Anton Kortunov
<toshic.toshic@gmail.com
>
224 * Fri Dec
2 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.23
225 - Added eblob_read_nocsum
() - read data without csum check
227 * Sat Nov
26 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.22
228 - Added eblob_csum_ok
()
230 * Thu Nov
24 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.21
231 - Skip malformed zero entries during iteration
232 - eblob_fill_write_control_from_ram
() should not check whether aligned size is enough when doing
read command
234 * Wed Nov
23 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.20
235 - Initialize eblob_write_control to zeroes
in eblob_plain_write
()
237 * Sat Nov
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.19
238 - Multiple prepare
/commit fixes
for POHMELFS. Better write debug.
240 * Mon Nov
14 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.18
241 - Added array of IO locks to protect state update during IO
242 - Simplified eblob_write
()
243 - Dropped openssl bits
245 * Mon Oct
31 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.17
246 - Fill
in-memory rbtree and bloom filter
for online generated sorted index
247 - Do not update
in-memory structures before pwrite
248 - Added eblob_get_types
function
250 * Tue Oct
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.16
251 - Don't compare flags
in bsearch callback
253 * Mon Oct
24 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.15
254 - Do not open index and data files again
in eblob_iterate
255 - Update unsorted indexes of removed records with BLOB_DISK_CTL_REMOVE flag
256 - Use
50 millions of records
in blob by default
257 - Added eblob_merge
(instead of old eblob_defrag
) which defragments and
258 merges
(multiple
) blobs into larger one
260 * Wed Oct
19 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.14
261 - Added bsearch statistics and on
-disk lookup debugs
262 - Use
40 indexes per range block. Use
128 bits per index
for bloom.
264 * Tue Oct
18 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.13
265 - Fixed sorted index bsearch range calculation
267 * Sat Oct
15 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.12
268 - Added bloom filter
in sorted index blocks
269 - Added tree
for range index search
271 * Wed Oct
12 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.11
272 - Only
read mapped data blob
if EBLOB_ITERATE_FLAGS_ALL flag is
set
274 * Tue Oct
11 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.12.10
275 - Returned offline defrag tool
277 - Check
if record is removed
in blob
279 * Wed Jun
29 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.6.0
280 - Added snappy compression support
281 - Added start
/max types into iterator. Exoport nice interface outside.
(0.5.2)
283 * Sat Jun
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.5.1
284 - Create new files
, if eblob dir is empty
286 * Sat Jun
25 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.5.0
287 - Added multiple columns support
289 * Tue Jun
22 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.4.4
290 - Added eblob_remove_hashed
() and eblob
::remove_hashed
()
292 * Tue Jun
21 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.4.3
293 - Return real data offset
, do not force clients to mess with sizeof
(struct
296 - Renames write methods
297 - Switched to fixed
-size eblob_key interface
298 - Added namespace zbr
299 - Added eblob range query
300 - Added remove
() method
303 * Tue Jun
7 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.13
304 - Overwrite
hash entry
if we asked to replace it and sizes match
305 - Use
(part of
) provided key as
hash table index
306 - Preallocate more space
for hash entries
in a bucket
307 - Save references to open data
/index failes
in iterator
308 - Initial implementation of startup defragmentation
309 - Extended eblob_iterator class to support index and data iterators
311 * Fri Jun
3 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.12
312 - Use the same mmap file
for subsequent starts by default
313 - Grow and start map file by
1 Gb
315 * Thu Jun
2 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.11
316 - Do not move entries around
, since we are closing down whole mapping anyway
317 - Use posix_fallocate
() to preallocate sufficiently large mmap files
318 - Added completion callback and matched callback counters into iterator
319 - Allocate
/free mmap_file and use eblob file by default as a base path
for mmap file
320 - truncate map file to
0 at cleanup
322 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.10
323 - Remap new portions of backend allocation file at different locations.
325 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.7
326 - Use mremap
(MREMAP_FIXED | MREMAP_MAYMOVE
)
328 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.5
329 - Fixed ostringstream and its std
::string dereference
331 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.4
332 - Added
read() methods
333 - Use ostringstream
for mmap file name
335 * Wed Jun
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.3
338 * Tue May
31 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.3.2
339 - Moved
in-memory index into memory mapped file
340 - Fair number of other changes
342 * Tue Mar
1 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.7-1
345 * Mon Feb
28 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.6-1
346 - Correctly
set index's disk_size to sizeof
(struct eblob_disk_control
)
348 * Mon Feb
28 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.5-1
349 - Added index generating defragmenter.
351 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.2-1
352 - Do not lock entries
in eblob_hash_insert_raw
() since they should be locked via mlockall
().
354 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.2-1
355 - Set memory locking limits to infinity
if EBLOB_HASH_MLOCK is
set in config.
357 * Tue Feb
8 2011 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.1-1
358 - Use mlockall
(MCL_CURRENT | MCL_FUTURE
) to lock all current and future
359 allocations when EBLOB_HASH_MLOCK is
set.
361 * Mon Nov
29 2010 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.1.0-1
362 - Switched to sha512 and
64-byte IDs
364 * Sun Jul
28 2010 Evgeniy Polyakov
<zbr@ioremap.net
> - 0.0.1-1
365 - Initial build
for Fedora.