4 ** The author disclaims copyright to this source code. In place of
5 ** a legal notice, here is a blessing:
7 ** May you do good and not evil.
8 ** May you find forgiveness for yourself and forgive others.
9 ** May you share freely, never taking more than you give.
11 *************************************************************************
14 ** This file implements a external (disk-based) database using BTrees.
15 ** For a detailed discussion of BTrees, refer to
17 ** Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
18 ** "Sorting And Searching", pages 473-480. Addison-Wesley
19 ** Publishing Company, Reading, Massachusetts.
21 ** The basic idea is that each page of the file contains N database
22 ** entries and N+1 pointers to subpages.
24 ** ----------------------------------------------------------------
25 ** | Ptr(0) | Key(0) | Ptr(1) | Key(1) | ... | Key(N) | Ptr(N+1) |
26 ** ----------------------------------------------------------------
28 ** All of the keys on the page that Ptr(0) points to have values less
29 ** than Key(0). All of the keys on page Ptr(1) and its subpages have
30 ** values greater than Key(0) and less than Key(1). All of the keys
31 ** on Ptr(N+1) and its subpages have values greater than Key(N). And
34 ** Finding a particular key requires reading O(log(M)) pages from the
35 ** disk where M is the number of entries in the tree.
37 ** In this implementation, a single file can hold one or more separate
38 ** BTrees. Each BTree is identified by the index of its root page. The
39 ** key and data for any entry are combined to form the "payload". A
40 ** fixed amount of payload can be carried directly on the database
41 ** page. If the payload is larger than the preset amount then surplus
42 ** bytes are stored on overflow pages. The payload for an entry
43 ** and the preceding pointer are combined to form a "Cell". Each
44 ** page has a small header which contains the Ptr(N+1) pointer and other
45 ** information such as the size of key and data.
49 ** The file is divided into pages. The first page is called page 1,
50 ** the second is page 2, and so forth. A page number of zero indicates
51 ** "no such page". The page size can be anything between 512 and 65536.
52 ** Each page can be either a btree page, a freelist page or an overflow
55 ** The first page is always a btree page. The first 100 bytes of the first
56 ** page contain a special header (the "file header") that describes the file.
57 ** The format of the file header is as follows:
59 ** OFFSET SIZE DESCRIPTION
60 ** 0 16 Header string: "SQLite format 3\000"
61 ** 16 2 Page size in bytes.
62 ** 18 1 File format write version
63 ** 19 1 File format read version
64 ** 20 1 Bytes of unused space at the end of each page
65 ** 21 1 Max embedded payload fraction
66 ** 22 1 Min embedded payload fraction
67 ** 23 1 Min leaf payload fraction
68 ** 24 4 File change counter
69 ** 28 4 Reserved for future use
70 ** 32 4 First freelist page
71 ** 36 4 Number of freelist pages in the file
72 ** 40 60 15 4-byte meta values passed to higher layers
74 ** All of the integer values are big-endian (most significant byte first).
76 ** The file change counter is incremented when the database is changed more
77 ** than once within the same second. This counter, together with the
78 ** modification time of the file, allows other processes to know
79 ** when the file has changed and thus when they need to flush their
82 ** The max embedded payload fraction is the amount of the total usable
83 ** space in a page that can be consumed by a single cell for standard
84 ** B-tree (non-LEAFDATA) tables. A value of 255 means 100%. The default
85 ** is to limit the maximum cell size so that at least 4 cells will fit
86 ** on one page. Thus the default max embedded payload fraction is 64.
88 ** If the payload for a cell is larger than the max payload, then extra
89 ** payload is spilled to overflow pages. Once an overflow page is allocated,
90 ** as many bytes as possible are moved into the overflow pages without letting
91 ** the cell size drop below the min embedded payload fraction.
93 ** The min leaf payload fraction is like the min embedded payload fraction
94 ** except that it applies to leaf nodes in a LEAFDATA tree. The maximum
95 ** payload fraction for a LEAFDATA tree is always 100% (or 255) and it
96 ** not specified in the header.
98 ** Each btree pages is divided into three sections: The header, the
99 ** cell pointer array, and the cell area area. Page 1 also has a 100-byte
100 ** file header that occurs before the page header.
102 ** |----------------|
103 ** | file header | 100 bytes. Page 1 only.
104 ** |----------------|
105 ** | page header | 8 bytes for leaves. 12 bytes for interior nodes
106 ** |----------------|
107 ** | cell pointer | | 2 bytes per cell. Sorted order.
108 ** | array | | Grows downward
110 ** |----------------|
113 ** |----------------| ^ Grows upwards
114 ** | cell content | | Arbitrary order interspersed with freeblocks.
115 ** | area | | and free space fragments.
116 ** |----------------|
118 ** The page headers looks like this:
120 ** OFFSET SIZE DESCRIPTION
121 ** 0 1 Flags. 1: intkey, 2: zerodata, 4: leafdata, 8: leaf
122 ** 1 2 byte offset to the first freeblock
123 ** 3 2 number of cells on this page
124 ** 5 2 first byte of the cell content area
125 ** 7 1 number of fragmented free bytes
126 ** 8 4 Right child (the Ptr(N+1) value). Omitted on leaves.
128 ** The flags define the format of this btree page. The leaf flag means that
129 ** this page has no children. The zerodata flag means that this page carries
130 ** only keys and no data. The intkey flag means that the key is a integer
131 ** which is stored in the key size entry of the cell header rather than in
134 ** The cell pointer array begins on the first byte after the page header.
135 ** The cell pointer array contains zero or more 2-byte numbers which are
136 ** offsets from the beginning of the page to the cell content in the cell
137 ** content area. The cell pointers occur in sorted order. The system strives
138 ** to keep free space after the last cell pointer so that new cells can
139 ** be easily added without having to defragment the page.
141 ** Cell content is stored at the very end of the page and grows toward the
142 ** beginning of the page.
144 ** Unused space within the cell content area is collected into a linked list of
145 ** freeblocks. Each freeblock is at least 4 bytes in size. The byte offset
146 ** to the first freeblock is given in the header. Freeblocks occur in
147 ** increasing order. Because a freeblock must be at least 4 bytes in size,
148 ** any group of 3 or fewer unused bytes in the cell content area cannot
149 ** exist on the freeblock chain. A group of 3 or fewer free bytes is called
150 ** a fragment. The total number of bytes in all fragments is recorded.
151 ** in the page header at offset 7.
154 ** 2 Byte offset of the next freeblock
155 ** 2 Bytes in this freeblock
157 ** Cells are of variable length. Cells are stored in the cell content area at
158 ** the end of the page. Pointers to the cells are in the cell pointer array
159 ** that immediately follows the page header. Cells is not necessarily
160 ** contiguous or in order, but cell pointers are contiguous and in order.
162 ** Cell content makes use of variable length integers. A variable
163 ** length integer is 1 to 9 bytes where the lower 7 bits of each
164 ** byte are used. The integer consists of all bytes that have bit 8 set and
165 ** the first byte with bit 8 clear. The most significant byte of the integer
166 ** appears first. A variable-length integer may not be more than 9 bytes long.
167 ** As a special case, all 8 bytes of the 9th byte are used as data. This
168 ** allows a 64-bit integer to be encoded in 9 bytes.
170 ** 0x00 becomes 0x00000000
171 ** 0x7f becomes 0x0000007f
172 ** 0x81 0x00 becomes 0x00000080
173 ** 0x82 0x00 becomes 0x00000100
174 ** 0x80 0x7f becomes 0x0000007f
175 ** 0x8a 0x91 0xd1 0xac 0x78 becomes 0x12345678
176 ** 0x81 0x81 0x81 0x81 0x01 becomes 0x10204081
178 ** Variable length integers are used for rowids and to hold the number of
179 ** bytes of key and data in a btree cell.
181 ** The content of a cell looks like this:
184 ** 4 Page number of the left child. Omitted if leaf flag is set.
185 ** var Number of bytes of data. Omitted if the zerodata flag is set.
186 ** var Number of bytes of key. Or the key itself if intkey flag is set.
188 ** 4 First page of the overflow chain. Omitted if no overflow
190 ** Overflow pages form a linked list. Each page except the last is completely
191 ** filled with data (pagesize - 4 bytes). The last page can have as little
192 ** as 1 byte of data.
195 ** 4 Page number of next overflow page
198 ** Freelist pages come in two subtypes: trunk pages and leaf pages. The
199 ** file header points to first in a linked list of trunk page. Each trunk
200 ** page points to multiple leaf pages. The content of a leaf page is
201 ** unspecified. A trunk page looks like this:
204 ** 4 Page number of next trunk page
205 ** 4 Number of leaf pointers on this page
206 ** * zero or more pages numbers of leaves
208 #include "sqliteInt.h"
214 /* Round up a number to the next larger multiple of 8. This is used
215 ** to force 8-byte alignment on 64-bit architectures.
217 #define ROUND8(x) ((x+7)&~7)
220 /* The following value is the maximum cell size assuming a maximum page
223 #define MX_CELL_SIZE(pBt) (pBt->pageSize-8)
225 /* The maximum number of cells on a single page of the database. This
226 ** assumes a minimum cell size of 3 bytes. Such small cells will be
227 ** exceedingly rare, but they are possible.
229 #define MX_CELL(pBt) ((pBt->pageSize-8)/3)
231 /* Forward declarations */
232 typedef struct MemPage MemPage
;
235 ** This is a magic string that appears at the beginning of every
236 ** SQLite database in order to identify the file as a real database.
237 ** 123456789 123456 */
238 static const char zMagicHeader
[] = "SQLite format 3";
241 ** Page type flags. An ORed combination of these flags appear as the
242 ** first byte of every BTree page.
244 #define PTF_INTKEY 0x01
245 #define PTF_ZERODATA 0x02
246 #define PTF_LEAFDATA 0x04
247 #define PTF_LEAF 0x08
250 ** As each page of the file is loaded into memory, an instance of the following
251 ** structure is appended and initialized to zero. This structure stores
252 ** information about the page that is decoded from the raw file page.
254 ** The pParent field points back to the parent page. This allows us to
255 ** walk up the BTree from any leaf to the root. Care must be taken to
256 ** unref() the parent page pointer when this page is no longer referenced.
257 ** The pageDestructor() routine handles that chore.
260 u8 isInit
; /* True if previously initialized. MUST BE FIRST! */
261 u8 idxShift
; /* True if Cell indices have changed */
262 u8 nOverflow
; /* Number of overflow cell bodies in aCell[] */
263 u8 intKey
; /* True if intkey flag is set */
264 u8 leaf
; /* True if leaf flag is set */
265 u8 zeroData
; /* True if table stores keys only */
266 u8 leafData
; /* True if tables stores data on leaves only */
267 u8 hasData
; /* True if this page stores data */
268 u8 hdrOffset
; /* 100 for page 1. 0 otherwise */
269 u8 childPtrSize
; /* 0 if leaf==1. 4 if leaf==0 */
270 u16 maxLocal
; /* Copy of Btree.maxLocal or Btree.maxLeaf */
271 u16 minLocal
; /* Copy of Btree.minLocal or Btree.minLeaf */
272 u16 cellOffset
; /* Index in aData of first cell pointer */
273 u16 idxParent
; /* Index in parent of this node */
274 u16 nFree
; /* Number of free bytes on the page */
275 u16 nCell
; /* Number of cells on this page, local and ovfl */
276 struct _OvflCell
{ /* Cells that will not fit on aData[] */
277 u8
*pCell
; /* Pointers to the body of the overflow cell */
278 u16 idx
; /* Insert this cell before idx-th non-overflow cell */
280 struct Btree
*pBt
; /* Pointer back to BTree structure */
281 u8
*aData
; /* Pointer back to the start of the page */
282 Pgno pgno
; /* Page number for this page */
283 MemPage
*pParent
; /* The parent of this page. NULL for root */
287 ** The in-memory image of a disk page has the auxiliary information appended
288 ** to the end. EXTRA_SIZE is the number of bytes of space needed to hold
289 ** that extra information.
291 #define EXTRA_SIZE sizeof(MemPage)
294 ** Everything we need to know about an open database
297 Pager
*pPager
; /* The page cache */
298 BtCursor
*pCursor
; /* A list of all open cursors */
299 MemPage
*pPage1
; /* First page of the database */
300 u8 inTrans
; /* True if a transaction is in progress */
301 u8 inStmt
; /* True if we are in a statement subtransaction */
302 u8 readOnly
; /* True if the underlying file is readonly */
303 u8 maxEmbedFrac
; /* Maximum payload as % of total page size */
304 u8 minEmbedFrac
; /* Minimum payload as % of total page size */
305 u8 minLeafFrac
; /* Minimum leaf payload as % of total page size */
306 u8 pageSizeFixed
; /* True if the page size can no longer be changed */
307 #ifndef SQLITE_OMIT_AUTOVACUUM
308 u8 autoVacuum
; /* True if database supports auto-vacuum */
310 u16 pageSize
; /* Total number of bytes on a page */
311 u16 usableSize
; /* Number of usable bytes on each page */
312 int maxLocal
; /* Maximum local payload in non-LEAFDATA tables */
313 int minLocal
; /* Minimum local payload in non-LEAFDATA tables */
314 int maxLeaf
; /* Maximum local payload in a LEAFDATA table */
315 int minLeaf
; /* Minimum local payload in a LEAFDATA table */
316 BusyHandler
*pBusyHandler
; /* Callback for when there is lock contention */
321 ** Btree.inTrans may take one of the following values.
325 #define TRANS_WRITE 2
328 ** An instance of the following structure is used to hold information
329 ** about a cell. The parseCellPtr() function fills in this structure
330 ** based on information extract from the raw disk page.
332 typedef struct CellInfo CellInfo
;
334 u8
*pCell
; /* Pointer to the start of cell content */
335 i64 nKey
; /* The key for INTKEY tables, or number of bytes in key */
336 u32 nData
; /* Number of bytes of data */
337 u16 nHeader
; /* Size of the cell content header in bytes */
338 u16 nLocal
; /* Amount of payload held locally */
339 u16 iOverflow
; /* Offset to overflow page number. Zero if no overflow */
340 u16 nSize
; /* Size of the cell content on the main b-tree page */
344 ** A cursor is a pointer to a particular entry in the BTree.
345 ** The entry is identified by its MemPage and the index in
346 ** MemPage.aCell[] of the entry.
349 Btree
*pBt
; /* The Btree to which this cursor belongs */
350 BtCursor
*pNext
, *pPrev
; /* Forms a linked list of all cursors */
351 int (*xCompare
)(void*,int,const void*,int,const void*); /* Key comp func */
352 void *pArg
; /* First arg to xCompare() */
353 Pgno pgnoRoot
; /* The root page of this tree */
354 MemPage
*pPage
; /* Page that contains the entry */
355 int idx
; /* Index of the entry in pPage->aCell[] */
356 CellInfo info
; /* A parse of the cell we are pointing at */
357 u8 wrFlag
; /* True if writable */
358 u8 isValid
; /* TRUE if points to a valid entry */
362 ** The TRACE macro will print high-level status information about the
363 ** btree operation when the global variable sqlite3_btree_trace is
367 # define TRACE(X) if( sqlite3_btree_trace )\
368 { sqlite3DebugPrintf X; fflush(stdout); }
372 int sqlite3_btree_trace
=0; /* True to enable tracing */
375 ** Forward declaration
377 static int checkReadLocks(Btree
*,Pgno
,BtCursor
*);
380 ** Read or write a two- and four-byte big-endian integer values.
382 static u32
get2byte(unsigned char *p
){
383 return (p
[0]<<8) | p
[1];
385 static u32
get4byte(unsigned char *p
){
386 return (p
[0]<<24) | (p
[1]<<16) | (p
[2]<<8) | p
[3];
388 static void put2byte(unsigned char *p
, u32 v
){
392 static void put4byte(unsigned char *p
, u32 v
){
400 ** Routines to read and write variable-length integers. These used to
401 ** be defined locally, but now we use the varint routines in the util.c
404 #define getVarint sqlite3GetVarint
405 #define getVarint32 sqlite3GetVarint32
406 #define putVarint sqlite3PutVarint
408 /* The database page the PENDING_BYTE occupies. This page is never used.
409 ** TODO: This macro is very similary to PAGER_MJ_PGNO() in pager.c. They
410 ** should possibly be consolidated (presumably in pager.h).
412 #define PENDING_BYTE_PAGE(pBt) ((PENDING_BYTE/(pBt)->pageSize)+1)
414 #ifndef SQLITE_OMIT_AUTOVACUUM
416 ** These macros define the location of the pointer-map entry for a
417 ** database page. The first argument to each is the number of usable
418 ** bytes on each page of the database (often 1024). The second is the
419 ** page number to look up in the pointer map.
421 ** PTRMAP_PAGENO returns the database page number of the pointer-map
422 ** page that stores the required pointer. PTRMAP_PTROFFSET returns
423 ** the offset of the requested map entry.
425 ** If the pgno argument passed to PTRMAP_PAGENO is a pointer-map page,
426 ** then pgno is returned. So (pgno==PTRMAP_PAGENO(pgsz, pgno)) can be
427 ** used to test if pgno is a pointer-map page. PTRMAP_ISPAGE implements
430 #define PTRMAP_PAGENO(pgsz, pgno) (((pgno-2)/(pgsz/5+1))*(pgsz/5+1)+2)
431 #define PTRMAP_PTROFFSET(pgsz, pgno) (((pgno-2)%(pgsz/5+1)-1)*5)
432 #define PTRMAP_ISPAGE(pgsz, pgno) (PTRMAP_PAGENO(pgsz,pgno)==pgno)
435 ** The pointer map is a lookup table that identifies the parent page for
436 ** each child page in the database file. The parent page is the page that
437 ** contains a pointer to the child. Every page in the database contains
438 ** 0 or 1 parent pages. (In this context 'database page' refers
439 ** to any page that is not part of the pointer map itself.) Each pointer map
440 ** entry consists of a single byte 'type' and a 4 byte parent page number.
441 ** The PTRMAP_XXX identifiers below are the valid types.
443 ** The purpose of the pointer map is to facility moving pages from one
444 ** position in the file to another as part of autovacuum. When a page
445 ** is moved, the pointer in its parent must be updated to point to the
446 ** new location. The pointer map is used to locate the parent page quickly.
448 ** PTRMAP_ROOTPAGE: The database page is a root-page. The page-number is not
449 ** used in this case.
451 ** PTRMAP_FREEPAGE: The database page is an unused (free) page. The page-number
452 ** is not used in this case.
454 ** PTRMAP_OVERFLOW1: The database page is the first page in a list of
455 ** overflow pages. The page number identifies the page that
456 ** contains the cell with a pointer to this overflow page.
458 ** PTRMAP_OVERFLOW2: The database page is the second or later page in a list of
459 ** overflow pages. The page-number identifies the previous
460 ** page in the overflow page list.
462 ** PTRMAP_BTREE: The database page is a non-root btree page. The page number
463 ** identifies the parent page in the btree.
465 #define PTRMAP_ROOTPAGE 1
466 #define PTRMAP_FREEPAGE 2
467 #define PTRMAP_OVERFLOW1 3
468 #define PTRMAP_OVERFLOW2 4
469 #define PTRMAP_BTREE 5
472 ** Write an entry into the pointer map.
474 ** This routine updates the pointer map entry for page number 'key'
475 ** so that it maps to type 'eType' and parent page number 'pgno'.
476 ** An error code is returned if something goes wrong, otherwise SQLITE_OK.
478 static int ptrmapPut(Btree
*pBt
, Pgno key
, u8 eType
, Pgno parent
){
479 u8
*pPtrmap
; /* The pointer map page */
480 Pgno iPtrmap
; /* The pointer map page number */
481 int offset
; /* Offset in pointer map page */
484 assert( pBt
->autoVacuum
);
486 return SQLITE_CORRUPT
;
488 iPtrmap
= PTRMAP_PAGENO(pBt
->usableSize
, key
);
489 rc
= sqlite3pager_get(pBt
->pPager
, iPtrmap
, (void **)&pPtrmap
);
493 offset
= PTRMAP_PTROFFSET(pBt
->usableSize
, key
);
495 if( eType
!=pPtrmap
[offset
] || get4byte(&pPtrmap
[offset
+1])!=parent
){
496 TRACE(("PTRMAP_UPDATE: %d->(%d,%d)\n", key
, eType
, parent
));
497 rc
= sqlite3pager_write(pPtrmap
);
499 pPtrmap
[offset
] = eType
;
500 put4byte(&pPtrmap
[offset
+1], parent
);
504 sqlite3pager_unref(pPtrmap
);
509 ** Read an entry from the pointer map.
511 ** This routine retrieves the pointer map entry for page 'key', writing
512 ** the type and parent page number to *pEType and *pPgno respectively.
513 ** An error code is returned if something goes wrong, otherwise SQLITE_OK.
515 static int ptrmapGet(Btree
*pBt
, Pgno key
, u8
*pEType
, Pgno
*pPgno
){
516 int iPtrmap
; /* Pointer map page index */
517 u8
*pPtrmap
; /* Pointer map page data */
518 int offset
; /* Offset of entry in pointer map */
521 iPtrmap
= PTRMAP_PAGENO(pBt
->usableSize
, key
);
522 rc
= sqlite3pager_get(pBt
->pPager
, iPtrmap
, (void **)&pPtrmap
);
527 offset
= PTRMAP_PTROFFSET(pBt
->usableSize
, key
);
528 if( pEType
) *pEType
= pPtrmap
[offset
];
529 if( pPgno
) *pPgno
= get4byte(&pPtrmap
[offset
+1]);
531 sqlite3pager_unref(pPtrmap
);
532 if( *pEType
<1 || *pEType
>5 ) return SQLITE_CORRUPT
;
536 #endif /* SQLITE_OMIT_AUTOVACUUM */
539 ** Given a btree page and a cell index (0 means the first cell on
540 ** the page, 1 means the second cell, and so forth) return a pointer
541 ** to the cell content.
543 ** This routine works only for pages that do not contain overflow cells.
545 static u8
*findCell(MemPage
*pPage
, int iCell
){
546 u8
*data
= pPage
->aData
;
548 assert( iCell
<get2byte(&data
[pPage
->hdrOffset
+3]) );
549 return data
+ get2byte(&data
[pPage
->cellOffset
+2*iCell
]);
553 ** This a more complex version of findCell() that works for
554 ** pages that do contain overflow cells. See insert
556 static u8
*findOverflowCell(MemPage
*pPage
, int iCell
){
558 for(i
=pPage
->nOverflow
-1; i
>=0; i
--){
560 struct _OvflCell
*pOvfl
;
561 pOvfl
= &pPage
->aOvfl
[i
];
570 return findCell(pPage
, iCell
);
574 ** Parse a cell content block and fill in the CellInfo structure. There
575 ** are two versions of this function. parseCell() takes a cell index
576 ** as the second argument and parseCellPtr() takes a pointer to the
577 ** body of the cell as its second argument.
579 static void parseCellPtr(
580 MemPage
*pPage
, /* Page containing the cell */
581 u8
*pCell
, /* Pointer to the cell text. */
582 CellInfo
*pInfo
/* Fill in this structure */
584 int n
; /* Number bytes in cell content header */
585 u32 nPayload
; /* Number of bytes of cell payload */
587 pInfo
->pCell
= pCell
;
588 assert( pPage
->leaf
==0 || pPage
->leaf
==1 );
589 n
= pPage
->childPtrSize
;
590 assert( n
==4-4*pPage
->leaf
);
591 if( pPage
->hasData
){
592 n
+= getVarint32(&pCell
[n
], &nPayload
);
596 n
+= getVarint(&pCell
[n
], (u64
*)&pInfo
->nKey
);
598 pInfo
->nData
= nPayload
;
599 if( !pPage
->intKey
){
600 nPayload
+= pInfo
->nKey
;
602 if( nPayload
<=pPage
->maxLocal
){
603 /* This is the (easy) common case where the entire payload fits
604 ** on the local page. No overflow is required.
606 int nSize
; /* Total size of cell content in bytes */
607 pInfo
->nLocal
= nPayload
;
608 pInfo
->iOverflow
= 0;
609 nSize
= nPayload
+ n
;
611 nSize
= 4; /* Minimum cell size is 4 */
613 pInfo
->nSize
= nSize
;
615 /* If the payload will not fit completely on the local page, we have
616 ** to decide how much to store locally and how much to spill onto
617 ** overflow pages. The strategy is to minimize the amount of unused
618 ** space on overflow pages while keeping the amount of local storage
619 ** in between minLocal and maxLocal.
621 ** Warning: changing the way overflow payload is distributed in any
622 ** way will result in an incompatible file format.
624 int minLocal
; /* Minimum amount of payload held locally */
625 int maxLocal
; /* Maximum amount of payload held locally */
626 int surplus
; /* Overflow payload available for local storage */
628 minLocal
= pPage
->minLocal
;
629 maxLocal
= pPage
->maxLocal
;
630 surplus
= minLocal
+ (nPayload
- minLocal
)%(pPage
->pBt
->usableSize
- 4);
631 if( surplus
<= maxLocal
){
632 pInfo
->nLocal
= surplus
;
634 pInfo
->nLocal
= minLocal
;
636 pInfo
->iOverflow
= pInfo
->nLocal
+ n
;
637 pInfo
->nSize
= pInfo
->iOverflow
+ 4;
640 static void parseCell(
641 MemPage
*pPage
, /* Page containing the cell */
642 int iCell
, /* The cell index. First cell is 0 */
643 CellInfo
*pInfo
/* Fill in this structure */
645 parseCellPtr(pPage
, findCell(pPage
, iCell
), pInfo
);
649 ** Compute the total number of bytes that a Cell needs in the cell
650 ** data area of the btree-page. The return number includes the cell
651 ** data header and the local payload, but not any overflow page or
652 ** the space used by the cell pointer.
655 static int cellSize(MemPage
*pPage
, int iCell
){
657 parseCell(pPage
, iCell
, &info
);
661 static int cellSizePtr(MemPage
*pPage
, u8
*pCell
){
663 parseCellPtr(pPage
, pCell
, &info
);
667 #ifndef SQLITE_OMIT_AUTOVACUUM
669 ** If the cell pCell, part of page pPage contains a pointer
670 ** to an overflow page, insert an entry into the pointer-map
671 ** for the overflow page.
673 static int ptrmapPutOvflPtr(MemPage
*pPage
, u8
*pCell
){
676 parseCellPtr(pPage
, pCell
, &info
);
677 if( (info
.nData
+(pPage
->intKey
?0:info
.nKey
))>info
.nLocal
){
678 Pgno ovfl
= get4byte(&pCell
[info
.iOverflow
]);
679 return ptrmapPut(pPage
->pBt
, ovfl
, PTRMAP_OVERFLOW1
, pPage
->pgno
);
685 ** If the cell with index iCell on page pPage contains a pointer
686 ** to an overflow page, insert an entry into the pointer-map
687 ** for the overflow page.
689 static int ptrmapPutOvfl(MemPage
*pPage
, int iCell
){
691 pCell
= findOverflowCell(pPage
, iCell
);
692 return ptrmapPutOvflPtr(pPage
, pCell
);
698 ** Do sanity checking on a page. Throw an exception if anything is
701 ** This routine is used for internal error checking only. It is omitted
704 #if defined(BTREE_DEBUG) && !defined(NDEBUG) && 0
705 static void _pageIntegrity(MemPage
*pPage
){
708 int i
, j
, idx
, c
, pc
, hdr
, nFree
;
710 int nCell
, cellLimit
;
713 used
= sqliteMallocRaw( pPage
->pBt
->pageSize
);
714 if( used
==0 ) return;
715 usableSize
= pPage
->pBt
->usableSize
;
716 assert( pPage
->aData
==&((unsigned char*)pPage
)[-pPage
->pBt
->pageSize
] );
717 hdr
= pPage
->hdrOffset
;
718 assert( hdr
==(pPage
->pgno
==1 ? 100 : 0) );
719 assert( pPage
->pgno
==sqlite3pager_pagenumber(pPage
->aData
) );
720 c
= pPage
->aData
[hdr
];
722 assert( pPage
->leaf
== ((c
& PTF_LEAF
)!=0) );
723 assert( pPage
->zeroData
== ((c
& PTF_ZERODATA
)!=0) );
724 assert( pPage
->leafData
== ((c
& PTF_LEAFDATA
)!=0) );
725 assert( pPage
->intKey
== ((c
& (PTF_INTKEY
|PTF_LEAFDATA
))!=0) );
726 assert( pPage
->hasData
==
727 !(pPage
->zeroData
|| (!pPage
->leaf
&& pPage
->leafData
)) );
728 assert( pPage
->cellOffset
==pPage
->hdrOffset
+12-4*pPage
->leaf
);
729 assert( pPage
->nCell
= get2byte(&pPage
->aData
[hdr
+3]) );
732 memset(used
, 0, usableSize
);
733 for(i
=0; i
<hdr
+10-pPage
->leaf
*4; i
++) used
[i
] = 1;
735 pc
= get2byte(&data
[hdr
+1]);
738 assert( pc
>0 && pc
<usableSize
-4 );
739 size
= get2byte(&data
[pc
+2]);
740 assert( pc
+size
<=usableSize
);
742 for(i
=pc
; i
<pc
+size
; i
++){
743 assert( used
[i
]==0 );
746 pc
= get2byte(&data
[pc
]);
749 nCell
= get2byte(&data
[hdr
+3]);
750 cellLimit
= get2byte(&data
[hdr
+5]);
751 assert( pPage
->isInit
==0
752 || pPage
->nFree
==nFree
+data
[hdr
+7]+cellLimit
-(cellOffset
+2*nCell
) );
753 cellOffset
= pPage
->cellOffset
;
754 for(i
=0; i
<nCell
; i
++){
756 pc
= get2byte(&data
[cellOffset
+2*i
]);
757 assert( pc
>0 && pc
<usableSize
-4 );
758 size
= cellSize(pPage
, &data
[pc
]);
759 assert( pc
+size
<=usableSize
);
760 for(j
=pc
; j
<pc
+size
; j
++){
761 assert( used
[j
]==0 );
765 for(i
=cellOffset
+2*nCell
; i
<cellimit
; i
++){
766 assert( used
[i
]==0 );
770 for(i
=0; i
<usableSize
; i
++){
771 assert( used
[i
]<=1 );
772 if( used
[i
]==0 ) nFree
++;
774 assert( nFree
==data
[hdr
+7] );
777 #define pageIntegrity(X) _pageIntegrity(X)
779 # define pageIntegrity(X)
783 ** Defragment the page given. All Cells are moved to the
784 ** beginning of the page and all free space is collected
785 ** into one big FreeBlk at the end of the page.
787 static int defragmentPage(MemPage
*pPage
){
788 int i
; /* Loop counter */
789 int pc
; /* Address of a i-th cell */
790 int addr
; /* Offset of first byte after cell pointer array */
791 int hdr
; /* Offset to the page header */
792 int size
; /* Size of a cell */
793 int usableSize
; /* Number of usable bytes on a page */
794 int cellOffset
; /* Offset to the cell pointer array */
795 int brk
; /* Offset to the cell content area */
796 int nCell
; /* Number of cells on the page */
797 unsigned char *data
; /* The page data */
798 unsigned char *temp
; /* Temp area for cell content */
800 assert( sqlite3pager_iswriteable(pPage
->aData
) );
801 assert( pPage
->pBt
!=0 );
802 assert( pPage
->pBt
->usableSize
<= SQLITE_MAX_PAGE_SIZE
);
803 assert( pPage
->nOverflow
==0 );
804 temp
= sqliteMalloc( pPage
->pBt
->pageSize
);
805 if( temp
==0 ) return SQLITE_NOMEM
;
807 hdr
= pPage
->hdrOffset
;
808 cellOffset
= pPage
->cellOffset
;
809 nCell
= pPage
->nCell
;
810 assert( nCell
==get2byte(&data
[hdr
+3]) );
811 usableSize
= pPage
->pBt
->usableSize
;
812 brk
= get2byte(&data
[hdr
+5]);
813 memcpy(&temp
[brk
], &data
[brk
], usableSize
- brk
);
815 for(i
=0; i
<nCell
; i
++){
816 u8
*pAddr
; /* The i-th cell pointer */
817 pAddr
= &data
[cellOffset
+ i
*2];
818 pc
= get2byte(pAddr
);
819 assert( pc
<pPage
->pBt
->usableSize
);
820 size
= cellSizePtr(pPage
, &temp
[pc
]);
822 memcpy(&data
[brk
], &temp
[pc
], size
);
823 put2byte(pAddr
, brk
);
825 assert( brk
>=cellOffset
+2*nCell
);
826 put2byte(&data
[hdr
+5], brk
);
830 addr
= cellOffset
+2*nCell
;
831 memset(&data
[addr
], 0, brk
-addr
);
837 ** Allocate nByte bytes of space on a page.
839 ** Return the index into pPage->aData[] of the first byte of
840 ** the new allocation. Or return 0 if there is not enough free
841 ** space on the page to satisfy the allocation request.
843 ** If the page contains nBytes of free space but does not contain
844 ** nBytes of contiguous free space, then this routine automatically
845 ** calls defragementPage() to consolidate all free space before
846 ** allocating the new chunk.
848 static int allocateSpace(MemPage
*pPage
, int nByte
){
858 assert( sqlite3pager_iswriteable(data
) );
859 assert( pPage
->pBt
);
860 if( nByte
<4 ) nByte
= 4;
861 if( pPage
->nFree
<nByte
|| pPage
->nOverflow
>0 ) return 0;
862 pPage
->nFree
-= nByte
;
863 hdr
= pPage
->hdrOffset
;
867 /* Search the freelist looking for a slot big enough to satisfy the
870 while( (pc
= get2byte(&data
[addr
]))>0 ){
871 size
= get2byte(&data
[pc
+2]);
874 memcpy(&data
[addr
], &data
[pc
], 2);
875 data
[hdr
+7] = nFrag
+ size
- nByte
;
878 put2byte(&data
[pc
+2], size
-nByte
);
879 return pc
+ size
- nByte
;
886 /* Allocate memory from the gap in between the cell pointer array
887 ** and the cell content area.
889 top
= get2byte(&data
[hdr
+5]);
890 nCell
= get2byte(&data
[hdr
+3]);
891 cellOffset
= pPage
->cellOffset
;
892 if( nFrag
>=60 || cellOffset
+ 2*nCell
> top
- nByte
){
893 if( defragmentPage(pPage
) ) return 0;
894 top
= get2byte(&data
[hdr
+5]);
897 assert( cellOffset
+ 2*nCell
<= top
);
898 put2byte(&data
[hdr
+5], top
);
903 ** Return a section of the pPage->aData to the freelist.
904 ** The first byte of the new free block is pPage->aDisk[start]
905 ** and the size of the block is "size" bytes.
907 ** Most of the effort here is involved in coalesing adjacent
908 ** free blocks into a single big free block.
910 static void freeSpace(MemPage
*pPage
, int start
, int size
){
911 int addr
, pbegin
, hdr
;
912 unsigned char *data
= pPage
->aData
;
914 assert( pPage
->pBt
!=0 );
915 assert( sqlite3pager_iswriteable(data
) );
916 assert( start
>=pPage
->hdrOffset
+6+(pPage
->leaf
?0:4) );
917 assert( (start
+ size
)<=pPage
->pBt
->usableSize
);
918 if( size
<4 ) size
= 4;
920 /* Add the space back into the linked list of freeblocks */
921 hdr
= pPage
->hdrOffset
;
923 while( (pbegin
= get2byte(&data
[addr
]))<start
&& pbegin
>0 ){
924 assert( pbegin
<=pPage
->pBt
->usableSize
-4 );
925 assert( pbegin
>addr
);
928 assert( pbegin
<=pPage
->pBt
->usableSize
-4 );
929 assert( pbegin
>addr
|| pbegin
==0 );
930 put2byte(&data
[addr
], start
);
931 put2byte(&data
[start
], pbegin
);
932 put2byte(&data
[start
+2], size
);
933 pPage
->nFree
+= size
;
935 /* Coalesce adjacent free blocks */
936 addr
= pPage
->hdrOffset
+ 1;
937 while( (pbegin
= get2byte(&data
[addr
]))>0 ){
939 assert( pbegin
>addr
);
940 assert( pbegin
<=pPage
->pBt
->usableSize
-4 );
941 pnext
= get2byte(&data
[pbegin
]);
942 psize
= get2byte(&data
[pbegin
+2]);
943 if( pbegin
+ psize
+ 3 >= pnext
&& pnext
>0 ){
944 int frag
= pnext
- (pbegin
+psize
);
945 assert( frag
<=data
[pPage
->hdrOffset
+7] );
946 data
[pPage
->hdrOffset
+7] -= frag
;
947 put2byte(&data
[pbegin
], get2byte(&data
[pnext
]));
948 put2byte(&data
[pbegin
+2], pnext
+get2byte(&data
[pnext
+2])-pbegin
);
954 /* If the cell content area begins with a freeblock, remove it. */
955 if( data
[hdr
+1]==data
[hdr
+5] && data
[hdr
+2]==data
[hdr
+6] ){
957 pbegin
= get2byte(&data
[hdr
+1]);
958 memcpy(&data
[hdr
+1], &data
[pbegin
], 2);
959 top
= get2byte(&data
[hdr
+5]);
960 put2byte(&data
[hdr
+5], top
+ get2byte(&data
[pbegin
+2]));
965 ** Decode the flags byte (the first byte of the header) for a page
966 ** and initialize fields of the MemPage structure accordingly.
968 static void decodeFlags(MemPage
*pPage
, int flagByte
){
969 Btree
*pBt
; /* A copy of pPage->pBt */
971 assert( pPage
->hdrOffset
==(pPage
->pgno
==1 ? 100 : 0) );
972 pPage
->intKey
= (flagByte
& (PTF_INTKEY
|PTF_LEAFDATA
))!=0;
973 pPage
->zeroData
= (flagByte
& PTF_ZERODATA
)!=0;
974 pPage
->leaf
= (flagByte
& PTF_LEAF
)!=0;
975 pPage
->childPtrSize
= 4*(pPage
->leaf
==0);
977 if( flagByte
& PTF_LEAFDATA
){
979 pPage
->maxLocal
= pBt
->maxLeaf
;
980 pPage
->minLocal
= pBt
->minLeaf
;
983 pPage
->maxLocal
= pBt
->maxLocal
;
984 pPage
->minLocal
= pBt
->minLocal
;
986 pPage
->hasData
= !(pPage
->zeroData
|| (!pPage
->leaf
&& pPage
->leafData
));
990 ** Initialize the auxiliary information for a disk block.
992 ** The pParent parameter must be a pointer to the MemPage which
993 ** is the parent of the page being initialized. The root of a
994 ** BTree has no parent and so for that page, pParent==NULL.
996 ** Return SQLITE_OK on success. If we see that the page does
997 ** not contain a well-formed database page, then return
998 ** SQLITE_CORRUPT. Note that a return of SQLITE_OK does not
999 ** guarantee that the page is well-formed. It only shows that
1000 ** we failed to detect any corruption.
1002 static int initPage(
1003 MemPage
*pPage
, /* The page to be initialized */
1004 MemPage
*pParent
/* The parent. Might be NULL */
1006 int pc
; /* Address of a freeblock within pPage->aData[] */
1007 int hdr
; /* Offset to beginning of page header */
1008 u8
*data
; /* Equal to pPage->aData */
1009 Btree
*pBt
; /* The main btree structure */
1010 int usableSize
; /* Amount of usable space on each page */
1011 int cellOffset
; /* Offset from start of page to first cell pointer */
1012 int nFree
; /* Number of unused bytes on the page */
1013 int top
; /* First byte of the cell content area */
1017 assert( pParent
==0 || pParent
->pBt
==pBt
);
1018 assert( pPage
->pgno
==sqlite3pager_pagenumber(pPage
->aData
) );
1019 assert( pPage
->aData
== &((unsigned char*)pPage
)[-pBt
->pageSize
] );
1020 if( pPage
->pParent
!=pParent
&& (pPage
->pParent
!=0 || pPage
->isInit
) ){
1021 /* The parent page should never change unless the file is corrupt */
1022 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1024 if( pPage
->isInit
) return SQLITE_OK
;
1025 if( pPage
->pParent
==0 && pParent
!=0 ){
1026 pPage
->pParent
= pParent
;
1027 sqlite3pager_ref(pParent
->aData
);
1029 hdr
= pPage
->hdrOffset
;
1030 data
= pPage
->aData
;
1031 decodeFlags(pPage
, data
[hdr
]);
1032 pPage
->nOverflow
= 0;
1033 pPage
->idxShift
= 0;
1034 usableSize
= pBt
->usableSize
;
1035 pPage
->cellOffset
= cellOffset
= hdr
+ 12 - 4*pPage
->leaf
;
1036 top
= get2byte(&data
[hdr
+5]);
1037 pPage
->nCell
= get2byte(&data
[hdr
+3]);
1038 if( pPage
->nCell
>MX_CELL(pBt
) ){
1039 /* To many cells for a single page. The page must be corrupt */
1040 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1042 if( pPage
->nCell
==0 && pParent
!=0 && pParent
->pgno
!=1 ){
1043 /* All pages must have at least one cell, except for root pages */
1044 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1047 /* Compute the total free space on the page */
1048 pc
= get2byte(&data
[hdr
+1]);
1049 nFree
= data
[hdr
+7] + top
- (cellOffset
+ 2*pPage
->nCell
);
1052 if( pc
>usableSize
-4 ){
1053 /* Free block is off the page */
1054 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1056 next
= get2byte(&data
[pc
]);
1057 size
= get2byte(&data
[pc
+2]);
1058 if( next
>0 && next
<=pc
+size
+3 ){
1059 /* Free blocks must be in accending order */
1060 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1065 pPage
->nFree
= nFree
;
1066 if( nFree
>=usableSize
){
1067 /* Free space cannot exceed total page size */
1068 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1072 pageIntegrity(pPage
);
1077 ** Set up a raw page so that it looks like a database page holding
1080 static void zeroPage(MemPage
*pPage
, int flags
){
1081 unsigned char *data
= pPage
->aData
;
1082 Btree
*pBt
= pPage
->pBt
;
1083 int hdr
= pPage
->hdrOffset
;
1086 assert( sqlite3pager_pagenumber(data
)==pPage
->pgno
);
1087 assert( &data
[pBt
->pageSize
] == (unsigned char*)pPage
);
1088 assert( sqlite3pager_iswriteable(data
) );
1089 memset(&data
[hdr
], 0, pBt
->usableSize
- hdr
);
1091 first
= hdr
+ 8 + 4*((flags
&PTF_LEAF
)==0);
1092 memset(&data
[hdr
+1], 0, 4);
1094 put2byte(&data
[hdr
+5], pBt
->usableSize
);
1095 pPage
->nFree
= pBt
->usableSize
- first
;
1096 decodeFlags(pPage
, flags
);
1097 pPage
->hdrOffset
= hdr
;
1098 pPage
->cellOffset
= first
;
1099 pPage
->nOverflow
= 0;
1100 pPage
->idxShift
= 0;
1103 pageIntegrity(pPage
);
1107 ** Get a page from the pager. Initialize the MemPage.pBt and
1108 ** MemPage.aData elements if needed.
1110 static int getPage(Btree
*pBt
, Pgno pgno
, MemPage
**ppPage
){
1112 unsigned char *aData
;
1114 rc
= sqlite3pager_get(pBt
->pPager
, pgno
, (void**)&aData
);
1116 pPage
= (MemPage
*)&aData
[pBt
->pageSize
];
1117 pPage
->aData
= aData
;
1120 pPage
->hdrOffset
= pPage
->pgno
==1 ? 100 : 0;
1126 ** Get a page from the pager and initialize it. This routine
1127 ** is just a convenience wrapper around separate calls to
1128 ** getPage() and initPage().
1130 static int getAndInitPage(
1131 Btree
*pBt
, /* The database file */
1132 Pgno pgno
, /* Number of the page to get */
1133 MemPage
**ppPage
, /* Write the page pointer here */
1134 MemPage
*pParent
/* Parent of the page */
1138 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
1140 rc
= getPage(pBt
, pgno
, ppPage
);
1141 if( rc
==SQLITE_OK
&& (*ppPage
)->isInit
==0 ){
1142 rc
= initPage(*ppPage
, pParent
);
1148 ** Release a MemPage. This should be called once for each prior
1151 static void releasePage(MemPage
*pPage
){
1153 assert( pPage
->aData
);
1154 assert( pPage
->pBt
);
1155 assert( &pPage
->aData
[pPage
->pBt
->pageSize
]==(unsigned char*)pPage
);
1156 sqlite3pager_unref(pPage
->aData
);
1161 ** This routine is called when the reference count for a page
1162 ** reaches zero. We need to unref the pParent pointer when that
1165 static void pageDestructor(void *pData
, int pageSize
){
1167 assert( (pageSize
& 7)==0 );
1168 pPage
= (MemPage
*)&((char*)pData
)[pageSize
];
1169 if( pPage
->pParent
){
1170 MemPage
*pParent
= pPage
->pParent
;
1172 releasePage(pParent
);
1178 ** During a rollback, when the pager reloads information into the cache
1179 ** so that the cache is restored to its original state at the start of
1180 ** the transaction, for each page restored this routine is called.
1182 ** This routine needs to reset the extra data section at the end of the
1183 ** page to agree with the restored data.
1185 static void pageReinit(void *pData
, int pageSize
){
1187 assert( (pageSize
& 7)==0 );
1188 pPage
= (MemPage
*)&((char*)pData
)[pageSize
];
1189 if( pPage
->isInit
){
1191 initPage(pPage
, pPage
->pParent
);
1196 ** Open a database file.
1198 ** zFilename is the name of the database file. If zFilename is NULL
1199 ** a new database with a random name is created. This randomly named
1200 ** database file will be deleted when sqlite3BtreeClose() is called.
1202 int sqlite3BtreeOpen(
1203 const char *zFilename
, /* Name of the file containing the BTree database */
1204 Btree
**ppBtree
, /* Pointer to new Btree object written here */
1205 int flags
/* Options */
1210 unsigned char zDbHeader
[100];
1213 ** The following asserts make sure that structures used by the btree are
1214 ** the right size. This is to guard against size changes that result
1215 ** when compiling on a different architecture.
1217 assert( sizeof(i64
)==8 );
1218 assert( sizeof(u64
)==8 );
1219 assert( sizeof(u32
)==4 );
1220 assert( sizeof(u16
)==2 );
1221 assert( sizeof(Pgno
)==4 );
1223 pBt
= sqliteMalloc( sizeof(*pBt
) );
1226 return SQLITE_NOMEM
;
1228 rc
= sqlite3pager_open(&pBt
->pPager
, zFilename
, EXTRA_SIZE
, flags
);
1229 if( rc
!=SQLITE_OK
){
1230 if( pBt
->pPager
) sqlite3pager_close(pBt
->pPager
);
1235 sqlite3pager_set_destructor(pBt
->pPager
, pageDestructor
);
1236 sqlite3pager_set_reiniter(pBt
->pPager
, pageReinit
);
1239 pBt
->readOnly
= sqlite3pager_isreadonly(pBt
->pPager
);
1240 sqlite3pager_read_fileheader(pBt
->pPager
, sizeof(zDbHeader
), zDbHeader
);
1241 pBt
->pageSize
= get2byte(&zDbHeader
[16]);
1242 if( pBt
->pageSize
<512 || pBt
->pageSize
>SQLITE_MAX_PAGE_SIZE
1243 || ((pBt
->pageSize
-1)&pBt
->pageSize
)!=0 ){
1244 pBt
->pageSize
= SQLITE_DEFAULT_PAGE_SIZE
;
1245 pBt
->maxEmbedFrac
= 64; /* 25% */
1246 pBt
->minEmbedFrac
= 32; /* 12.5% */
1247 pBt
->minLeafFrac
= 32; /* 12.5% */
1248 #ifndef SQLITE_OMIT_AUTOVACUUM
1249 /* If the magic name ":memory:" will create an in-memory database, then
1250 ** do not set the auto-vacuum flag, even if SQLITE_DEFAULT_AUTOVACUUM
1251 ** is true. On the other hand, if SQLITE_OMIT_MEMORYDB has been defined,
1252 ** then ":memory:" is just a regular file-name. Respect the auto-vacuum
1253 ** default in this case.
1255 #ifndef SQLITE_OMIT_MEMORYDB
1256 if( zFilename
&& strcmp(zFilename
,":memory:") ){
1260 pBt
->autoVacuum
= SQLITE_DEFAULT_AUTOVACUUM
;
1265 nReserve
= zDbHeader
[20];
1266 pBt
->maxEmbedFrac
= zDbHeader
[21];
1267 pBt
->minEmbedFrac
= zDbHeader
[22];
1268 pBt
->minLeafFrac
= zDbHeader
[23];
1269 pBt
->pageSizeFixed
= 1;
1270 #ifndef SQLITE_OMIT_AUTOVACUUM
1271 pBt
->autoVacuum
= (get4byte(&zDbHeader
[36 + 4*4])?1:0);
1274 pBt
->usableSize
= pBt
->pageSize
- nReserve
;
1275 assert( (pBt
->pageSize
& 7)==0 ); /* 8-byte alignment of pageSize */
1276 sqlite3pager_set_pagesize(pBt
->pPager
, pBt
->pageSize
);
1282 ** Close an open database and invalidate all cursors.
1284 int sqlite3BtreeClose(Btree
*pBt
){
1285 while( pBt
->pCursor
){
1286 sqlite3BtreeCloseCursor(pBt
->pCursor
);
1288 sqlite3pager_close(pBt
->pPager
);
1294 ** Change the busy handler callback function.
1296 int sqlite3BtreeSetBusyHandler(Btree
*pBt
, BusyHandler
*pHandler
){
1297 pBt
->pBusyHandler
= pHandler
;
1298 sqlite3pager_set_busyhandler(pBt
->pPager
, pHandler
);
1303 ** Change the limit on the number of pages allowed in the cache.
1305 ** The maximum number of cache pages is set to the absolute
1306 ** value of mxPage. If mxPage is negative, the pager will
1307 ** operate asynchronously - it will not stop to do fsync()s
1308 ** to insure data is written to the disk surface before
1309 ** continuing. Transactions still work if synchronous is off,
1310 ** and the database cannot be corrupted if this program
1311 ** crashes. But if the operating system crashes or there is
1312 ** an abrupt power failure when synchronous is off, the database
1313 ** could be left in an inconsistent and unrecoverable state.
1314 ** Synchronous is on by default so database corruption is not
1315 ** normally a worry.
1317 int sqlite3BtreeSetCacheSize(Btree
*pBt
, int mxPage
){
1318 sqlite3pager_set_cachesize(pBt
->pPager
, mxPage
);
1323 ** Change the way data is synced to disk in order to increase or decrease
1324 ** how well the database resists damage due to OS crashes and power
1325 ** failures. Level 1 is the same as asynchronous (no syncs() occur and
1326 ** there is a high probability of damage) Level 2 is the default. There
1327 ** is a very low but non-zero probability of damage. Level 3 reduces the
1328 ** probability of damage to near zero but with a write performance reduction.
1330 #ifndef SQLITE_OMIT_PAGER_PRAGMAS
1331 int sqlite3BtreeSetSafetyLevel(Btree
*pBt
, int level
){
1332 sqlite3pager_set_safety_level(pBt
->pPager
, level
);
1337 #if !defined(SQLITE_OMIT_PAGER_PRAGMAS) || !defined(SQLITE_OMIT_VACUUM)
1339 ** Change the default pages size and the number of reserved bytes per page.
1341 ** The page size must be a power of 2 between 512 and 65536. If the page
1342 ** size supplied does not meet this constraint then the page size is not
1345 ** Page sizes are constrained to be a power of two so that the region
1346 ** of the database file used for locking (beginning at PENDING_BYTE,
1347 ** the first byte past the 1GB boundary, 0x40000000) needs to occur
1348 ** at the beginning of a page.
1350 ** If parameter nReserve is less than zero, then the number of reserved
1351 ** bytes per page is left unchanged.
1353 int sqlite3BtreeSetPageSize(Btree
*pBt
, int pageSize
, int nReserve
){
1354 if( pBt
->pageSizeFixed
){
1355 return SQLITE_READONLY
;
1358 nReserve
= pBt
->pageSize
- pBt
->usableSize
;
1360 if( pageSize
>=512 && pageSize
<=SQLITE_MAX_PAGE_SIZE
&&
1361 ((pageSize
-1)&pageSize
)==0 ){
1362 assert( (pageSize
& 7)==0 );
1363 pBt
->pageSize
= sqlite3pager_set_pagesize(pBt
->pPager
, pageSize
);
1365 pBt
->usableSize
= pBt
->pageSize
- nReserve
;
1370 ** Return the currently defined page size
1372 int sqlite3BtreeGetPageSize(Btree
*pBt
){
1373 return pBt
->pageSize
;
1375 int sqlite3BtreeGetReserve(Btree
*pBt
){
1376 return pBt
->pageSize
- pBt
->usableSize
;
1378 #endif /* !defined(SQLITE_OMIT_PAGER_PRAGMAS) || !defined(SQLITE_OMIT_VACUUM) */
1381 ** Change the 'auto-vacuum' property of the database. If the 'autoVacuum'
1382 ** parameter is non-zero, then auto-vacuum mode is enabled. If zero, it
1383 ** is disabled. The default value for the auto-vacuum property is
1384 ** determined by the SQLITE_DEFAULT_AUTOVACUUM macro.
1386 int sqlite3BtreeSetAutoVacuum(Btree
*pBt
, int autoVacuum
){
1387 #ifdef SQLITE_OMIT_AUTOVACUUM
1388 return SQLITE_READONLY
;
1390 if( pBt
->pageSizeFixed
){
1391 return SQLITE_READONLY
;
1393 pBt
->autoVacuum
= (autoVacuum
?1:0);
1399 ** Return the value of the 'auto-vacuum' property. If auto-vacuum is
1400 ** enabled 1 is returned. Otherwise 0.
1402 int sqlite3BtreeGetAutoVacuum(Btree
*pBt
){
1403 #ifdef SQLITE_OMIT_AUTOVACUUM
1406 return pBt
->autoVacuum
;
1412 ** Get a reference to pPage1 of the database file. This will
1413 ** also acquire a readlock on that file.
1415 ** SQLITE_OK is returned on success. If the file is not a
1416 ** well-formed database file, then SQLITE_CORRUPT is returned.
1417 ** SQLITE_BUSY is returned if the database is locked. SQLITE_NOMEM
1418 ** is returned if we run out of memory. SQLITE_PROTOCOL is returned
1419 ** if there is a locking protocol violation.
1421 static int lockBtree(Btree
*pBt
){
1424 if( pBt
->pPage1
) return SQLITE_OK
;
1425 rc
= getPage(pBt
, 1, &pPage1
);
1426 if( rc
!=SQLITE_OK
) return rc
;
1429 /* Do some checking to help insure the file we opened really is
1430 ** a valid database file.
1433 if( sqlite3pager_pagecount(pBt
->pPager
)>0 ){
1434 u8
*page1
= pPage1
->aData
;
1435 if( memcmp(page1
, zMagicHeader
, 16)!=0 ){
1436 goto page1_init_failed
;
1438 if( page1
[18]>1 || page1
[19]>1 ){
1439 goto page1_init_failed
;
1441 pageSize
= get2byte(&page1
[16]);
1442 if( ((pageSize
-1)&pageSize
)!=0 ){
1443 goto page1_init_failed
;
1445 assert( (pageSize
& 7)==0 );
1446 pBt
->pageSize
= pageSize
;
1447 pBt
->usableSize
= pageSize
- page1
[20];
1448 if( pBt
->usableSize
<500 ){
1449 goto page1_init_failed
;
1451 pBt
->maxEmbedFrac
= page1
[21];
1452 pBt
->minEmbedFrac
= page1
[22];
1453 pBt
->minLeafFrac
= page1
[23];
1454 #ifndef SQLITE_OMIT_AUTOVACUUM
1455 pBt
->autoVacuum
= (get4byte(&page1
[36 + 4*4])?1:0);
1459 /* maxLocal is the maximum amount of payload to store locally for
1460 ** a cell. Make sure it is small enough so that at least minFanout
1461 ** cells can will fit on one page. We assume a 10-byte page header.
1462 ** Besides the payload, the cell must store:
1463 ** 2-byte pointer to the cell
1464 ** 4-byte child pointer
1465 ** 9-byte nKey value
1466 ** 4-byte nData value
1467 ** 4-byte overflow page pointer
1468 ** So a cell consists of a 2-byte poiner, a header which is as much as
1469 ** 17 bytes long, 0 to N bytes of payload, and an optional 4 byte overflow
1472 pBt
->maxLocal
= (pBt
->usableSize
-12)*pBt
->maxEmbedFrac
/255 - 23;
1473 pBt
->minLocal
= (pBt
->usableSize
-12)*pBt
->minEmbedFrac
/255 - 23;
1474 pBt
->maxLeaf
= pBt
->usableSize
- 35;
1475 pBt
->minLeaf
= (pBt
->usableSize
-12)*pBt
->minLeafFrac
/255 - 23;
1476 if( pBt
->minLocal
>pBt
->maxLocal
|| pBt
->maxLocal
<0 ){
1477 goto page1_init_failed
;
1479 assert( pBt
->maxLeaf
+ 23 <= MX_CELL_SIZE(pBt
) );
1480 pBt
->pPage1
= pPage1
;
1484 releasePage(pPage1
);
1490 ** This routine works like lockBtree() except that it also invokes the
1491 ** busy callback if there is lock contention.
1493 static int lockBtreeWithRetry(Btree
*pBt
){
1495 if( pBt
->inTrans
==TRANS_NONE
){
1496 rc
= sqlite3BtreeBeginTrans(pBt
, 0);
1497 pBt
->inTrans
= TRANS_NONE
;
1504 ** If there are no outstanding cursors and we are not in the middle
1505 ** of a transaction but there is a read lock on the database, then
1506 ** this routine unrefs the first page of the database file which
1507 ** has the effect of releasing the read lock.
1509 ** If there are any outstanding cursors, this routine is a no-op.
1511 ** If there is a transaction in progress, this routine is a no-op.
1513 static void unlockBtreeIfUnused(Btree
*pBt
){
1514 if( pBt
->inTrans
==TRANS_NONE
&& pBt
->pCursor
==0 && pBt
->pPage1
!=0 ){
1515 if( pBt
->pPage1
->aData
==0 ){
1516 MemPage
*pPage
= pBt
->pPage1
;
1517 pPage
->aData
= &((char*)pPage
)[-pBt
->pageSize
];
1521 releasePage(pBt
->pPage1
);
1528 ** Create a new database by initializing the first page of the
1531 static int newDatabase(Btree
*pBt
){
1533 unsigned char *data
;
1535 if( sqlite3pager_pagecount(pBt
->pPager
)>0 ) return SQLITE_OK
;
1539 rc
= sqlite3pager_write(data
);
1541 memcpy(data
, zMagicHeader
, sizeof(zMagicHeader
));
1542 assert( sizeof(zMagicHeader
)==16 );
1543 put2byte(&data
[16], pBt
->pageSize
);
1546 data
[20] = pBt
->pageSize
- pBt
->usableSize
;
1547 data
[21] = pBt
->maxEmbedFrac
;
1548 data
[22] = pBt
->minEmbedFrac
;
1549 data
[23] = pBt
->minLeafFrac
;
1550 memset(&data
[24], 0, 100-24);
1551 zeroPage(pP1
, PTF_INTKEY
|PTF_LEAF
|PTF_LEAFDATA
);
1552 pBt
->pageSizeFixed
= 1;
1553 #ifndef SQLITE_OMIT_AUTOVACUUM
1554 if( pBt
->autoVacuum
){
1555 put4byte(&data
[36 + 4*4], 1);
1562 ** Attempt to start a new transaction. A write-transaction
1563 ** is started if the second argument is nonzero, otherwise a read-
1564 ** transaction. If the second argument is 2 or more and exclusive
1565 ** transaction is started, meaning that no other process is allowed
1566 ** to access the database. A preexisting transaction may not be
1567 ** upgraded to exclusive by calling this routine a second time - the
1568 ** exclusivity flag only works for a new transaction.
1570 ** A write-transaction must be started before attempting any
1571 ** changes to the database. None of the following routines
1572 ** will work unless a transaction is started first:
1574 ** sqlite3BtreeCreateTable()
1575 ** sqlite3BtreeCreateIndex()
1576 ** sqlite3BtreeClearTable()
1577 ** sqlite3BtreeDropTable()
1578 ** sqlite3BtreeInsert()
1579 ** sqlite3BtreeDelete()
1580 ** sqlite3BtreeUpdateMeta()
1582 ** If an initial attempt to acquire the lock fails because of lock contention
1583 ** and the database was previously unlocked, then invoke the busy handler
1584 ** if there is one. But if there was previously a read-lock, do not
1585 ** invoke the busy handler - just return SQLITE_BUSY. SQLITE_BUSY is
1586 ** returned when there is already a read-lock in order to avoid a deadlock.
1588 ** Suppose there are two processes A and B. A has a read lock and B has
1589 ** a reserved lock. B tries to promote to exclusive but is blocked because
1590 ** of A's read lock. A tries to promote to reserved but is blocked by B.
1591 ** One or the other of the two processes must give way or there can be
1592 ** no progress. By returning SQLITE_BUSY and not invoking the busy callback
1593 ** when A already has a read lock, we encourage A to give up and let B
1596 int sqlite3BtreeBeginTrans(Btree
*pBt
, int wrflag
){
1601 /* If the btree is already in a write-transaction, or it
1602 ** is already in a read-transaction and a read-transaction
1603 ** is requested, this is a no-op.
1605 if( pBt
->inTrans
==TRANS_WRITE
|| (pBt
->inTrans
==TRANS_READ
&& !wrflag
) ){
1609 /* Write transactions are not possible on a read-only database */
1610 if( pBt
->readOnly
&& wrflag
){
1611 return SQLITE_READONLY
;
1615 if( pBt
->pPage1
==0 ){
1616 rc
= lockBtree(pBt
);
1619 if( rc
==SQLITE_OK
&& wrflag
){
1620 rc
= sqlite3pager_begin(pBt
->pPage1
->aData
, wrflag
>1);
1621 if( rc
==SQLITE_OK
){
1622 rc
= newDatabase(pBt
);
1626 if( rc
==SQLITE_OK
){
1627 pBt
->inTrans
= (wrflag
?TRANS_WRITE
:TRANS_READ
);
1628 if( wrflag
) pBt
->inStmt
= 0;
1630 unlockBtreeIfUnused(pBt
);
1632 }while( rc
==SQLITE_BUSY
&& pBt
->inTrans
==TRANS_NONE
&&
1633 (pH
= pBt
->pBusyHandler
)!=0 &&
1634 pH
->xFunc
&& pH
->xFunc(pH
->pArg
, busy
++)
1639 #ifndef SQLITE_OMIT_AUTOVACUUM
1642 ** Set the pointer-map entries for all children of page pPage. Also, if
1643 ** pPage contains cells that point to overflow pages, set the pointer
1644 ** map entries for the overflow pages as well.
1646 static int setChildPtrmaps(MemPage
*pPage
){
1647 int i
; /* Counter variable */
1648 int nCell
; /* Number of cells in page pPage */
1649 int rc
= SQLITE_OK
; /* Return code */
1650 Btree
*pBt
= pPage
->pBt
;
1651 int isInitOrig
= pPage
->isInit
;
1652 Pgno pgno
= pPage
->pgno
;
1655 nCell
= pPage
->nCell
;
1657 for(i
=0; i
<nCell
; i
++){
1658 u8
*pCell
= findCell(pPage
, i
);
1660 rc
= ptrmapPutOvflPtr(pPage
, pCell
);
1661 if( rc
!=SQLITE_OK
){
1662 goto set_child_ptrmaps_out
;
1666 Pgno childPgno
= get4byte(pCell
);
1667 rc
= ptrmapPut(pBt
, childPgno
, PTRMAP_BTREE
, pgno
);
1668 if( rc
!=SQLITE_OK
) goto set_child_ptrmaps_out
;
1673 Pgno childPgno
= get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]);
1674 rc
= ptrmapPut(pBt
, childPgno
, PTRMAP_BTREE
, pgno
);
1677 set_child_ptrmaps_out
:
1678 pPage
->isInit
= isInitOrig
;
1683 ** Somewhere on pPage, which is guarenteed to be a btree page, not an overflow
1684 ** page, is a pointer to page iFrom. Modify this pointer so that it points to
1685 ** iTo. Parameter eType describes the type of pointer to be modified, as
1688 ** PTRMAP_BTREE: pPage is a btree-page. The pointer points at a child
1691 ** PTRMAP_OVERFLOW1: pPage is a btree-page. The pointer points at an overflow
1692 ** page pointed to by one of the cells on pPage.
1694 ** PTRMAP_OVERFLOW2: pPage is an overflow-page. The pointer points at the next
1695 ** overflow page in the list.
1697 static int modifyPagePointer(MemPage
*pPage
, Pgno iFrom
, Pgno iTo
, u8 eType
){
1698 if( eType
==PTRMAP_OVERFLOW2
){
1699 /* The pointer is always the first 4 bytes of the page in this case. */
1700 if( get4byte(pPage
->aData
)!=iFrom
){
1701 return SQLITE_CORRUPT
;
1703 put4byte(pPage
->aData
, iTo
);
1705 int isInitOrig
= pPage
->isInit
;
1710 nCell
= pPage
->nCell
;
1712 for(i
=0; i
<nCell
; i
++){
1713 u8
*pCell
= findCell(pPage
, i
);
1714 if( eType
==PTRMAP_OVERFLOW1
){
1716 parseCellPtr(pPage
, pCell
, &info
);
1717 if( info
.iOverflow
){
1718 if( iFrom
==get4byte(&pCell
[info
.iOverflow
]) ){
1719 put4byte(&pCell
[info
.iOverflow
], iTo
);
1724 if( get4byte(pCell
)==iFrom
){
1725 put4byte(pCell
, iTo
);
1732 if( eType
!=PTRMAP_BTREE
||
1733 get4byte(&pPage
->aData
[pPage
->hdrOffset
+8])!=iFrom
){
1734 return SQLITE_CORRUPT
;
1736 put4byte(&pPage
->aData
[pPage
->hdrOffset
+8], iTo
);
1739 pPage
->isInit
= isInitOrig
;
1746 ** Move the open database page pDbPage to location iFreePage in the
1747 ** database. The pDbPage reference remains valid.
1749 static int relocatePage(
1750 Btree
*pBt
, /* Btree */
1751 MemPage
*pDbPage
, /* Open page to move */
1752 u8 eType
, /* Pointer map 'type' entry for pDbPage */
1753 Pgno iPtrPage
, /* Pointer map 'page-no' entry for pDbPage */
1754 Pgno iFreePage
/* The location to move pDbPage to */
1756 MemPage
*pPtrPage
; /* The page that contains a pointer to pDbPage */
1757 Pgno iDbPage
= pDbPage
->pgno
;
1758 Pager
*pPager
= pBt
->pPager
;
1761 assert( eType
==PTRMAP_OVERFLOW2
|| eType
==PTRMAP_OVERFLOW1
||
1762 eType
==PTRMAP_BTREE
|| eType
==PTRMAP_ROOTPAGE
);
1764 /* Move page iDbPage from it's current location to page number iFreePage */
1765 TRACE(("AUTOVACUUM: Moving %d to free page %d (ptr page %d type %d)\n",
1766 iDbPage
, iFreePage
, iPtrPage
, eType
));
1767 rc
= sqlite3pager_movepage(pPager
, pDbPage
->aData
, iFreePage
);
1768 if( rc
!=SQLITE_OK
){
1771 pDbPage
->pgno
= iFreePage
;
1773 /* If pDbPage was a btree-page, then it may have child pages and/or cells
1774 ** that point to overflow pages. The pointer map entries for all these
1775 ** pages need to be changed.
1777 ** If pDbPage is an overflow page, then the first 4 bytes may store a
1778 ** pointer to a subsequent overflow page. If this is the case, then
1779 ** the pointer map needs to be updated for the subsequent overflow page.
1781 if( eType
==PTRMAP_BTREE
|| eType
==PTRMAP_ROOTPAGE
){
1782 rc
= setChildPtrmaps(pDbPage
);
1783 if( rc
!=SQLITE_OK
){
1787 Pgno nextOvfl
= get4byte(pDbPage
->aData
);
1789 rc
= ptrmapPut(pBt
, nextOvfl
, PTRMAP_OVERFLOW2
, iFreePage
);
1790 if( rc
!=SQLITE_OK
){
1796 /* Fix the database pointer on page iPtrPage that pointed at iDbPage so
1797 ** that it points at iFreePage. Also fix the pointer map entry for
1800 if( eType
!=PTRMAP_ROOTPAGE
){
1801 rc
= getPage(pBt
, iPtrPage
, &pPtrPage
);
1802 if( rc
!=SQLITE_OK
){
1805 rc
= sqlite3pager_write(pPtrPage
->aData
);
1806 if( rc
!=SQLITE_OK
){
1807 releasePage(pPtrPage
);
1810 rc
= modifyPagePointer(pPtrPage
, iDbPage
, iFreePage
, eType
);
1811 releasePage(pPtrPage
);
1812 if( rc
==SQLITE_OK
){
1813 rc
= ptrmapPut(pBt
, iFreePage
, eType
, iPtrPage
);
1819 /* Forward declaration required by autoVacuumCommit(). */
1820 static int allocatePage(Btree
*, MemPage
**, Pgno
*, Pgno
, u8
);
1823 ** This routine is called prior to sqlite3pager_commit when a transaction
1824 ** is commited for an auto-vacuum database.
1826 static int autoVacuumCommit(Btree
*pBt
, Pgno
*nTrunc
){
1827 Pager
*pPager
= pBt
->pPager
;
1828 Pgno nFreeList
; /* Number of pages remaining on the free-list. */
1829 int nPtrMap
; /* Number of pointer-map pages deallocated */
1830 Pgno origSize
; /* Pages in the database file */
1831 Pgno finSize
; /* Pages in the database file after truncation */
1832 int rc
; /* Return code */
1834 int pgsz
= pBt
->pageSize
; /* Page size for this database */
1835 Pgno iDbPage
; /* The database page to move */
1836 MemPage
*pDbMemPage
= 0; /* "" */
1837 Pgno iPtrPage
; /* The page that contains a pointer to iDbPage */
1838 Pgno iFreePage
; /* The free-list page to move iDbPage to */
1839 MemPage
*pFreeMemPage
= 0; /* "" */
1842 int nRef
= *sqlite3pager_stats(pPager
);
1845 assert( pBt
->autoVacuum
);
1846 if( PTRMAP_ISPAGE(pgsz
, sqlite3pager_pagecount(pPager
)) ){
1847 return SQLITE_CORRUPT
;
1850 /* Figure out how many free-pages are in the database. If there are no
1851 ** free pages, then auto-vacuum is a no-op.
1853 nFreeList
= get4byte(&pBt
->pPage1
->aData
[36]);
1859 origSize
= sqlite3pager_pagecount(pPager
);
1860 nPtrMap
= (nFreeList
-origSize
+PTRMAP_PAGENO(pgsz
, origSize
)+pgsz
/5)/(pgsz
/5);
1861 finSize
= origSize
- nFreeList
- nPtrMap
;
1862 if( origSize
>PENDING_BYTE_PAGE(pBt
) && finSize
<=PENDING_BYTE_PAGE(pBt
) ){
1864 if( PTRMAP_ISPAGE(pBt
->usableSize
, finSize
) ){
1868 TRACE(("AUTOVACUUM: Begin (db size %d->%d)\n", origSize
, finSize
));
1870 /* Variable 'finSize' will be the size of the file in pages after
1871 ** the auto-vacuum has completed (the current file size minus the number
1872 ** of pages on the free list). Loop through the pages that lie beyond
1873 ** this mark, and if they are not already on the free list, move them
1874 ** to a free page earlier in the file (somewhere before finSize).
1876 for( iDbPage
=finSize
+1; iDbPage
<=origSize
; iDbPage
++ ){
1877 /* If iDbPage is a pointer map page, or the pending-byte page, skip it. */
1878 if( PTRMAP_ISPAGE(pgsz
, iDbPage
) || iDbPage
==PENDING_BYTE_PAGE(pBt
) ){
1882 rc
= ptrmapGet(pBt
, iDbPage
, &eType
, &iPtrPage
);
1883 if( rc
!=SQLITE_OK
) goto autovacuum_out
;
1884 if( eType
==PTRMAP_ROOTPAGE
){
1885 rc
= SQLITE_CORRUPT
;
1886 goto autovacuum_out
;
1889 /* If iDbPage is free, do not swap it. */
1890 if( eType
==PTRMAP_FREEPAGE
){
1893 rc
= getPage(pBt
, iDbPage
, &pDbMemPage
);
1894 if( rc
!=SQLITE_OK
) goto autovacuum_out
;
1896 /* Find the next page in the free-list that is not already at the end
1897 ** of the file. A page can be pulled off the free list using the
1898 ** allocatePage() routine.
1902 releasePage(pFreeMemPage
);
1905 rc
= allocatePage(pBt
, &pFreeMemPage
, &iFreePage
, 0, 0);
1906 if( rc
!=SQLITE_OK
){
1907 releasePage(pDbMemPage
);
1908 goto autovacuum_out
;
1910 assert( iFreePage
<=origSize
);
1911 }while( iFreePage
>finSize
);
1912 releasePage(pFreeMemPage
);
1915 rc
= relocatePage(pBt
, pDbMemPage
, eType
, iPtrPage
, iFreePage
);
1916 releasePage(pDbMemPage
);
1917 if( rc
!=SQLITE_OK
) goto autovacuum_out
;
1920 /* The entire free-list has been swapped to the end of the file. So
1921 ** truncate the database file to finSize pages and consider the
1924 rc
= sqlite3pager_write(pBt
->pPage1
->aData
);
1925 if( rc
!=SQLITE_OK
) goto autovacuum_out
;
1926 put4byte(&pBt
->pPage1
->aData
[32], 0);
1927 put4byte(&pBt
->pPage1
->aData
[36], 0);
1928 if( rc
!=SQLITE_OK
) goto autovacuum_out
;
1932 assert( nRef
==*sqlite3pager_stats(pPager
) );
1933 if( rc
!=SQLITE_OK
){
1934 sqlite3pager_rollback(pPager
);
1941 ** Commit the transaction currently in progress.
1943 ** This will release the write lock on the database file. If there
1944 ** are no active cursors, it also releases the read lock.
1946 int sqlite3BtreeCommit(Btree
*pBt
){
1948 if( pBt
->inTrans
==TRANS_WRITE
){
1949 rc
= sqlite3pager_commit(pBt
->pPager
);
1951 pBt
->inTrans
= TRANS_NONE
;
1953 unlockBtreeIfUnused(pBt
);
1959 ** Return the number of write-cursors open on this handle. This is for use
1960 ** in assert() expressions, so it is only compiled if NDEBUG is not
1963 static int countWriteCursors(Btree
*pBt
){
1966 for(pCur
=pBt
->pCursor
; pCur
; pCur
=pCur
->pNext
){
1967 if( pCur
->wrFlag
) r
++;
1975 ** Print debugging information about all cursors to standard output.
1977 void sqlite3BtreeCursorList(Btree
*pBt
){
1979 for(pCur
=pBt
->pCursor
; pCur
; pCur
=pCur
->pNext
){
1980 MemPage
*pPage
= pCur
->pPage
;
1981 char *zMode
= pCur
->wrFlag
? "rw" : "ro";
1982 sqlite3DebugPrintf("CURSOR %p rooted at %4d(%s) currently at %d.%d%s\n",
1983 pCur
, pCur
->pgnoRoot
, zMode
,
1984 pPage
? pPage
->pgno
: 0, pCur
->idx
,
1985 pCur
->isValid
? "" : " eof"
1992 ** Rollback the transaction in progress. All cursors will be
1993 ** invalided by this operation. Any attempt to use a cursor
1994 ** that was open at the beginning of this operation will result
1997 ** This will release the write lock on the database file. If there
1998 ** are no active cursors, it also releases the read lock.
2000 int sqlite3BtreeRollback(Btree
*pBt
){
2003 if( pBt
->inTrans
==TRANS_WRITE
){
2004 rc
= sqlite3pager_rollback(pBt
->pPager
);
2005 /* The rollback may have destroyed the pPage1->aData value. So
2006 ** call getPage() on page 1 again to make sure pPage1->aData is
2007 ** set correctly. */
2008 if( getPage(pBt
, 1, &pPage1
)==SQLITE_OK
){
2009 releasePage(pPage1
);
2011 assert( countWriteCursors(pBt
)==0 );
2013 pBt
->inTrans
= TRANS_NONE
;
2015 unlockBtreeIfUnused(pBt
);
2020 ** Start a statement subtransaction. The subtransaction can
2021 ** can be rolled back independently of the main transaction.
2022 ** You must start a transaction before starting a subtransaction.
2023 ** The subtransaction is ended automatically if the main transaction
2024 ** commits or rolls back.
2026 ** Only one subtransaction may be active at a time. It is an error to try
2027 ** to start a new subtransaction if another subtransaction is already active.
2029 ** Statement subtransactions are used around individual SQL statements
2030 ** that are contained within a BEGIN...COMMIT block. If a constraint
2031 ** error occurs within the statement, the effect of that one statement
2032 ** can be rolled back without having to rollback the entire transaction.
2034 int sqlite3BtreeBeginStmt(Btree
*pBt
){
2036 if( (pBt
->inTrans
!=TRANS_WRITE
) || pBt
->inStmt
){
2037 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
2039 rc
= pBt
->readOnly
? SQLITE_OK
: sqlite3pager_stmt_begin(pBt
->pPager
);
2046 ** Commit the statment subtransaction currently in progress. If no
2047 ** subtransaction is active, this is a no-op.
2049 int sqlite3BtreeCommitStmt(Btree
*pBt
){
2051 if( pBt
->inStmt
&& !pBt
->readOnly
){
2052 rc
= sqlite3pager_stmt_commit(pBt
->pPager
);
2061 ** Rollback the active statement subtransaction. If no subtransaction
2062 ** is active this routine is a no-op.
2064 ** All cursors will be invalidated by this operation. Any attempt
2065 ** to use a cursor that was open at the beginning of this operation
2066 ** will result in an error.
2068 int sqlite3BtreeRollbackStmt(Btree
*pBt
){
2070 if( pBt
->inStmt
==0 || pBt
->readOnly
) return SQLITE_OK
;
2071 rc
= sqlite3pager_stmt_rollback(pBt
->pPager
);
2072 assert( countWriteCursors(pBt
)==0 );
2078 ** Default key comparison function to be used if no comparison function
2079 ** is specified on the sqlite3BtreeCursor() call.
2081 static int dfltCompare(
2082 void *NotUsed
, /* User data is not used */
2083 int n1
, const void *p1
, /* First key to compare */
2084 int n2
, const void *p2
/* Second key to compare */
2087 c
= memcmp(p1
, p2
, n1
<n2
? n1
: n2
);
2095 ** Create a new cursor for the BTree whose root is on the page
2096 ** iTable. The act of acquiring a cursor gets a read lock on
2097 ** the database file.
2099 ** If wrFlag==0, then the cursor can only be used for reading.
2100 ** If wrFlag==1, then the cursor can be used for reading or for
2101 ** writing if other conditions for writing are also met. These
2102 ** are the conditions that must be met in order for writing to
2105 ** 1: The cursor must have been opened with wrFlag==1
2107 ** 2: No other cursors may be open with wrFlag==0 on the same table
2109 ** 3: The database must be writable (not on read-only media)
2111 ** 4: There must be an active transaction.
2113 ** Condition 2 warrants further discussion. If any cursor is opened
2114 ** on a table with wrFlag==0, that prevents all other cursors from
2115 ** writing to that table. This is a kind of "read-lock". When a cursor
2116 ** is opened with wrFlag==0 it is guaranteed that the table will not
2117 ** change as long as the cursor is open. This allows the cursor to
2118 ** do a sequential scan of the table without having to worry about
2119 ** entries being inserted or deleted during the scan. Cursors should
2120 ** be opened with wrFlag==0 only if this read-lock property is needed.
2121 ** That is to say, cursors should be opened with wrFlag==0 only if they
2122 ** intend to use the sqlite3BtreeNext() system call. All other cursors
2123 ** should be opened with wrFlag==1 even if they never really intend
2126 ** No checking is done to make sure that page iTable really is the
2127 ** root page of a b-tree. If it is not, then the cursor acquired
2128 ** will not work correctly.
2130 ** The comparison function must be logically the same for every cursor
2131 ** on a particular table. Changing the comparison function will result
2132 ** in incorrect operations. If the comparison function is NULL, a
2133 ** default comparison function is used. The comparison function is
2134 ** always ignored for INTKEY tables.
2136 int sqlite3BtreeCursor(
2137 Btree
*pBt
, /* The btree */
2138 int iTable
, /* Root page of table to open */
2139 int wrFlag
, /* 1 to write. 0 read-only */
2140 int (*xCmp
)(void*,int,const void*,int,const void*), /* Key Comparison func */
2141 void *pArg
, /* First arg to xCompare() */
2142 BtCursor
**ppCur
/* Write new cursor here */
2149 if( pBt
->readOnly
){
2150 return SQLITE_READONLY
;
2152 if( checkReadLocks(pBt
, iTable
, 0) ){
2153 return SQLITE_LOCKED
;
2156 if( pBt
->pPage1
==0 ){
2157 rc
= lockBtreeWithRetry(pBt
);
2158 if( rc
!=SQLITE_OK
){
2162 pCur
= sqliteMallocRaw( sizeof(*pCur
) );
2165 goto create_cursor_exception
;
2167 pCur
->pgnoRoot
= (Pgno
)iTable
;
2168 pCur
->pPage
= 0; /* For exit-handler, in case getAndInitPage() fails. */
2169 if( iTable
==1 && sqlite3pager_pagecount(pBt
->pPager
)==0 ){
2171 goto create_cursor_exception
;
2173 rc
= getAndInitPage(pBt
, pCur
->pgnoRoot
, &pCur
->pPage
, 0);
2174 if( rc
!=SQLITE_OK
){
2175 goto create_cursor_exception
;
2177 pCur
->xCompare
= xCmp
? xCmp
: dfltCompare
;
2180 pCur
->wrFlag
= wrFlag
;
2182 memset(&pCur
->info
, 0, sizeof(pCur
->info
));
2183 pCur
->pNext
= pBt
->pCursor
;
2185 pCur
->pNext
->pPrev
= pCur
;
2188 pBt
->pCursor
= pCur
;
2193 create_cursor_exception
:
2195 releasePage(pCur
->pPage
);
2198 unlockBtreeIfUnused(pBt
);
2202 #if 0 /* Not Used */
2204 ** Change the value of the comparison function used by a cursor.
2206 void sqlite3BtreeSetCompare(
2207 BtCursor
*pCur
, /* The cursor to whose comparison function is changed */
2208 int(*xCmp
)(void*,int,const void*,int,const void*), /* New comparison func */
2209 void *pArg
/* First argument to xCmp() */
2211 pCur
->xCompare
= xCmp
? xCmp
: dfltCompare
;
2217 ** Close a cursor. The read lock on the database file is released
2218 ** when the last cursor is closed.
2220 int sqlite3BtreeCloseCursor(BtCursor
*pCur
){
2221 Btree
*pBt
= pCur
->pBt
;
2223 pCur
->pPrev
->pNext
= pCur
->pNext
;
2225 pBt
->pCursor
= pCur
->pNext
;
2228 pCur
->pNext
->pPrev
= pCur
->pPrev
;
2230 releasePage(pCur
->pPage
);
2231 unlockBtreeIfUnused(pBt
);
2237 ** Make a temporary cursor by filling in the fields of pTempCur.
2238 ** The temporary cursor is not on the cursor list for the Btree.
2240 static void getTempCursor(BtCursor
*pCur
, BtCursor
*pTempCur
){
2241 memcpy(pTempCur
, pCur
, sizeof(*pCur
));
2242 pTempCur
->pNext
= 0;
2243 pTempCur
->pPrev
= 0;
2244 if( pTempCur
->pPage
){
2245 sqlite3pager_ref(pTempCur
->pPage
->aData
);
2250 ** Delete a temporary cursor such as was made by the CreateTemporaryCursor()
2253 static void releaseTempCursor(BtCursor
*pCur
){
2255 sqlite3pager_unref(pCur
->pPage
->aData
);
2260 ** Make sure the BtCursor.info field of the given cursor is valid.
2261 ** If it is not already valid, call parseCell() to fill it in.
2263 ** BtCursor.info is a cache of the information in the current cell.
2264 ** Using this cache reduces the number of calls to parseCell().
2266 static void getCellInfo(BtCursor
*pCur
){
2267 if( pCur
->info
.nSize
==0 ){
2268 parseCell(pCur
->pPage
, pCur
->idx
, &pCur
->info
);
2272 memset(&info
, 0, sizeof(info
));
2273 parseCell(pCur
->pPage
, pCur
->idx
, &info
);
2274 assert( memcmp(&info
, &pCur
->info
, sizeof(info
))==0 );
2280 ** Set *pSize to the size of the buffer needed to hold the value of
2281 ** the key for the current entry. If the cursor is not pointing
2282 ** to a valid entry, *pSize is set to 0.
2284 ** For a table with the INTKEY flag set, this routine returns the key
2285 ** itself, not the number of bytes in the key.
2287 int sqlite3BtreeKeySize(BtCursor
*pCur
, i64
*pSize
){
2288 if( !pCur
->isValid
){
2292 *pSize
= pCur
->info
.nKey
;
2298 ** Set *pSize to the number of bytes of data in the entry the
2299 ** cursor currently points to. Always return SQLITE_OK.
2300 ** Failure is not possible. If the cursor is not currently
2301 ** pointing to an entry (which can happen, for example, if
2302 ** the database is empty) then *pSize is set to 0.
2304 int sqlite3BtreeDataSize(BtCursor
*pCur
, u32
*pSize
){
2305 if( !pCur
->isValid
){
2306 /* Not pointing at a valid entry - set *pSize to 0. */
2310 *pSize
= pCur
->info
.nData
;
2316 ** Read payload information from the entry that the pCur cursor is
2317 ** pointing to. Begin reading the payload at "offset" and read
2318 ** a total of "amt" bytes. Put the result in zBuf.
2320 ** This routine does not make a distinction between key and data.
2321 ** It just reads bytes from the payload area. Data might appear
2322 ** on the main page or be scattered out on multiple overflow pages.
2324 static int getPayload(
2325 BtCursor
*pCur
, /* Cursor pointing to entry to read from */
2326 int offset
, /* Begin reading this far into payload */
2327 int amt
, /* Read this many bytes */
2328 unsigned char *pBuf
, /* Write the bytes into this buffer */
2329 int skipKey
/* offset begins at data if this is true */
2331 unsigned char *aPayload
;
2339 assert( pCur
!=0 && pCur
->pPage
!=0 );
2340 assert( pCur
->isValid
);
2342 pPage
= pCur
->pPage
;
2343 pageIntegrity(pPage
);
2344 assert( pCur
->idx
>=0 && pCur
->idx
<pPage
->nCell
);
2346 aPayload
= pCur
->info
.pCell
;
2347 aPayload
+= pCur
->info
.nHeader
;
2348 if( pPage
->intKey
){
2351 nKey
= pCur
->info
.nKey
;
2353 assert( offset
>=0 );
2357 if( offset
+amt
> nKey
+pCur
->info
.nData
){
2358 return SQLITE_ERROR
;
2360 if( offset
<pCur
->info
.nLocal
){
2362 if( a
+offset
>pCur
->info
.nLocal
){
2363 a
= pCur
->info
.nLocal
- offset
;
2365 memcpy(pBuf
, &aPayload
[offset
], a
);
2373 offset
-= pCur
->info
.nLocal
;
2375 ovflSize
= pBt
->usableSize
- 4;
2377 nextPage
= get4byte(&aPayload
[pCur
->info
.nLocal
]);
2378 while( amt
>0 && nextPage
){
2379 rc
= sqlite3pager_get(pBt
->pPager
, nextPage
, (void**)&aPayload
);
2383 nextPage
= get4byte(aPayload
);
2384 if( offset
<ovflSize
){
2386 if( a
+ offset
> ovflSize
){
2387 a
= ovflSize
- offset
;
2389 memcpy(pBuf
, &aPayload
[offset
+4], a
);
2396 sqlite3pager_unref(aPayload
);
2401 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
2407 ** Read part of the key associated with cursor pCur. Exactly
2408 ** "amt" bytes will be transfered into pBuf[]. The transfer
2409 ** begins at "offset".
2411 ** Return SQLITE_OK on success or an error code if anything goes
2412 ** wrong. An error is returned if "offset+amt" is larger than
2413 ** the available payload.
2415 int sqlite3BtreeKey(BtCursor
*pCur
, u32 offset
, u32 amt
, void *pBuf
){
2416 assert( pCur
->isValid
);
2417 assert( pCur
->pPage
!=0 );
2418 if( pCur
->pPage
->intKey
){
2419 return SQLITE_CORRUPT
;
2421 assert( pCur
->pPage
->intKey
==0 );
2422 assert( pCur
->idx
>=0 && pCur
->idx
<pCur
->pPage
->nCell
);
2423 return getPayload(pCur
, offset
, amt
, (unsigned char*)pBuf
, 0);
2427 ** Read part of the data associated with cursor pCur. Exactly
2428 ** "amt" bytes will be transfered into pBuf[]. The transfer
2429 ** begins at "offset".
2431 ** Return SQLITE_OK on success or an error code if anything goes
2432 ** wrong. An error is returned if "offset+amt" is larger than
2433 ** the available payload.
2435 int sqlite3BtreeData(BtCursor
*pCur
, u32 offset
, u32 amt
, void *pBuf
){
2436 assert( pCur
->isValid
);
2437 assert( pCur
->pPage
!=0 );
2438 assert( pCur
->idx
>=0 && pCur
->idx
<pCur
->pPage
->nCell
);
2439 return getPayload(pCur
, offset
, amt
, pBuf
, 1);
2443 ** Return a pointer to payload information from the entry that the
2444 ** pCur cursor is pointing to. The pointer is to the beginning of
2445 ** the key if skipKey==0 and it points to the beginning of data if
2446 ** skipKey==1. The number of bytes of available key/data is written
2447 ** into *pAmt. If *pAmt==0, then the value returned will not be
2450 ** This routine is an optimization. It is common for the entire key
2451 ** and data to fit on the local page and for there to be no overflow
2452 ** pages. When that is so, this routine can be used to access the
2453 ** key and data without making a copy. If the key and/or data spills
2454 ** onto overflow pages, then getPayload() must be used to reassembly
2455 ** the key/data and copy it into a preallocated buffer.
2457 ** The pointer returned by this routine looks directly into the cached
2458 ** page of the database. The data might change or move the next time
2459 ** any btree routine is called.
2461 static const unsigned char *fetchPayload(
2462 BtCursor
*pCur
, /* Cursor pointing to entry to read from */
2463 int *pAmt
, /* Write the number of available bytes here */
2464 int skipKey
/* read beginning at data if this is true */
2466 unsigned char *aPayload
;
2472 assert( pCur
!=0 && pCur
->pPage
!=0 );
2473 assert( pCur
->isValid
);
2475 pPage
= pCur
->pPage
;
2476 pageIntegrity(pPage
);
2477 assert( pCur
->idx
>=0 && pCur
->idx
<pPage
->nCell
);
2479 aPayload
= pCur
->info
.pCell
;
2480 aPayload
+= pCur
->info
.nHeader
;
2481 if( pPage
->intKey
){
2484 nKey
= pCur
->info
.nKey
;
2488 nLocal
= pCur
->info
.nLocal
- nKey
;
2490 nLocal
= pCur
->info
.nLocal
;
2501 ** For the entry that cursor pCur is point to, return as
2502 ** many bytes of the key or data as are available on the local
2503 ** b-tree page. Write the number of available bytes into *pAmt.
2505 ** The pointer returned is ephemeral. The key/data may move
2506 ** or be destroyed on the next call to any Btree routine.
2508 ** These routines is used to get quick access to key and data
2509 ** in the common case where no overflow pages are used.
2511 const void *sqlite3BtreeKeyFetch(BtCursor
*pCur
, int *pAmt
){
2512 return (const void*)fetchPayload(pCur
, pAmt
, 0);
2514 const void *sqlite3BtreeDataFetch(BtCursor
*pCur
, int *pAmt
){
2515 return (const void*)fetchPayload(pCur
, pAmt
, 1);
2520 ** Move the cursor down to a new child page. The newPgno argument is the
2521 ** page number of the child page to move to.
2523 static int moveToChild(BtCursor
*pCur
, u32 newPgno
){
2527 Btree
*pBt
= pCur
->pBt
;
2529 assert( pCur
->isValid
);
2530 rc
= getAndInitPage(pBt
, newPgno
, &pNewPage
, pCur
->pPage
);
2532 pageIntegrity(pNewPage
);
2533 pNewPage
->idxParent
= pCur
->idx
;
2534 pOldPage
= pCur
->pPage
;
2535 pOldPage
->idxShift
= 0;
2536 releasePage(pOldPage
);
2537 pCur
->pPage
= pNewPage
;
2539 pCur
->info
.nSize
= 0;
2540 if( pNewPage
->nCell
<1 ){
2541 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
2547 ** Return true if the page is the virtual root of its table.
2549 ** The virtual root page is the root page for most tables. But
2550 ** for the table rooted on page 1, sometime the real root page
2551 ** is empty except for the right-pointer. In such cases the
2552 ** virtual root page is the page that the right-pointer of page
2553 ** 1 is pointing to.
2555 static int isRootPage(MemPage
*pPage
){
2556 MemPage
*pParent
= pPage
->pParent
;
2557 if( pParent
==0 ) return 1;
2558 if( pParent
->pgno
>1 ) return 0;
2559 if( get2byte(&pParent
->aData
[pParent
->hdrOffset
+3])==0 ) return 1;
2564 ** Move the cursor up to the parent page.
2566 ** pCur->idx is set to the cell index that contains the pointer
2567 ** to the page we are coming from. If we are coming from the
2568 ** right-most child page then pCur->idx is set to one more than
2569 ** the largest cell index.
2571 static void moveToParent(BtCursor
*pCur
){
2577 assert( pCur
->isValid
);
2578 pPage
= pCur
->pPage
;
2580 assert( !isRootPage(pPage
) );
2581 pageIntegrity(pPage
);
2582 pParent
= pPage
->pParent
;
2583 assert( pParent
!=0 );
2584 pageIntegrity(pParent
);
2585 idxParent
= pPage
->idxParent
;
2586 sqlite3pager_ref(pParent
->aData
);
2587 oldPgno
= pPage
->pgno
;
2589 pCur
->pPage
= pParent
;
2590 pCur
->info
.nSize
= 0;
2591 assert( pParent
->idxShift
==0 );
2592 pCur
->idx
= idxParent
;
2596 ** Move the cursor to the root page
2598 static int moveToRoot(BtCursor
*pCur
){
2601 Btree
*pBt
= pCur
->pBt
;
2603 rc
= getAndInitPage(pBt
, pCur
->pgnoRoot
, &pRoot
, 0);
2608 releasePage(pCur
->pPage
);
2609 pageIntegrity(pRoot
);
2610 pCur
->pPage
= pRoot
;
2612 pCur
->info
.nSize
= 0;
2613 if( pRoot
->nCell
==0 && !pRoot
->leaf
){
2615 assert( pRoot
->pgno
==1 );
2616 subpage
= get4byte(&pRoot
->aData
[pRoot
->hdrOffset
+8]);
2617 assert( subpage
>0 );
2619 rc
= moveToChild(pCur
, subpage
);
2621 pCur
->isValid
= pCur
->pPage
->nCell
>0;
2626 ** Move the cursor down to the left-most leaf entry beneath the
2627 ** entry to which it is currently pointing.
2629 static int moveToLeftmost(BtCursor
*pCur
){
2634 assert( pCur
->isValid
);
2635 while( !(pPage
= pCur
->pPage
)->leaf
){
2636 assert( pCur
->idx
>=0 && pCur
->idx
<pPage
->nCell
);
2637 pgno
= get4byte(findCell(pPage
, pCur
->idx
));
2638 rc
= moveToChild(pCur
, pgno
);
2645 ** Move the cursor down to the right-most leaf entry beneath the
2646 ** page to which it is currently pointing. Notice the difference
2647 ** between moveToLeftmost() and moveToRightmost(). moveToLeftmost()
2648 ** finds the left-most entry beneath the *entry* whereas moveToRightmost()
2649 ** finds the right-most entry beneath the *page*.
2651 static int moveToRightmost(BtCursor
*pCur
){
2656 assert( pCur
->isValid
);
2657 while( !(pPage
= pCur
->pPage
)->leaf
){
2658 pgno
= get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]);
2659 pCur
->idx
= pPage
->nCell
;
2660 rc
= moveToChild(pCur
, pgno
);
2663 pCur
->idx
= pPage
->nCell
- 1;
2664 pCur
->info
.nSize
= 0;
2668 /* Move the cursor to the first entry in the table. Return SQLITE_OK
2669 ** on success. Set *pRes to 0 if the cursor actually points to something
2670 ** or set *pRes to 1 if the table is empty.
2672 int sqlite3BtreeFirst(BtCursor
*pCur
, int *pRes
){
2674 rc
= moveToRoot(pCur
);
2676 if( pCur
->isValid
==0 ){
2677 assert( pCur
->pPage
->nCell
==0 );
2681 assert( pCur
->pPage
->nCell
>0 );
2683 rc
= moveToLeftmost(pCur
);
2687 /* Move the cursor to the last entry in the table. Return SQLITE_OK
2688 ** on success. Set *pRes to 0 if the cursor actually points to something
2689 ** or set *pRes to 1 if the table is empty.
2691 int sqlite3BtreeLast(BtCursor
*pCur
, int *pRes
){
2693 rc
= moveToRoot(pCur
);
2695 if( pCur
->isValid
==0 ){
2696 assert( pCur
->pPage
->nCell
==0 );
2700 assert( pCur
->isValid
);
2702 rc
= moveToRightmost(pCur
);
2706 /* Move the cursor so that it points to an entry near pKey/nKey.
2707 ** Return a success code.
2709 ** For INTKEY tables, only the nKey parameter is used. pKey is
2710 ** ignored. For other tables, nKey is the number of bytes of data
2711 ** in nKey. The comparison function specified when the cursor was
2712 ** created is used to compare keys.
2714 ** If an exact match is not found, then the cursor is always
2715 ** left pointing at a leaf page which would hold the entry if it
2716 ** were present. The cursor might point to an entry that comes
2717 ** before or after the key.
2719 ** The result of comparing the key with the entry to which the
2720 ** cursor is written to *pRes if pRes!=NULL. The meaning of
2721 ** this value is as follows:
2723 ** *pRes<0 The cursor is left pointing at an entry that
2724 ** is smaller than pKey or if the table is empty
2725 ** and the cursor is therefore left point to nothing.
2727 ** *pRes==0 The cursor is left pointing at an entry that
2728 ** exactly matches pKey.
2730 ** *pRes>0 The cursor is left pointing at an entry that
2731 ** is larger than pKey.
2733 int sqlite3BtreeMoveto(BtCursor
*pCur
, const void *pKey
, i64 nKey
, int *pRes
){
2735 rc
= moveToRoot(pCur
);
2737 assert( pCur
->pPage
);
2738 assert( pCur
->pPage
->isInit
);
2739 if( pCur
->isValid
==0 ){
2741 assert( pCur
->pPage
->nCell
==0 );
2747 MemPage
*pPage
= pCur
->pPage
;
2748 int c
= -1; /* pRes return if table is empty must be -1 */
2750 upr
= pPage
->nCell
-1;
2751 if( !pPage
->intKey
&& pKey
==0 ){
2752 return SQLITE_CORRUPT
;
2754 pageIntegrity(pPage
);
2758 pCur
->idx
= (lwr
+upr
)/2;
2759 pCur
->info
.nSize
= 0;
2760 sqlite3BtreeKeySize(pCur
, &nCellKey
);
2761 if( pPage
->intKey
){
2762 if( nCellKey
<nKey
){
2764 }else if( nCellKey
>nKey
){
2771 pCellKey
= (void *)fetchPayload(pCur
, &available
, 0);
2772 if( available
>=nCellKey
){
2773 c
= pCur
->xCompare(pCur
->pArg
, nCellKey
, pCellKey
, nKey
, pKey
);
2775 pCellKey
= sqliteMallocRaw( nCellKey
);
2776 if( pCellKey
==0 ) return SQLITE_NOMEM
;
2777 rc
= sqlite3BtreeKey(pCur
, 0, nCellKey
, (void *)pCellKey
);
2778 c
= pCur
->xCompare(pCur
->pArg
, nCellKey
, pCellKey
, nKey
, pKey
);
2779 sqliteFree(pCellKey
);
2784 if( pPage
->leafData
&& !pPage
->leaf
){
2789 if( pRes
) *pRes
= 0;
2799 assert( lwr
==upr
+1 );
2800 assert( pPage
->isInit
);
2803 }else if( lwr
>=pPage
->nCell
){
2804 chldPg
= get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]);
2806 chldPg
= get4byte(findCell(pPage
, lwr
));
2809 assert( pCur
->idx
>=0 && pCur
->idx
<pCur
->pPage
->nCell
);
2810 if( pRes
) *pRes
= c
;
2814 pCur
->info
.nSize
= 0;
2815 rc
= moveToChild(pCur
, chldPg
);
2824 ** Return TRUE if the cursor is not pointing at an entry of the table.
2826 ** TRUE will be returned after a call to sqlite3BtreeNext() moves
2827 ** past the last entry in the table or sqlite3BtreePrev() moves past
2828 ** the first entry. TRUE is also returned if the table is empty.
2830 int sqlite3BtreeEof(BtCursor
*pCur
){
2831 return pCur
->isValid
==0;
2835 ** Advance the cursor to the next entry in the database. If
2836 ** successful then set *pRes=0. If the cursor
2837 ** was already pointing to the last entry in the database before
2838 ** this routine was called, then set *pRes=1.
2840 int sqlite3BtreeNext(BtCursor
*pCur
, int *pRes
){
2842 MemPage
*pPage
= pCur
->pPage
;
2845 if( pCur
->isValid
==0 ){
2849 assert( pPage
->isInit
);
2850 assert( pCur
->idx
<pPage
->nCell
);
2853 pCur
->info
.nSize
= 0;
2854 if( pCur
->idx
>=pPage
->nCell
){
2856 rc
= moveToChild(pCur
, get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]));
2858 rc
= moveToLeftmost(pCur
);
2863 if( isRootPage(pPage
) ){
2869 pPage
= pCur
->pPage
;
2870 }while( pCur
->idx
>=pPage
->nCell
);
2872 if( pPage
->leafData
){
2873 rc
= sqlite3BtreeNext(pCur
, pRes
);
2883 rc
= moveToLeftmost(pCur
);
2888 ** Step the cursor to the back to the previous entry in the database. If
2889 ** successful then set *pRes=0. If the cursor
2890 ** was already pointing to the first entry in the database before
2891 ** this routine was called, then set *pRes=1.
2893 int sqlite3BtreePrevious(BtCursor
*pCur
, int *pRes
){
2897 if( pCur
->isValid
==0 ){
2902 pPage
= pCur
->pPage
;
2903 assert( pPage
->isInit
);
2904 assert( pCur
->idx
>=0 );
2906 pgno
= get4byte( findCell(pPage
, pCur
->idx
) );
2907 rc
= moveToChild(pCur
, pgno
);
2909 rc
= moveToRightmost(pCur
);
2911 while( pCur
->idx
==0 ){
2912 if( isRootPage(pPage
) ){
2918 pPage
= pCur
->pPage
;
2921 pCur
->info
.nSize
= 0;
2922 if( pPage
->leafData
&& !pPage
->leaf
){
2923 rc
= sqlite3BtreePrevious(pCur
, pRes
);
2933 ** Allocate a new page from the database file.
2935 ** The new page is marked as dirty. (In other words, sqlite3pager_write()
2936 ** has already been called on the new page.) The new page has also
2937 ** been referenced and the calling routine is responsible for calling
2938 ** sqlite3pager_unref() on the new page when it is done.
2940 ** SQLITE_OK is returned on success. Any other return value indicates
2941 ** an error. *ppPage and *pPgno are undefined in the event of an error.
2942 ** Do not invoke sqlite3pager_unref() on *ppPage if an error is returned.
2944 ** If the "nearby" parameter is not 0, then a (feeble) effort is made to
2945 ** locate a page close to the page number "nearby". This can be used in an
2946 ** attempt to keep related pages close to each other in the database file,
2947 ** which in turn can make database access faster.
2949 ** If the "exact" parameter is not 0, and the page-number nearby exists
2950 ** anywhere on the free-list, then it is guarenteed to be returned. This
2951 ** is only used by auto-vacuum databases when allocating a new table.
2953 static int allocatePage(
2962 int n
; /* Number of pages on the freelist */
2963 int k
; /* Number of leaves on the trunk of the freelist */
2965 pPage1
= pBt
->pPage1
;
2966 n
= get4byte(&pPage1
->aData
[36]);
2968 /* There are pages on the freelist. Reuse one of those pages. */
2969 MemPage
*pTrunk
= 0;
2971 MemPage
*pPrevTrunk
= 0;
2972 u8 searchList
= 0; /* If the free-list must be searched for 'nearby' */
2974 /* If the 'exact' parameter was true and a query of the pointer-map
2975 ** shows that the page 'nearby' is somewhere on the free-list, then
2976 ** the entire-list will be searched for that page.
2978 #ifndef SQLITE_OMIT_AUTOVACUUM
2982 assert( pBt
->autoVacuum
);
2983 rc
= ptrmapGet(pBt
, nearby
, &eType
, 0);
2985 if( eType
==PTRMAP_FREEPAGE
){
2992 /* Decrement the free-list count by 1. Set iTrunk to the index of the
2993 ** first free-list trunk page. iPrevTrunk is initially 1.
2995 rc
= sqlite3pager_write(pPage1
->aData
);
2997 put4byte(&pPage1
->aData
[36], n
-1);
2999 /* The code within this loop is run only once if the 'searchList' variable
3000 ** is not true. Otherwise, it runs once for each trunk-page on the
3001 ** free-list until the page 'nearby' is located.
3004 pPrevTrunk
= pTrunk
;
3006 iTrunk
= get4byte(&pPrevTrunk
->aData
[0]);
3008 iTrunk
= get4byte(&pPage1
->aData
[32]);
3010 rc
= getPage(pBt
, iTrunk
, &pTrunk
);
3012 releasePage(pPrevTrunk
);
3016 /* TODO: This should move to after the loop? */
3017 rc
= sqlite3pager_write(pTrunk
->aData
);
3019 releasePage(pTrunk
);
3020 releasePage(pPrevTrunk
);
3024 k
= get4byte(&pTrunk
->aData
[4]);
3025 if( k
==0 && !searchList
){
3026 /* The trunk has no leaves and the list is not being searched.
3027 ** So extract the trunk page itself and use it as the newly
3028 ** allocated page */
3029 assert( pPrevTrunk
==0 );
3031 memcpy(&pPage1
->aData
[32], &pTrunk
->aData
[0], 4);
3034 TRACE(("ALLOCATE: %d trunk - %d free pages left\n", *pPgno
, n
-1));
3035 }else if( k
>pBt
->usableSize
/4 - 8 ){
3036 /* Value of k is out of range. Database corruption */
3037 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
3038 #ifndef SQLITE_OMIT_AUTOVACUUM
3039 }else if( searchList
&& nearby
==iTrunk
){
3040 /* The list is being searched and this trunk page is the page
3041 ** to allocate, regardless of whether it has leaves.
3043 assert( *pPgno
==iTrunk
);
3048 memcpy(&pPage1
->aData
[32], &pTrunk
->aData
[0], 4);
3050 memcpy(&pPrevTrunk
->aData
[0], &pTrunk
->aData
[0], 4);
3053 /* The trunk page is required by the caller but it contains
3054 ** pointers to free-list leaves. The first leaf becomes a trunk
3055 ** page in this case.
3058 Pgno iNewTrunk
= get4byte(&pTrunk
->aData
[8]);
3059 rc
= getPage(pBt
, iNewTrunk
, &pNewTrunk
);
3060 if( rc
!=SQLITE_OK
){
3061 releasePage(pTrunk
);
3062 releasePage(pPrevTrunk
);
3065 rc
= sqlite3pager_write(pNewTrunk
->aData
);
3066 if( rc
!=SQLITE_OK
){
3067 releasePage(pNewTrunk
);
3068 releasePage(pTrunk
);
3069 releasePage(pPrevTrunk
);
3072 memcpy(&pNewTrunk
->aData
[0], &pTrunk
->aData
[0], 4);
3073 put4byte(&pNewTrunk
->aData
[4], k
-1);
3074 memcpy(&pNewTrunk
->aData
[8], &pTrunk
->aData
[12], (k
-1)*4);
3076 put4byte(&pPage1
->aData
[32], iNewTrunk
);
3078 put4byte(&pPrevTrunk
->aData
[0], iNewTrunk
);
3080 releasePage(pNewTrunk
);
3083 TRACE(("ALLOCATE: %d trunk - %d free pages left\n", *pPgno
, n
-1));
3086 /* Extract a leaf from the trunk */
3089 unsigned char *aData
= pTrunk
->aData
;
3093 dist
= get4byte(&aData
[8]) - nearby
;
3094 if( dist
<0 ) dist
= -dist
;
3096 int d2
= get4byte(&aData
[8+i
*4]) - nearby
;
3097 if( d2
<0 ) d2
= -d2
;
3107 iPage
= get4byte(&aData
[8+closest
*4]);
3108 if( !searchList
|| iPage
==nearby
){
3110 if( *pPgno
>sqlite3pager_pagecount(pBt
->pPager
) ){
3111 /* Free page off the end of the file */
3112 return SQLITE_CORRUPT
; /* bkpt-CORRUPT */
3114 TRACE(("ALLOCATE: %d was leaf %d of %d on trunk %d"
3115 ": %d more free pages\n",
3116 *pPgno
, closest
+1, k
, pTrunk
->pgno
, n
-1));
3118 memcpy(&aData
[8+closest
*4], &aData
[4+k
*4], 4);
3120 put4byte(&aData
[4], k
-1);
3121 rc
= getPage(pBt
, *pPgno
, ppPage
);
3122 if( rc
==SQLITE_OK
){
3123 sqlite3pager_dont_rollback((*ppPage
)->aData
);
3124 rc
= sqlite3pager_write((*ppPage
)->aData
);
3125 if( rc
!=SQLITE_OK
){
3126 releasePage(*ppPage
);
3132 releasePage(pPrevTrunk
);
3133 }while( searchList
);
3134 releasePage(pTrunk
);
3136 /* There are no pages on the freelist, so create a new page at the
3137 ** end of the file */
3138 *pPgno
= sqlite3pager_pagecount(pBt
->pPager
) + 1;
3140 #ifndef SQLITE_OMIT_AUTOVACUUM
3141 if( pBt
->autoVacuum
&& PTRMAP_ISPAGE(pBt
->usableSize
, *pPgno
) ){
3142 /* If *pPgno refers to a pointer-map page, allocate two new pages
3143 ** at the end of the file instead of one. The first allocated page
3144 ** becomes a new pointer-map page, the second is used by the caller.
3146 TRACE(("ALLOCATE: %d from end of file (pointer-map page)\n", *pPgno
));
3147 assert( *pPgno
!=PENDING_BYTE_PAGE(pBt
) );
3152 assert( *pPgno
!=PENDING_BYTE_PAGE(pBt
) );
3153 rc
= getPage(pBt
, *pPgno
, ppPage
);
3155 rc
= sqlite3pager_write((*ppPage
)->aData
);
3156 if( rc
!=SQLITE_OK
){
3157 releasePage(*ppPage
);
3159 TRACE(("ALLOCATE: %d from end of file\n", *pPgno
));
3162 assert( *pPgno
!=PENDING_BYTE_PAGE(pBt
) );
3167 ** Add a page of the database file to the freelist.
3169 ** sqlite3pager_unref() is NOT called for pPage.
3171 static int freePage(MemPage
*pPage
){
3172 Btree
*pBt
= pPage
->pBt
;
3173 MemPage
*pPage1
= pBt
->pPage1
;
3176 /* Prepare the page for freeing */
3177 assert( pPage
->pgno
>1 );
3179 releasePage(pPage
->pParent
);
3182 /* Increment the free page count on pPage1 */
3183 rc
= sqlite3pager_write(pPage1
->aData
);
3185 n
= get4byte(&pPage1
->aData
[36]);
3186 put4byte(&pPage1
->aData
[36], n
+1);
3188 #ifndef SQLITE_OMIT_AUTOVACUUM
3189 /* If the database supports auto-vacuum, write an entry in the pointer-map
3190 ** to indicate that the page is free.
3192 if( pBt
->autoVacuum
){
3193 rc
= ptrmapPut(pBt
, pPage
->pgno
, PTRMAP_FREEPAGE
, 0);
3199 /* This is the first free page */
3200 rc
= sqlite3pager_write(pPage
->aData
);
3202 memset(pPage
->aData
, 0, 8);
3203 put4byte(&pPage1
->aData
[32], pPage
->pgno
);
3204 TRACE(("FREE-PAGE: %d first\n", pPage
->pgno
));
3206 /* Other free pages already exist. Retrive the first trunk page
3207 ** of the freelist and find out how many leaves it has. */
3209 rc
= getPage(pBt
, get4byte(&pPage1
->aData
[32]), &pTrunk
);
3211 k
= get4byte(&pTrunk
->aData
[4]);
3212 if( k
>=pBt
->usableSize
/4 - 8 ){
3213 /* The trunk is full. Turn the page being freed into a new
3214 ** trunk page with no leaves. */
3215 rc
= sqlite3pager_write(pPage
->aData
);
3217 put4byte(pPage
->aData
, pTrunk
->pgno
);
3218 put4byte(&pPage
->aData
[4], 0);
3219 put4byte(&pPage1
->aData
[32], pPage
->pgno
);
3220 TRACE(("FREE-PAGE: %d new trunk page replacing %d\n",
3221 pPage
->pgno
, pTrunk
->pgno
));
3223 /* Add the newly freed page as a leaf on the current trunk */
3224 rc
= sqlite3pager_write(pTrunk
->aData
);
3226 put4byte(&pTrunk
->aData
[4], k
+1);
3227 put4byte(&pTrunk
->aData
[8+k
*4], pPage
->pgno
);
3228 sqlite3pager_dont_write(pBt
->pPager
, pPage
->pgno
);
3229 TRACE(("FREE-PAGE: %d leaf on trunk page %d\n",pPage
->pgno
,pTrunk
->pgno
));
3231 releasePage(pTrunk
);
3237 ** Free any overflow pages associated with the given Cell.
3239 static int clearCell(MemPage
*pPage
, unsigned char *pCell
){
3240 Btree
*pBt
= pPage
->pBt
;
3245 parseCellPtr(pPage
, pCell
, &info
);
3246 if( info
.iOverflow
==0 ){
3247 return SQLITE_OK
; /* No overflow pages. Return without doing anything */
3249 ovflPgno
= get4byte(&pCell
[info
.iOverflow
]);
3250 while( ovflPgno
!=0 ){
3252 if( ovflPgno
>sqlite3pager_pagecount(pBt
->pPager
) ){
3253 return SQLITE_CORRUPT
;
3255 rc
= getPage(pBt
, ovflPgno
, &pOvfl
);
3257 ovflPgno
= get4byte(pOvfl
->aData
);
3258 rc
= freePage(pOvfl
);
3259 sqlite3pager_unref(pOvfl
->aData
);
3266 ** Create the byte sequence used to represent a cell on page pPage
3267 ** and write that byte sequence into pCell[]. Overflow pages are
3268 ** allocated and filled in as necessary. The calling procedure
3269 ** is responsible for making sure sufficient space has been allocated
3272 ** Note that pCell does not necessary need to point to the pPage->aData
3273 ** area. pCell might point to some temporary storage. The cell will
3274 ** be constructed in this temporary area then copied into pPage->aData
3277 static int fillInCell(
3278 MemPage
*pPage
, /* The page that contains the cell */
3279 unsigned char *pCell
, /* Complete text of the cell */
3280 const void *pKey
, i64 nKey
, /* The key */
3281 const void *pData
,int nData
, /* The data */
3282 int *pnSize
/* Write cell size here */
3289 MemPage
*pToRelease
= 0;
3290 unsigned char *pPrior
;
3291 unsigned char *pPayload
;
3292 Btree
*pBt
= pPage
->pBt
;
3297 /* Fill in the header. */
3302 if( pPage
->hasData
){
3303 nHeader
+= putVarint(&pCell
[nHeader
], nData
);
3307 nHeader
+= putVarint(&pCell
[nHeader
], *(u64
*)&nKey
);
3308 parseCellPtr(pPage
, pCell
, &info
);
3309 assert( info
.nHeader
==nHeader
);
3310 assert( info
.nKey
==nKey
);
3311 assert( info
.nData
==nData
);
3313 /* Fill in the payload */
3315 if( pPage
->intKey
){
3324 *pnSize
= info
.nSize
;
3325 spaceLeft
= info
.nLocal
;
3326 pPayload
= &pCell
[nHeader
];
3327 pPrior
= &pCell
[info
.iOverflow
];
3329 while( nPayload
>0 ){
3331 #ifndef SQLITE_OMIT_AUTOVACUUM
3332 Pgno pgnoPtrmap
= pgnoOvfl
; /* Overflow page pointer-map entry page */
3334 rc
= allocatePage(pBt
, &pOvfl
, &pgnoOvfl
, pgnoOvfl
, 0);
3335 #ifndef SQLITE_OMIT_AUTOVACUUM
3336 /* If the database supports auto-vacuum, and the second or subsequent
3337 ** overflow page is being allocated, add an entry to the pointer-map
3338 ** for that page now. The entry for the first overflow page will be
3339 ** added later, by the insertCell() routine.
3341 if( pBt
->autoVacuum
&& pgnoPtrmap
!=0 && rc
==SQLITE_OK
){
3342 rc
= ptrmapPut(pBt
, pgnoOvfl
, PTRMAP_OVERFLOW2
, pgnoPtrmap
);
3346 releasePage(pToRelease
);
3347 /* clearCell(pPage, pCell); */
3350 put4byte(pPrior
, pgnoOvfl
);
3351 releasePage(pToRelease
);
3353 pPrior
= pOvfl
->aData
;
3354 put4byte(pPrior
, 0);
3355 pPayload
= &pOvfl
->aData
[4];
3356 spaceLeft
= pBt
->usableSize
- 4;
3359 if( n
>spaceLeft
) n
= spaceLeft
;
3360 if( n
>nSrc
) n
= nSrc
;
3361 memcpy(pPayload
, pSrc
, n
);
3372 releasePage(pToRelease
);
3377 ** Change the MemPage.pParent pointer on the page whose number is
3378 ** given in the second argument so that MemPage.pParent holds the
3379 ** pointer in the third argument.
3381 static int reparentPage(Btree
*pBt
, Pgno pgno
, MemPage
*pNewParent
, int idx
){
3383 unsigned char *aData
;
3385 if( pgno
==0 ) return SQLITE_OK
;
3386 assert( pBt
->pPager
!=0 );
3387 aData
= sqlite3pager_lookup(pBt
->pPager
, pgno
);
3389 pThis
= (MemPage
*)&aData
[pBt
->pageSize
];
3390 assert( pThis
->aData
==aData
);
3391 if( pThis
->isInit
){
3392 if( pThis
->pParent
!=pNewParent
){
3393 if( pThis
->pParent
) sqlite3pager_unref(pThis
->pParent
->aData
);
3394 pThis
->pParent
= pNewParent
;
3395 if( pNewParent
) sqlite3pager_ref(pNewParent
->aData
);
3397 pThis
->idxParent
= idx
;
3399 sqlite3pager_unref(aData
);
3402 #ifndef SQLITE_OMIT_AUTOVACUUM
3403 if( pBt
->autoVacuum
){
3404 return ptrmapPut(pBt
, pgno
, PTRMAP_BTREE
, pNewParent
->pgno
);
3413 ** Change the pParent pointer of all children of pPage to point back
3416 ** In other words, for every child of pPage, invoke reparentPage()
3417 ** to make sure that each child knows that pPage is its parent.
3419 ** This routine gets called after you memcpy() one page into
3422 static int reparentChildPages(MemPage
*pPage
){
3424 Btree
*pBt
= pPage
->pBt
;
3427 if( pPage
->leaf
) return SQLITE_OK
;
3429 for(i
=0; i
<pPage
->nCell
; i
++){
3430 u8
*pCell
= findCell(pPage
, i
);
3432 rc
= reparentPage(pBt
, get4byte(pCell
), pPage
, i
);
3433 if( rc
!=SQLITE_OK
) return rc
;
3437 rc
= reparentPage(pBt
, get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]),
3439 pPage
->idxShift
= 0;
3445 ** Remove the i-th cell from pPage. This routine effects pPage only.
3446 ** The cell content is not freed or deallocated. It is assumed that
3447 ** the cell content has been copied someplace else. This routine just
3448 ** removes the reference to the cell from pPage.
3450 ** "sz" must be the number of bytes in the cell.
3452 static void dropCell(MemPage
*pPage
, int idx
, int sz
){
3453 int i
; /* Loop counter */
3454 int pc
; /* Offset to cell content of cell being deleted */
3455 u8
*data
; /* pPage->aData */
3456 u8
*ptr
; /* Used to move bytes around within data[] */
3458 assert( idx
>=0 && idx
<pPage
->nCell
);
3459 assert( sz
==cellSize(pPage
, idx
) );
3460 assert( sqlite3pager_iswriteable(pPage
->aData
) );
3461 data
= pPage
->aData
;
3462 ptr
= &data
[pPage
->cellOffset
+ 2*idx
];
3464 assert( pc
>10 && pc
+sz
<=pPage
->pBt
->usableSize
);
3465 freeSpace(pPage
, pc
, sz
);
3466 for(i
=idx
+1; i
<pPage
->nCell
; i
++, ptr
+=2){
3471 put2byte(&data
[pPage
->hdrOffset
+3], pPage
->nCell
);
3473 pPage
->idxShift
= 1;
3477 ** Insert a new cell on pPage at cell index "i". pCell points to the
3478 ** content of the cell.
3480 ** If the cell content will fit on the page, then put it there. If it
3481 ** will not fit, then make a copy of the cell content into pTemp if
3482 ** pTemp is not null. Regardless of pTemp, allocate a new entry
3483 ** in pPage->aOvfl[] and make it point to the cell content (either
3484 ** in pTemp or the original pCell) and also record its index.
3485 ** Allocating a new entry in pPage->aCell[] implies that
3486 ** pPage->nOverflow is incremented.
3488 ** If nSkip is non-zero, then do not copy the first nSkip bytes of the
3489 ** cell. The caller will overwrite them after this function returns. If
3490 ** nSkip is non-zero, then pCell may not point to an invalid memory location
3491 ** (but pCell+nSkip is always valid).
3493 static int insertCell(
3494 MemPage
*pPage
, /* Page into which we are copying */
3495 int i
, /* New cell becomes the i-th cell of the page */
3496 u8
*pCell
, /* Content of the new cell */
3497 int sz
, /* Bytes of content in pCell */
3498 u8
*pTemp
, /* Temp storage space for pCell, if needed */
3499 u8 nSkip
/* Do not write the first nSkip bytes of the cell */
3501 int idx
; /* Where to write new cell content in data[] */
3502 int j
; /* Loop counter */
3503 int top
; /* First byte of content for any cell in data[] */
3504 int end
; /* First byte past the last cell pointer in data[] */
3505 int ins
; /* Index in data[] where new cell pointer is inserted */
3506 int hdr
; /* Offset into data[] of the page header */
3507 int cellOffset
; /* Address of first cell pointer in data[] */
3508 u8
*data
; /* The content of the whole page */
3509 u8
*ptr
; /* Used for moving information around in data[] */
3511 assert( i
>=0 && i
<=pPage
->nCell
+pPage
->nOverflow
);
3512 assert( sz
==cellSizePtr(pPage
, pCell
) );
3513 assert( sqlite3pager_iswriteable(pPage
->aData
) );
3514 if( pPage
->nOverflow
|| sz
+2>pPage
->nFree
){
3516 memcpy(pTemp
+nSkip
, pCell
+nSkip
, sz
-nSkip
);
3519 j
= pPage
->nOverflow
++;
3520 assert( j
<sizeof(pPage
->aOvfl
)/sizeof(pPage
->aOvfl
[0]) );
3521 pPage
->aOvfl
[j
].pCell
= pCell
;
3522 pPage
->aOvfl
[j
].idx
= i
;
3525 data
= pPage
->aData
;
3526 hdr
= pPage
->hdrOffset
;
3527 top
= get2byte(&data
[hdr
+5]);
3528 cellOffset
= pPage
->cellOffset
;
3529 end
= cellOffset
+ 2*pPage
->nCell
+ 2;
3530 ins
= cellOffset
+ 2*i
;
3531 if( end
> top
- sz
){
3532 int rc
= defragmentPage(pPage
);
3533 if( rc
!=SQLITE_OK
) return rc
;
3534 top
= get2byte(&data
[hdr
+5]);
3535 assert( end
+ sz
<= top
);
3537 idx
= allocateSpace(pPage
, sz
);
3539 assert( end
<= get2byte(&data
[hdr
+5]) );
3542 memcpy(&data
[idx
+nSkip
], pCell
+nSkip
, sz
-nSkip
);
3543 for(j
=end
-2, ptr
=&data
[j
]; j
>ins
; j
-=2, ptr
-=2){
3547 put2byte(&data
[ins
], idx
);
3548 put2byte(&data
[hdr
+3], pPage
->nCell
);
3549 pPage
->idxShift
= 1;
3550 pageIntegrity(pPage
);
3551 #ifndef SQLITE_OMIT_AUTOVACUUM
3552 if( pPage
->pBt
->autoVacuum
){
3553 /* The cell may contain a pointer to an overflow page. If so, write
3554 ** the entry for the overflow page into the pointer map.
3557 parseCellPtr(pPage
, pCell
, &info
);
3558 if( (info
.nData
+(pPage
->intKey
?0:info
.nKey
))>info
.nLocal
){
3559 Pgno pgnoOvfl
= get4byte(&pCell
[info
.iOverflow
]);
3560 int rc
= ptrmapPut(pPage
->pBt
, pgnoOvfl
, PTRMAP_OVERFLOW1
, pPage
->pgno
);
3561 if( rc
!=SQLITE_OK
) return rc
;
3571 ** Add a list of cells to a page. The page should be initially empty.
3572 ** The cells are guaranteed to fit on the page.
3574 static void assemblePage(
3575 MemPage
*pPage
, /* The page to be assemblied */
3576 int nCell
, /* The number of cells to add to this page */
3577 u8
**apCell
, /* Pointers to cell bodies */
3578 int *aSize
/* Sizes of the cells */
3580 int i
; /* Loop counter */
3581 int totalSize
; /* Total size of all cells */
3582 int hdr
; /* Index of page header */
3583 int cellptr
; /* Address of next cell pointer */
3584 int cellbody
; /* Address of next cell body */
3585 u8
*data
; /* Data for the page */
3587 assert( pPage
->nOverflow
==0 );
3589 for(i
=0; i
<nCell
; i
++){
3590 totalSize
+= aSize
[i
];
3592 assert( totalSize
+2*nCell
<=pPage
->nFree
);
3593 assert( pPage
->nCell
==0 );
3594 cellptr
= pPage
->cellOffset
;
3595 data
= pPage
->aData
;
3596 hdr
= pPage
->hdrOffset
;
3597 put2byte(&data
[hdr
+3], nCell
);
3598 cellbody
= allocateSpace(pPage
, totalSize
);
3599 assert( cellbody
>0 );
3600 assert( pPage
->nFree
>= 2*nCell
);
3601 pPage
->nFree
-= 2*nCell
;
3602 for(i
=0; i
<nCell
; i
++){
3603 put2byte(&data
[cellptr
], cellbody
);
3604 memcpy(&data
[cellbody
], apCell
[i
], aSize
[i
]);
3606 cellbody
+= aSize
[i
];
3608 assert( cellbody
==pPage
->pBt
->usableSize
);
3609 pPage
->nCell
= nCell
;
3613 ** The following parameters determine how many adjacent pages get involved
3614 ** in a balancing operation. NN is the number of neighbors on either side
3615 ** of the page that participate in the balancing operation. NB is the
3616 ** total number of pages that participate, including the target page and
3617 ** NN neighbors on either side.
3619 ** The minimum value of NN is 1 (of course). Increasing NN above 1
3620 ** (to 2 or 3) gives a modest improvement in SELECT and DELETE performance
3621 ** in exchange for a larger degradation in INSERT and UPDATE performance.
3622 ** The value of NN appears to give the best results overall.
3624 #define NN 1 /* Number of neighbors on either side of pPage */
3625 #define NB (NN*2+1) /* Total pages involved in the balance */
3627 /* Forward reference */
3628 static int balance(MemPage
*, int);
3630 #ifndef SQLITE_OMIT_QUICKBALANCE
3632 ** This version of balance() handles the common special case where
3633 ** a new entry is being inserted on the extreme right-end of the
3634 ** tree, in other words, when the new entry will become the largest
3635 ** entry in the tree.
3637 ** Instead of trying balance the 3 right-most leaf pages, just add
3638 ** a new page to the right-hand side and put the one new entry in
3639 ** that page. This leaves the right side of the tree somewhat
3640 ** unbalanced. But odds are that we will be inserting new entries
3641 ** at the end soon afterwards so the nearly empty page will quickly
3642 ** fill up. On average.
3644 ** pPage is the leaf page which is the right-most page in the tree.
3645 ** pParent is its parent. pPage must have a single overflow entry
3646 ** which is also the right-most entry on the page.
3648 static int balance_quick(MemPage
*pPage
, MemPage
*pParent
){
3655 Btree
*pBt
= pPage
->pBt
;
3656 int parentIdx
= pParent
->nCell
; /* pParent new divider cell index */
3657 int parentSize
; /* Size of new divider cell */
3658 u8 parentCell
[64]; /* Space for the new divider cell */
3660 /* Allocate a new page. Insert the overflow cell from pPage
3661 ** into it. Then remove the overflow cell from pPage.
3663 rc
= allocatePage(pBt
, &pNew
, &pgnoNew
, 0, 0);
3664 if( rc
!=SQLITE_OK
){
3667 pCell
= pPage
->aOvfl
[0].pCell
;
3668 szCell
= cellSizePtr(pPage
, pCell
);
3669 zeroPage(pNew
, pPage
->aData
[0]);
3670 assemblePage(pNew
, 1, &pCell
, &szCell
);
3671 pPage
->nOverflow
= 0;
3673 /* Set the parent of the newly allocated page to pParent. */
3674 pNew
->pParent
= pParent
;
3675 sqlite3pager_ref(pParent
->aData
);
3677 /* pPage is currently the right-child of pParent. Change this
3678 ** so that the right-child is the new page allocated above and
3679 ** pPage is the next-to-right child.
3681 assert( pPage
->nCell
>0 );
3682 parseCellPtr(pPage
, findCell(pPage
, pPage
->nCell
-1), &info
);
3683 rc
= fillInCell(pParent
, parentCell
, 0, info
.nKey
, 0, 0, &parentSize
);
3684 if( rc
!=SQLITE_OK
){
3687 assert( parentSize
<64 );
3688 rc
= insertCell(pParent
, parentIdx
, parentCell
, parentSize
, 0, 4);
3689 if( rc
!=SQLITE_OK
){
3692 put4byte(findOverflowCell(pParent
,parentIdx
), pPage
->pgno
);
3693 put4byte(&pParent
->aData
[pParent
->hdrOffset
+8], pgnoNew
);
3695 #ifndef SQLITE_OMIT_AUTOVACUUM
3696 /* If this is an auto-vacuum database, update the pointer map
3697 ** with entries for the new page, and any pointer from the
3698 ** cell on the page to an overflow page.
3700 if( pBt
->autoVacuum
){
3701 rc
= ptrmapPut(pBt
, pgnoNew
, PTRMAP_BTREE
, pParent
->pgno
);
3702 if( rc
!=SQLITE_OK
){
3705 rc
= ptrmapPutOvfl(pNew
, 0);
3706 if( rc
!=SQLITE_OK
){
3712 /* Release the reference to the new page and balance the parent page,
3713 ** in case the divider cell inserted caused it to become overfull.
3716 return balance(pParent
, 0);
3718 #endif /* SQLITE_OMIT_QUICKBALANCE */
3721 ** The ISAUTOVACUUM macro is used within balance_nonroot() to determine
3722 ** if the database supports auto-vacuum or not. Because it is used
3723 ** within an expression that is an argument to another macro
3724 ** (sqliteMallocRaw), it is not possible to use conditional compilation.
3725 ** So, this macro is defined instead.
3727 #ifndef SQLITE_OMIT_AUTOVACUUM
3728 #define ISAUTOVACUUM (pBt->autoVacuum)
3730 #define ISAUTOVACUUM 0
3734 ** This routine redistributes Cells on pPage and up to NN*2 siblings
3735 ** of pPage so that all pages have about the same amount of free space.
3736 ** Usually NN siblings on either side of pPage is used in the balancing,
3737 ** though more siblings might come from one side if pPage is the first
3738 ** or last child of its parent. If pPage has fewer than 2*NN siblings
3739 ** (something which can only happen if pPage is the root page or a
3740 ** child of root) then all available siblings participate in the balancing.
3742 ** The number of siblings of pPage might be increased or decreased by one or
3743 ** two in an effort to keep pages nearly full but not over full. The root page
3744 ** is special and is allowed to be nearly empty. If pPage is
3745 ** the root page, then the depth of the tree might be increased
3746 ** or decreased by one, as necessary, to keep the root page from being
3747 ** overfull or completely empty.
3749 ** Note that when this routine is called, some of the Cells on pPage
3750 ** might not actually be stored in pPage->aData[]. This can happen
3751 ** if the page is overfull. Part of the job of this routine is to
3752 ** make sure all Cells for pPage once again fit in pPage->aData[].
3754 ** In the course of balancing the siblings of pPage, the parent of pPage
3755 ** might become overfull or underfull. If that happens, then this routine
3756 ** is called recursively on the parent.
3758 ** If this routine fails for any reason, it might leave the database
3759 ** in a corrupted state. So if this routine fails, the database should
3762 static int balance_nonroot(MemPage
*pPage
){
3763 MemPage
*pParent
; /* The parent of pPage */
3764 Btree
*pBt
; /* The whole database */
3765 int nCell
= 0; /* Number of cells in apCell[] */
3766 int nMaxCells
= 0; /* Allocated size of apCell, szCell, aFrom. */
3767 int nOld
; /* Number of pages in apOld[] */
3768 int nNew
; /* Number of pages in apNew[] */
3769 int nDiv
; /* Number of cells in apDiv[] */
3770 int i
, j
, k
; /* Loop counters */
3771 int idx
; /* Index of pPage in pParent->aCell[] */
3772 int nxDiv
; /* Next divider slot in pParent->aCell[] */
3773 int rc
; /* The return code */
3774 int leafCorrection
; /* 4 if pPage is a leaf. 0 if not */
3775 int leafData
; /* True if pPage is a leaf of a LEAFDATA tree */
3776 int usableSpace
; /* Bytes in pPage beyond the header */
3777 int pageFlags
; /* Value of pPage->aData[0] */
3778 int subtotal
; /* Subtotal of bytes in cells on one page */
3779 int iSpace
= 0; /* First unused byte of aSpace[] */
3780 MemPage
*apOld
[NB
]; /* pPage and up to two siblings */
3781 Pgno pgnoOld
[NB
]; /* Page numbers for each page in apOld[] */
3782 MemPage
*apCopy
[NB
]; /* Private copies of apOld[] pages */
3783 MemPage
*apNew
[NB
+2]; /* pPage and up to NB siblings after balancing */
3784 Pgno pgnoNew
[NB
+2]; /* Page numbers for each page in apNew[] */
3785 int idxDiv
[NB
]; /* Indices of divider cells in pParent */
3786 u8
*apDiv
[NB
]; /* Divider cells in pParent */
3787 int cntNew
[NB
+2]; /* Index in aCell[] of cell after i-th page */
3788 int szNew
[NB
+2]; /* Combined size of cells place on i-th page */
3789 u8
**apCell
= 0; /* All cells begin balanced */
3790 int *szCell
; /* Local size of all cells in apCell[] */
3791 u8
*aCopy
[NB
]; /* Space for holding data of apCopy[] */
3792 u8
*aSpace
; /* Space to hold copies of dividers cells */
3793 #ifndef SQLITE_OMIT_AUTOVACUUM
3798 ** Find the parent page.
3800 assert( pPage
->isInit
);
3801 assert( sqlite3pager_iswriteable(pPage
->aData
) );
3803 pParent
= pPage
->pParent
;
3804 sqlite3pager_write(pParent
->aData
);
3806 TRACE(("BALANCE: begin page %d child of %d\n", pPage
->pgno
, pParent
->pgno
));
3808 #ifndef SQLITE_OMIT_QUICKBALANCE
3810 ** A special case: If a new entry has just been inserted into a
3811 ** table (that is, a btree with integer keys and all data at the leaves)
3812 ** an the new entry is the right-most entry in the tree (it has the
3813 ** largest key) then use the special balance_quick() routine for
3814 ** balancing. balance_quick() is much faster and results in a tighter
3815 ** packing of data in the common case.
3820 pPage
->nOverflow
==1 &&
3821 pPage
->aOvfl
[0].idx
==pPage
->nCell
&&
3822 pPage
->pParent
->pgno
!=1 &&
3823 get4byte(&pParent
->aData
[pParent
->hdrOffset
+8])==pPage
->pgno
3826 ** TODO: Check the siblings to the left of pPage. It may be that
3827 ** they are not full and no new page is required.
3829 return balance_quick(pPage
, pParent
);
3834 ** Find the cell in the parent page whose left child points back
3835 ** to pPage. The "idx" variable is the index of that cell. If pPage
3836 ** is the rightmost child of pParent then set idx to pParent->nCell
3838 if( pParent
->idxShift
){
3841 assert( pgno
==sqlite3pager_pagenumber(pPage
->aData
) );
3842 for(idx
=0; idx
<pParent
->nCell
; idx
++){
3843 if( get4byte(findCell(pParent
, idx
))==pgno
){
3847 assert( idx
<pParent
->nCell
3848 || get4byte(&pParent
->aData
[pParent
->hdrOffset
+8])==pgno
);
3850 idx
= pPage
->idxParent
;
3854 ** Initialize variables so that it will be safe to jump
3855 ** directly to balance_cleanup at any moment.
3858 sqlite3pager_ref(pParent
->aData
);
3861 ** Find sibling pages to pPage and the cells in pParent that divide
3862 ** the siblings. An attempt is made to find NN siblings on either
3863 ** side of pPage. More siblings are taken from one side, however, if
3864 ** pPage there are fewer than NN siblings on the other side. If pParent
3865 ** has NB or fewer children then all children of pParent are taken.
3868 if( nxDiv
+ NB
> pParent
->nCell
){
3869 nxDiv
= pParent
->nCell
- NB
+ 1;
3875 for(i
=0, k
=nxDiv
; i
<NB
; i
++, k
++){
3876 if( k
<pParent
->nCell
){
3878 apDiv
[i
] = findCell(pParent
, k
);
3880 assert( !pParent
->leaf
);
3881 pgnoOld
[i
] = get4byte(apDiv
[i
]);
3882 }else if( k
==pParent
->nCell
){
3883 pgnoOld
[i
] = get4byte(&pParent
->aData
[pParent
->hdrOffset
+8]);
3887 rc
= getAndInitPage(pBt
, pgnoOld
[i
], &apOld
[i
], pParent
);
3888 if( rc
) goto balance_cleanup
;
3889 apOld
[i
]->idxParent
= k
;
3893 nMaxCells
+= 1+apOld
[i
]->nCell
+apOld
[i
]->nOverflow
;
3896 /* Make nMaxCells a multiple of 2 in order to preserve 8-byte
3898 nMaxCells
= (nMaxCells
+ 1)&~1;
3901 ** Allocate space for memory structures
3903 apCell
= sqliteMallocRaw(
3904 nMaxCells
*sizeof(u8
*) /* apCell */
3905 + nMaxCells
*sizeof(int) /* szCell */
3906 + ROUND8(sizeof(MemPage
))*NB
/* aCopy */
3907 + pBt
->pageSize
*(5+NB
) /* aSpace */
3908 + (ISAUTOVACUUM
? nMaxCells
: 0) /* aFrom */
3912 goto balance_cleanup
;
3914 szCell
= (int*)&apCell
[nMaxCells
];
3915 aCopy
[0] = (u8
*)&szCell
[nMaxCells
];
3916 assert( ((aCopy
[0] - (u8
*)apCell
) & 7)==0 ); /* 8-byte alignment required */
3917 for(i
=1; i
<NB
; i
++){
3918 aCopy
[i
] = &aCopy
[i
-1][pBt
->pageSize
+ROUND8(sizeof(MemPage
))];
3919 assert( ((aCopy
[i
] - (u8
*)apCell
) & 7)==0 ); /* 8-byte alignment required */
3921 aSpace
= &aCopy
[NB
-1][pBt
->pageSize
+ROUND8(sizeof(MemPage
))];
3922 assert( ((aSpace
- (u8
*)apCell
) & 7)==0 ); /* 8-byte alignment required */
3923 #ifndef SQLITE_OMIT_AUTOVACUUM
3924 if( pBt
->autoVacuum
){
3925 aFrom
= &aSpace
[5*pBt
->pageSize
];
3930 ** Make copies of the content of pPage and its siblings into aOld[].
3931 ** The rest of this function will use data from the copies rather
3932 ** that the original pages since the original pages will be in the
3933 ** process of being overwritten.
3935 for(i
=0; i
<nOld
; i
++){
3936 MemPage
*p
= apCopy
[i
] = (MemPage
*)&aCopy
[i
][pBt
->pageSize
];
3937 p
->aData
= &((u8
*)p
)[-pBt
->pageSize
];
3938 memcpy(p
->aData
, apOld
[i
]->aData
, pBt
->pageSize
+ sizeof(MemPage
));
3939 /* The memcpy() above changes the value of p->aData so we have to
3941 p
->aData
= &((u8
*)p
)[-pBt
->pageSize
];
3945 ** Load pointers to all cells on sibling pages and the divider cells
3946 ** into the local apCell[] array. Make copies of the divider cells
3947 ** into space obtained form aSpace[] and remove the the divider Cells
3950 ** If the siblings are on leaf pages, then the child pointers of the
3951 ** divider cells are stripped from the cells before they are copied
3952 ** into aSpace[]. In this way, all cells in apCell[] are without
3953 ** child pointers. If siblings are not leaves, then all cell in
3954 ** apCell[] include child pointers. Either way, all cells in apCell[]
3957 ** leafCorrection: 4 if pPage is a leaf. 0 if pPage is not a leaf.
3958 ** leafData: 1 if pPage holds key+data and pParent holds only keys.
3961 leafCorrection
= pPage
->leaf
*4;
3962 leafData
= pPage
->leafData
&& pPage
->leaf
;
3963 for(i
=0; i
<nOld
; i
++){
3964 MemPage
*pOld
= apCopy
[i
];
3965 int limit
= pOld
->nCell
+pOld
->nOverflow
;
3966 for(j
=0; j
<limit
; j
++){
3967 assert( nCell
<nMaxCells
);
3968 apCell
[nCell
] = findOverflowCell(pOld
, j
);
3969 szCell
[nCell
] = cellSizePtr(pOld
, apCell
[nCell
]);
3970 #ifndef SQLITE_OMIT_AUTOVACUUM
3971 if( pBt
->autoVacuum
){
3974 for(a
=0; a
<pOld
->nOverflow
; a
++){
3975 if( pOld
->aOvfl
[a
].pCell
==apCell
[nCell
] ){
3976 aFrom
[nCell
] = 0xFF;
3985 int sz
= cellSizePtr(pParent
, apDiv
[i
]);
3987 /* With the LEAFDATA flag, pParent cells hold only INTKEYs that
3988 ** are duplicates of keys on the child pages. We need to remove
3989 ** the divider cells from pParent, but the dividers cells are not
3990 ** added to apCell[] because they are duplicates of child cells.
3992 dropCell(pParent
, nxDiv
, sz
);
3995 assert( nCell
<nMaxCells
);
3997 pTemp
= &aSpace
[iSpace
];
3999 assert( iSpace
<=pBt
->pageSize
*5 );
4000 memcpy(pTemp
, apDiv
[i
], sz
);
4001 apCell
[nCell
] = pTemp
+leafCorrection
;
4002 #ifndef SQLITE_OMIT_AUTOVACUUM
4003 if( pBt
->autoVacuum
){
4004 aFrom
[nCell
] = 0xFF;
4007 dropCell(pParent
, nxDiv
, sz
);
4008 szCell
[nCell
] -= leafCorrection
;
4009 assert( get4byte(pTemp
)==pgnoOld
[i
] );
4011 assert( leafCorrection
==0 );
4012 /* The right pointer of the child page pOld becomes the left
4013 ** pointer of the divider cell */
4014 memcpy(apCell
[nCell
], &pOld
->aData
[pOld
->hdrOffset
+8], 4);
4016 assert( leafCorrection
==4 );
4024 ** Figure out the number of pages needed to hold all nCell cells.
4025 ** Store this number in "k". Also compute szNew[] which is the total
4026 ** size of all cells on the i-th page and cntNew[] which is the index
4027 ** in apCell[] of the cell that divides page i from page i+1.
4028 ** cntNew[k] should equal nCell.
4030 ** Values computed by this block:
4032 ** k: The total number of sibling pages
4033 ** szNew[i]: Spaced used on the i-th sibling page.
4034 ** cntNew[i]: Index in apCell[] and szCell[] for the first cell to
4035 ** the right of the i-th sibling page.
4036 ** usableSpace: Number of bytes of space available on each sibling.
4039 usableSpace
= pBt
->usableSize
- 12 + leafCorrection
;
4040 for(subtotal
=k
=i
=0; i
<nCell
; i
++){
4041 assert( i
<nMaxCells
);
4042 subtotal
+= szCell
[i
] + 2;
4043 if( subtotal
> usableSpace
){
4044 szNew
[k
] = subtotal
- szCell
[i
];
4046 if( leafData
){ i
--; }
4051 szNew
[k
] = subtotal
;
4056 ** The packing computed by the previous block is biased toward the siblings
4057 ** on the left side. The left siblings are always nearly full, while the
4058 ** right-most sibling might be nearly empty. This block of code attempts
4059 ** to adjust the packing of siblings to get a better balance.
4061 ** This adjustment is more than an optimization. The packing above might
4062 ** be so out of balance as to be illegal. For example, the right-most
4063 ** sibling might be completely empty. This adjustment is not optional.
4065 for(i
=k
-1; i
>0; i
--){
4066 int szRight
= szNew
[i
]; /* Size of sibling on the right */
4067 int szLeft
= szNew
[i
-1]; /* Size of sibling on the left */
4068 int r
; /* Index of right-most cell in left sibling */
4069 int d
; /* Index of first cell to the left of right sibling */
4071 r
= cntNew
[i
-1] - 1;
4072 d
= r
+ 1 - leafData
;
4073 assert( d
<nMaxCells
);
4074 assert( r
<nMaxCells
);
4075 while( szRight
==0 || szRight
+szCell
[d
]+2<=szLeft
-(szCell
[r
]+2) ){
4076 szRight
+= szCell
[d
] + 2;
4077 szLeft
-= szCell
[r
] + 2;
4079 r
= cntNew
[i
-1] - 1;
4080 d
= r
+ 1 - leafData
;
4083 szNew
[i
-1] = szLeft
;
4085 assert( cntNew
[0]>0 );
4088 ** Allocate k new pages. Reuse old pages where possible.
4090 assert( pPage
->pgno
>1 );
4091 pageFlags
= pPage
->aData
[0];
4095 pNew
= apNew
[i
] = apOld
[i
];
4096 pgnoNew
[i
] = pgnoOld
[i
];
4098 rc
= sqlite3pager_write(pNew
->aData
);
4099 if( rc
) goto balance_cleanup
;
4101 rc
= allocatePage(pBt
, &pNew
, &pgnoNew
[i
], pgnoNew
[i
-1], 0);
4102 if( rc
) goto balance_cleanup
;
4106 zeroPage(pNew
, pageFlags
);
4109 /* Free any old pages that were not reused as new pages.
4112 rc
= freePage(apOld
[i
]);
4113 if( rc
) goto balance_cleanup
;
4114 releasePage(apOld
[i
]);
4120 ** Put the new pages in accending order. This helps to
4121 ** keep entries in the disk file in order so that a scan
4122 ** of the table is a linear scan through the file. That
4123 ** in turn helps the operating system to deliver pages
4124 ** from the disk more rapidly.
4126 ** An O(n^2) insertion sort algorithm is used, but since
4127 ** n is never more than NB (a small constant), that should
4128 ** not be a problem.
4130 ** When NB==3, this one optimization makes the database
4131 ** about 25% faster for large insertions and deletions.
4133 for(i
=0; i
<k
-1; i
++){
4134 int minV
= pgnoNew
[i
];
4136 for(j
=i
+1; j
<k
; j
++){
4137 if( pgnoNew
[j
]<(unsigned)minV
){
4147 pgnoNew
[i
] = pgnoNew
[minI
];
4148 apNew
[i
] = apNew
[minI
];
4153 TRACE(("BALANCE: old: %d %d %d new: %d(%d) %d(%d) %d(%d) %d(%d) %d(%d)\n",
4155 nOld
>=2 ? pgnoOld
[1] : 0,
4156 nOld
>=3 ? pgnoOld
[2] : 0,
4157 pgnoNew
[0], szNew
[0],
4158 nNew
>=2 ? pgnoNew
[1] : 0, nNew
>=2 ? szNew
[1] : 0,
4159 nNew
>=3 ? pgnoNew
[2] : 0, nNew
>=3 ? szNew
[2] : 0,
4160 nNew
>=4 ? pgnoNew
[3] : 0, nNew
>=4 ? szNew
[3] : 0,
4161 nNew
>=5 ? pgnoNew
[4] : 0, nNew
>=5 ? szNew
[4] : 0));
4164 ** Evenly distribute the data in apCell[] across the new pages.
4165 ** Insert divider cells into pParent as necessary.
4168 for(i
=0; i
<nNew
; i
++){
4169 /* Assemble the new sibling page. */
4170 MemPage
*pNew
= apNew
[i
];
4171 assert( j
<nMaxCells
);
4172 assert( pNew
->pgno
==pgnoNew
[i
] );
4173 assemblePage(pNew
, cntNew
[i
]-j
, &apCell
[j
], &szCell
[j
]);
4174 assert( pNew
->nCell
>0 );
4175 assert( pNew
->nOverflow
==0 );
4177 #ifndef SQLITE_OMIT_AUTOVACUUM
4178 /* If this is an auto-vacuum database, update the pointer map entries
4179 ** that point to the siblings that were rearranged. These can be: left
4180 ** children of cells, the right-child of the page, or overflow pages
4181 ** pointed to by cells.
4183 if( pBt
->autoVacuum
){
4184 for(k
=j
; k
<cntNew
[i
]; k
++){
4185 assert( k
<nMaxCells
);
4186 if( aFrom
[k
]==0xFF || apCopy
[aFrom
[k
]]->pgno
!=pNew
->pgno
){
4187 rc
= ptrmapPutOvfl(pNew
, k
-j
);
4188 if( rc
!=SQLITE_OK
){
4189 goto balance_cleanup
;
4198 /* If the sibling page assembled above was not the right-most sibling,
4199 ** insert a divider cell into the parent page.
4201 if( i
<nNew
-1 && j
<nCell
){
4206 assert( j
<nMaxCells
);
4208 sz
= szCell
[j
] + leafCorrection
;
4210 memcpy(&pNew
->aData
[8], pCell
, 4);
4212 }else if( leafData
){
4213 /* If the tree is a leaf-data tree, and the siblings are leaves,
4214 ** then there is no divider cell in apCell[]. Instead, the divider
4215 ** cell consists of the integer key for the right-most cell of
4216 ** the sibling-page assembled above only.
4220 parseCellPtr(pNew
, apCell
[j
], &info
);
4221 pCell
= &aSpace
[iSpace
];
4222 fillInCell(pParent
, pCell
, 0, info
.nKey
, 0, 0, &sz
);
4224 assert( iSpace
<=pBt
->pageSize
*5 );
4228 pTemp
= &aSpace
[iSpace
];
4230 assert( iSpace
<=pBt
->pageSize
*5 );
4232 rc
= insertCell(pParent
, nxDiv
, pCell
, sz
, pTemp
, 4);
4233 if( rc
!=SQLITE_OK
) goto balance_cleanup
;
4234 put4byte(findOverflowCell(pParent
,nxDiv
), pNew
->pgno
);
4235 #ifndef SQLITE_OMIT_AUTOVACUUM
4236 /* If this is an auto-vacuum database, and not a leaf-data tree,
4237 ** then update the pointer map with an entry for the overflow page
4238 ** that the cell just inserted points to (if any).
4240 if( pBt
->autoVacuum
&& !leafData
){
4241 rc
= ptrmapPutOvfl(pParent
, nxDiv
);
4242 if( rc
!=SQLITE_OK
){
4243 goto balance_cleanup
;
4252 if( (pageFlags
& PTF_LEAF
)==0 ){
4253 memcpy(&apNew
[nNew
-1]->aData
[8], &apCopy
[nOld
-1]->aData
[8], 4);
4255 if( nxDiv
==pParent
->nCell
+pParent
->nOverflow
){
4256 /* Right-most sibling is the right-most child of pParent */
4257 put4byte(&pParent
->aData
[pParent
->hdrOffset
+8], pgnoNew
[nNew
-1]);
4259 /* Right-most sibling is the left child of the first entry in pParent
4260 ** past the right-most divider entry */
4261 put4byte(findOverflowCell(pParent
, nxDiv
), pgnoNew
[nNew
-1]);
4265 ** Reparent children of all cells.
4267 for(i
=0; i
<nNew
; i
++){
4268 rc
= reparentChildPages(apNew
[i
]);
4269 if( rc
!=SQLITE_OK
) goto balance_cleanup
;
4271 rc
= reparentChildPages(pParent
);
4272 if( rc
!=SQLITE_OK
) goto balance_cleanup
;
4275 ** Balance the parent page. Note that the current page (pPage) might
4276 ** have been added to the freelist so it might no longer be initialized.
4277 ** But the parent page will always be initialized.
4279 assert( pParent
->isInit
);
4280 /* assert( pPage->isInit ); // No! pPage might have been added to freelist */
4281 /* pageIntegrity(pPage); // No! pPage might have been added to freelist */
4282 rc
= balance(pParent
, 0);
4285 ** Cleanup before returning.
4289 for(i
=0; i
<nOld
; i
++){
4290 releasePage(apOld
[i
]);
4292 for(i
=0; i
<nNew
; i
++){
4293 releasePage(apNew
[i
]);
4295 releasePage(pParent
);
4296 TRACE(("BALANCE: finished with %d: old=%d new=%d cells=%d\n",
4297 pPage
->pgno
, nOld
, nNew
, nCell
));
4302 ** This routine is called for the root page of a btree when the root
4303 ** page contains no cells. This is an opportunity to make the tree
4304 ** shallower by one level.
4306 static int balance_shallower(MemPage
*pPage
){
4307 MemPage
*pChild
; /* The only child page of pPage */
4308 Pgno pgnoChild
; /* Page number for pChild */
4309 int rc
= SQLITE_OK
; /* Return code from subprocedures */
4310 Btree
*pBt
; /* The main BTree structure */
4311 int mxCellPerPage
; /* Maximum number of cells per page */
4312 u8
**apCell
; /* All cells from pages being balanced */
4313 int *szCell
; /* Local size of all cells */
4315 assert( pPage
->pParent
==0 );
4316 assert( pPage
->nCell
==0 );
4318 mxCellPerPage
= MX_CELL(pBt
);
4319 apCell
= sqliteMallocRaw( mxCellPerPage
*(sizeof(u8
*)+sizeof(int)) );
4320 if( apCell
==0 ) return SQLITE_NOMEM
;
4321 szCell
= (int*)&apCell
[mxCellPerPage
];
4323 /* The table is completely empty */
4324 TRACE(("BALANCE: empty table %d\n", pPage
->pgno
));
4326 /* The root page is empty but has one child. Transfer the
4327 ** information from that one child into the root page if it
4328 ** will fit. This reduces the depth of the tree by one.
4330 ** If the root page is page 1, it has less space available than
4331 ** its child (due to the 100 byte header that occurs at the beginning
4332 ** of the database fle), so it might not be able to hold all of the
4333 ** information currently contained in the child. If this is the
4334 ** case, then do not do the transfer. Leave page 1 empty except
4335 ** for the right-pointer to the child page. The child page becomes
4336 ** the virtual root of the tree.
4338 pgnoChild
= get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]);
4339 assert( pgnoChild
>0 );
4340 assert( pgnoChild
<=sqlite3pager_pagecount(pPage
->pBt
->pPager
) );
4341 rc
= getPage(pPage
->pBt
, pgnoChild
, &pChild
);
4342 if( rc
) goto end_shallow_balance
;
4343 if( pPage
->pgno
==1 ){
4344 rc
= initPage(pChild
, pPage
);
4345 if( rc
) goto end_shallow_balance
;
4346 assert( pChild
->nOverflow
==0 );
4347 if( pChild
->nFree
>=100 ){
4348 /* The child information will fit on the root page, so do the
4351 zeroPage(pPage
, pChild
->aData
[0]);
4352 for(i
=0; i
<pChild
->nCell
; i
++){
4353 apCell
[i
] = findCell(pChild
,i
);
4354 szCell
[i
] = cellSizePtr(pChild
, apCell
[i
]);
4356 assemblePage(pPage
, pChild
->nCell
, apCell
, szCell
);
4357 /* Copy the right-pointer of the child to the parent. */
4358 put4byte(&pPage
->aData
[pPage
->hdrOffset
+8],
4359 get4byte(&pChild
->aData
[pChild
->hdrOffset
+8]));
4361 TRACE(("BALANCE: child %d transfer to page 1\n", pChild
->pgno
));
4363 /* The child has more information that will fit on the root.
4364 ** The tree is already balanced. Do nothing. */
4365 TRACE(("BALANCE: child %d will not fit on page 1\n", pChild
->pgno
));
4368 memcpy(pPage
->aData
, pChild
->aData
, pPage
->pBt
->usableSize
);
4371 rc
= initPage(pPage
, 0);
4372 assert( rc
==SQLITE_OK
);
4374 TRACE(("BALANCE: transfer child %d into root %d\n",
4375 pChild
->pgno
, pPage
->pgno
));
4377 rc
= reparentChildPages(pPage
);
4378 assert( pPage
->nOverflow
==0 );
4379 #ifndef SQLITE_OMIT_AUTOVACUUM
4380 if( pBt
->autoVacuum
){
4382 for(i
=0; i
<pPage
->nCell
; i
++){
4383 rc
= ptrmapPutOvfl(pPage
, i
);
4384 if( rc
!=SQLITE_OK
){
4385 goto end_shallow_balance
;
4390 if( rc
!=SQLITE_OK
) goto end_shallow_balance
;
4391 releasePage(pChild
);
4393 end_shallow_balance
:
4400 ** The root page is overfull
4402 ** When this happens, Create a new child page and copy the
4403 ** contents of the root into the child. Then make the root
4404 ** page an empty page with rightChild pointing to the new
4405 ** child. Finally, call balance_internal() on the new child
4406 ** to cause it to split.
4408 static int balance_deeper(MemPage
*pPage
){
4409 int rc
; /* Return value from subprocedures */
4410 MemPage
*pChild
; /* Pointer to a new child page */
4411 Pgno pgnoChild
; /* Page number of the new child page */
4412 Btree
*pBt
; /* The BTree */
4413 int usableSize
; /* Total usable size of a page */
4414 u8
*data
; /* Content of the parent page */
4415 u8
*cdata
; /* Content of the child page */
4416 int hdr
; /* Offset to page header in parent */
4417 int brk
; /* Offset to content of first cell in parent */
4419 assert( pPage
->pParent
==0 );
4420 assert( pPage
->nOverflow
>0 );
4422 rc
= allocatePage(pBt
, &pChild
, &pgnoChild
, pPage
->pgno
, 0);
4424 assert( sqlite3pager_iswriteable(pChild
->aData
) );
4425 usableSize
= pBt
->usableSize
;
4426 data
= pPage
->aData
;
4427 hdr
= pPage
->hdrOffset
;
4428 brk
= get2byte(&data
[hdr
+5]);
4429 cdata
= pChild
->aData
;
4430 memcpy(cdata
, &data
[hdr
], pPage
->cellOffset
+2*pPage
->nCell
-hdr
);
4431 memcpy(&cdata
[brk
], &data
[brk
], usableSize
-brk
);
4432 assert( pChild
->isInit
==0 );
4433 rc
= initPage(pChild
, pPage
);
4434 if( rc
) goto balancedeeper_out
;
4435 memcpy(pChild
->aOvfl
, pPage
->aOvfl
, pPage
->nOverflow
*sizeof(pPage
->aOvfl
[0]));
4436 pChild
->nOverflow
= pPage
->nOverflow
;
4437 if( pChild
->nOverflow
){
4440 assert( pChild
->nCell
==pPage
->nCell
);
4441 zeroPage(pPage
, pChild
->aData
[0] & ~PTF_LEAF
);
4442 put4byte(&pPage
->aData
[pPage
->hdrOffset
+8], pgnoChild
);
4443 TRACE(("BALANCE: copy root %d into %d\n", pPage
->pgno
, pChild
->pgno
));
4444 #ifndef SQLITE_OMIT_AUTOVACUUM
4445 if( pBt
->autoVacuum
){
4447 rc
= ptrmapPut(pBt
, pChild
->pgno
, PTRMAP_BTREE
, pPage
->pgno
);
4448 if( rc
) goto balancedeeper_out
;
4449 for(i
=0; i
<pChild
->nCell
; i
++){
4450 rc
= ptrmapPutOvfl(pChild
, i
);
4451 if( rc
!=SQLITE_OK
){
4457 rc
= balance_nonroot(pChild
);
4460 releasePage(pChild
);
4465 ** Decide if the page pPage needs to be balanced. If balancing is
4466 ** required, call the appropriate balancing routine.
4468 static int balance(MemPage
*pPage
, int insert
){
4470 if( pPage
->pParent
==0 ){
4471 if( pPage
->nOverflow
>0 ){
4472 rc
= balance_deeper(pPage
);
4474 if( rc
==SQLITE_OK
&& pPage
->nCell
==0 ){
4475 rc
= balance_shallower(pPage
);
4478 if( pPage
->nOverflow
>0 ||
4479 (!insert
&& pPage
->nFree
>pPage
->pBt
->usableSize
*2/3) ){
4480 rc
= balance_nonroot(pPage
);
4487 ** This routine checks all cursors that point to table pgnoRoot.
4488 ** If any of those cursors other than pExclude were opened with
4489 ** wrFlag==0 then this routine returns SQLITE_LOCKED. If all
4490 ** cursors that point to pgnoRoot were opened with wrFlag==1
4491 ** then this routine returns SQLITE_OK.
4493 ** In addition to checking for read-locks (where a read-lock
4494 ** means a cursor opened with wrFlag==0) this routine also moves
4495 ** all cursors other than pExclude so that they are pointing to the
4496 ** first Cell on root page. This is necessary because an insert
4497 ** or delete might change the number of cells on a page or delete
4498 ** a page entirely and we do not want to leave any cursors
4499 ** pointing to non-existant pages or cells.
4501 static int checkReadLocks(Btree
*pBt
, Pgno pgnoRoot
, BtCursor
*pExclude
){
4503 for(p
=pBt
->pCursor
; p
; p
=p
->pNext
){
4504 if( p
->pgnoRoot
!=pgnoRoot
|| p
==pExclude
) continue;
4505 if( p
->wrFlag
==0 ) return SQLITE_LOCKED
;
4506 if( p
->pPage
->pgno
!=p
->pgnoRoot
){
4514 ** Insert a new record into the BTree. The key is given by (pKey,nKey)
4515 ** and the data is given by (pData,nData). The cursor is used only to
4516 ** define what table the record should be inserted into. The cursor
4517 ** is left pointing at a random location.
4519 ** For an INTKEY table, only the nKey value of the key is used. pKey is
4520 ** ignored. For a ZERODATA table, the pData and nData are both ignored.
4522 int sqlite3BtreeInsert(
4523 BtCursor
*pCur
, /* Insert data into the table of this cursor */
4524 const void *pKey
, i64 nKey
, /* The key of the new record */
4525 const void *pData
, int nData
/* The data of the new record */
4531 Btree
*pBt
= pCur
->pBt
;
4532 unsigned char *oldCell
;
4533 unsigned char *newCell
= 0;
4535 if( pBt
->inTrans
!=TRANS_WRITE
){
4536 /* Must start a transaction before doing an insert */
4537 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
4539 assert( !pBt
->readOnly
);
4540 if( !pCur
->wrFlag
){
4541 return SQLITE_PERM
; /* Cursor not open for writing */
4543 if( checkReadLocks(pBt
, pCur
->pgnoRoot
, pCur
) ){
4544 return SQLITE_LOCKED
; /* The table pCur points to has a read lock */
4546 rc
= sqlite3BtreeMoveto(pCur
, pKey
, nKey
, &loc
);
4548 pPage
= pCur
->pPage
;
4549 assert( pPage
->intKey
|| nKey
>=0 );
4550 assert( pPage
->leaf
|| !pPage
->leafData
);
4551 TRACE(("INSERT: table=%d nkey=%lld ndata=%d page=%d %s\n",
4552 pCur
->pgnoRoot
, nKey
, nData
, pPage
->pgno
,
4553 loc
==0 ? "overwrite" : "new entry"));
4554 assert( pPage
->isInit
);
4555 rc
= sqlite3pager_write(pPage
->aData
);
4557 newCell
= sqliteMallocRaw( MX_CELL_SIZE(pBt
) );
4558 if( newCell
==0 ) return SQLITE_NOMEM
;
4559 rc
= fillInCell(pPage
, newCell
, pKey
, nKey
, pData
, nData
, &szNew
);
4560 if( rc
) goto end_insert
;
4561 assert( szNew
==cellSizePtr(pPage
, newCell
) );
4562 assert( szNew
<=MX_CELL_SIZE(pBt
) );
4563 if( loc
==0 && pCur
->isValid
){
4565 assert( pCur
->idx
>=0 && pCur
->idx
<pPage
->nCell
);
4566 oldCell
= findCell(pPage
, pCur
->idx
);
4568 memcpy(newCell
, oldCell
, 4);
4570 szOld
= cellSizePtr(pPage
, oldCell
);
4571 rc
= clearCell(pPage
, oldCell
);
4572 if( rc
) goto end_insert
;
4573 dropCell(pPage
, pCur
->idx
, szOld
);
4574 }else if( loc
<0 && pPage
->nCell
>0 ){
4575 assert( pPage
->leaf
);
4577 pCur
->info
.nSize
= 0;
4579 assert( pPage
->leaf
);
4581 rc
= insertCell(pPage
, pCur
->idx
, newCell
, szNew
, 0, 0);
4582 if( rc
!=SQLITE_OK
) goto end_insert
;
4583 rc
= balance(pPage
, 1);
4584 /* sqlite3BtreePageDump(pCur->pBt, pCur->pgnoRoot, 1); */
4585 /* fflush(stdout); */
4586 if( rc
==SQLITE_OK
){
4590 sqliteFree(newCell
);
4595 ** Delete the entry that the cursor is pointing to. The cursor
4596 ** is left pointing at a random location.
4598 int sqlite3BtreeDelete(BtCursor
*pCur
){
4599 MemPage
*pPage
= pCur
->pPage
;
4600 unsigned char *pCell
;
4603 Btree
*pBt
= pCur
->pBt
;
4605 assert( pPage
->isInit
);
4606 if( pBt
->inTrans
!=TRANS_WRITE
){
4607 /* Must start a transaction before doing a delete */
4608 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
4610 assert( !pBt
->readOnly
);
4611 if( pCur
->idx
>= pPage
->nCell
){
4612 return SQLITE_ERROR
; /* The cursor is not pointing to anything */
4614 if( !pCur
->wrFlag
){
4615 return SQLITE_PERM
; /* Did not open this cursor for writing */
4617 if( checkReadLocks(pBt
, pCur
->pgnoRoot
, pCur
) ){
4618 return SQLITE_LOCKED
; /* The table pCur points to has a read lock */
4620 rc
= sqlite3pager_write(pPage
->aData
);
4623 /* Locate the cell within it's page and leave pCell pointing to the
4624 ** data. The clearCell() call frees any overflow pages associated with the
4625 ** cell. The cell itself is still intact.
4627 pCell
= findCell(pPage
, pCur
->idx
);
4629 pgnoChild
= get4byte(pCell
);
4631 rc
= clearCell(pPage
, pCell
);
4636 ** The entry we are about to delete is not a leaf so if we do not
4637 ** do something we will leave a hole on an internal page.
4638 ** We have to fill the hole by moving in a cell from a leaf. The
4639 ** next Cell after the one to be deleted is guaranteed to exist and
4640 ** to be a leaf so we can use it.
4643 unsigned char *pNext
;
4646 unsigned char *tempCell
= 0;
4647 assert( !pPage
->leafData
);
4648 getTempCursor(pCur
, &leafCur
);
4649 rc
= sqlite3BtreeNext(&leafCur
, ¬Used
);
4650 if( rc
!=SQLITE_OK
){
4651 if( rc
!=SQLITE_NOMEM
){
4652 rc
= SQLITE_CORRUPT
; /* bkpt-CORRUPT */
4655 if( rc
==SQLITE_OK
){
4656 rc
= sqlite3pager_write(leafCur
.pPage
->aData
);
4658 if( rc
==SQLITE_OK
){
4659 TRACE(("DELETE: table=%d delete internal from %d replace from leaf %d\n",
4660 pCur
->pgnoRoot
, pPage
->pgno
, leafCur
.pPage
->pgno
));
4661 dropCell(pPage
, pCur
->idx
, cellSizePtr(pPage
, pCell
));
4662 pNext
= findCell(leafCur
.pPage
, leafCur
.idx
);
4663 szNext
= cellSizePtr(leafCur
.pPage
, pNext
);
4664 assert( MX_CELL_SIZE(pBt
)>=szNext
+4 );
4665 tempCell
= sqliteMallocRaw( MX_CELL_SIZE(pBt
) );
4670 if( rc
==SQLITE_OK
){
4671 rc
= insertCell(pPage
, pCur
->idx
, pNext
-4, szNext
+4, tempCell
, 0);
4673 if( rc
==SQLITE_OK
){
4674 put4byte(findOverflowCell(pPage
, pCur
->idx
), pgnoChild
);
4675 rc
= balance(pPage
, 0);
4677 if( rc
==SQLITE_OK
){
4678 dropCell(leafCur
.pPage
, leafCur
.idx
, szNext
);
4679 rc
= balance(leafCur
.pPage
, 0);
4681 sqliteFree(tempCell
);
4682 releaseTempCursor(&leafCur
);
4684 TRACE(("DELETE: table=%d delete from leaf %d\n",
4685 pCur
->pgnoRoot
, pPage
->pgno
));
4686 dropCell(pPage
, pCur
->idx
, cellSizePtr(pPage
, pCell
));
4687 rc
= balance(pPage
, 0);
4689 if( rc
==SQLITE_OK
){
4696 ** Create a new BTree table. Write into *piTable the page
4697 ** number for the root page of the new table.
4699 ** The type of type is determined by the flags parameter. Only the
4700 ** following values of flags are currently in use. Other values for
4701 ** flags might not work:
4703 ** BTREE_INTKEY|BTREE_LEAFDATA Used for SQL tables with rowid keys
4704 ** BTREE_ZERODATA Used for SQL indices
4706 int sqlite3BtreeCreateTable(Btree
*pBt
, int *piTable
, int flags
){
4710 if( pBt
->inTrans
!=TRANS_WRITE
){
4711 /* Must start a transaction first */
4712 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
4714 assert( !pBt
->readOnly
);
4716 /* It is illegal to create a table if any cursors are open on the
4717 ** database. This is because in auto-vacuum mode the backend may
4718 ** need to move a database page to make room for the new root-page.
4719 ** If an open cursor was using the page a problem would occur.
4722 return SQLITE_LOCKED
;
4725 #ifdef SQLITE_OMIT_AUTOVACUUM
4726 rc
= allocatePage(pBt
, &pRoot
, &pgnoRoot
, 1, 0);
4729 if( pBt
->autoVacuum
){
4730 Pgno pgnoMove
; /* Move a page here to make room for the root-page */
4731 MemPage
*pPageMove
; /* The page to move to. */
4733 /* Read the value of meta[3] from the database to determine where the
4734 ** root page of the new table should go. meta[3] is the largest root-page
4735 ** created so far, so the new root-page is (meta[3]+1).
4737 rc
= sqlite3BtreeGetMeta(pBt
, 4, &pgnoRoot
);
4738 if( rc
!=SQLITE_OK
) return rc
;
4741 /* The new root-page may not be allocated on a pointer-map page, or the
4742 ** PENDING_BYTE page.
4744 if( pgnoRoot
==PTRMAP_PAGENO(pBt
->usableSize
, pgnoRoot
) ||
4745 pgnoRoot
==PENDING_BYTE_PAGE(pBt
) ){
4748 assert( pgnoRoot
>=3 );
4750 /* Allocate a page. The page that currently resides at pgnoRoot will
4751 ** be moved to the allocated page (unless the allocated page happens
4752 ** to reside at pgnoRoot).
4754 rc
= allocatePage(pBt
, &pPageMove
, &pgnoMove
, pgnoRoot
, 1);
4755 if( rc
!=SQLITE_OK
){
4759 if( pgnoMove
!=pgnoRoot
){
4763 releasePage(pPageMove
);
4764 rc
= getPage(pBt
, pgnoRoot
, &pRoot
);
4765 if( rc
!=SQLITE_OK
){
4768 rc
= ptrmapGet(pBt
, pgnoRoot
, &eType
, &iPtrPage
);
4769 if( rc
!=SQLITE_OK
|| eType
==PTRMAP_ROOTPAGE
|| eType
==PTRMAP_FREEPAGE
){
4773 assert( eType
!=PTRMAP_ROOTPAGE
);
4774 assert( eType
!=PTRMAP_FREEPAGE
);
4775 rc
= sqlite3pager_write(pRoot
->aData
);
4776 if( rc
!=SQLITE_OK
){
4780 rc
= relocatePage(pBt
, pRoot
, eType
, iPtrPage
, pgnoMove
);
4782 if( rc
!=SQLITE_OK
){
4785 rc
= getPage(pBt
, pgnoRoot
, &pRoot
);
4786 if( rc
!=SQLITE_OK
){
4789 rc
= sqlite3pager_write(pRoot
->aData
);
4790 if( rc
!=SQLITE_OK
){
4798 /* Update the pointer-map and meta-data with the new root-page number. */
4799 rc
= ptrmapPut(pBt
, pgnoRoot
, PTRMAP_ROOTPAGE
, 0);
4804 rc
= sqlite3BtreeUpdateMeta(pBt
, 4, pgnoRoot
);
4811 rc
= allocatePage(pBt
, &pRoot
, &pgnoRoot
, 1, 0);
4815 assert( sqlite3pager_iswriteable(pRoot
->aData
) );
4816 zeroPage(pRoot
, flags
| PTF_LEAF
);
4817 sqlite3pager_unref(pRoot
->aData
);
4818 *piTable
= (int)pgnoRoot
;
4823 ** Erase the given database page and all its children. Return
4824 ** the page to the freelist.
4826 static int clearDatabasePage(
4827 Btree
*pBt
, /* The BTree that contains the table */
4828 Pgno pgno
, /* Page number to clear */
4829 MemPage
*pParent
, /* Parent page. NULL for the root */
4830 int freePageFlag
/* Deallocate page if true */
4834 unsigned char *pCell
;
4837 if( pgno
>sqlite3pager_pagecount(pBt
->pPager
) ){
4838 return SQLITE_CORRUPT
;
4841 rc
= getAndInitPage(pBt
, pgno
, &pPage
, pParent
);
4842 if( rc
) goto cleardatabasepage_out
;
4843 rc
= sqlite3pager_write(pPage
->aData
);
4844 if( rc
) goto cleardatabasepage_out
;
4845 for(i
=0; i
<pPage
->nCell
; i
++){
4846 pCell
= findCell(pPage
, i
);
4848 rc
= clearDatabasePage(pBt
, get4byte(pCell
), pPage
->pParent
, 1);
4849 if( rc
) goto cleardatabasepage_out
;
4851 rc
= clearCell(pPage
, pCell
);
4852 if( rc
) goto cleardatabasepage_out
;
4855 rc
= clearDatabasePage(pBt
, get4byte(&pPage
->aData
[8]), pPage
->pParent
, 1);
4856 if( rc
) goto cleardatabasepage_out
;
4859 rc
= freePage(pPage
);
4861 zeroPage(pPage
, pPage
->aData
[0] | PTF_LEAF
);
4864 cleardatabasepage_out
:
4870 ** Delete all information from a single table in the database. iTable is
4871 ** the page number of the root of the table. After this routine returns,
4872 ** the root page is empty, but still exists.
4874 ** This routine will fail with SQLITE_LOCKED if there are any open
4875 ** read cursors on the table. Open write cursors are moved to the
4876 ** root of the table.
4878 int sqlite3BtreeClearTable(Btree
*pBt
, int iTable
){
4881 if( pBt
->inTrans
!=TRANS_WRITE
){
4882 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
4884 for(pCur
=pBt
->pCursor
; pCur
; pCur
=pCur
->pNext
){
4885 if( pCur
->pgnoRoot
==(Pgno
)iTable
){
4886 if( pCur
->wrFlag
==0 ) return SQLITE_LOCKED
;
4890 rc
= clearDatabasePage(pBt
, (Pgno
)iTable
, 0, 0);
4892 sqlite3BtreeRollback(pBt
);
4898 ** Erase all information in a table and add the root of the table to
4899 ** the freelist. Except, the root of the principle table (the one on
4900 ** page 1) is never added to the freelist.
4902 ** This routine will fail with SQLITE_LOCKED if there are any open
4903 ** cursors on the table.
4905 ** If AUTOVACUUM is enabled and the page at iTable is not the last
4906 ** root page in the database file, then the last root page
4907 ** in the database file is moved into the slot formerly occupied by
4908 ** iTable and that last slot formerly occupied by the last root page
4909 ** is added to the freelist instead of iTable. In this say, all
4910 ** root pages are kept at the beginning of the database file, which
4911 ** is necessary for AUTOVACUUM to work right. *piMoved is set to the
4912 ** page number that used to be the last root page in the file before
4913 ** the move. If no page gets moved, *piMoved is set to 0.
4914 ** The last root page is recorded in meta[3] and the value of
4915 ** meta[3] is updated by this procedure.
4917 int sqlite3BtreeDropTable(Btree
*pBt
, int iTable
, int *piMoved
){
4921 if( pBt
->inTrans
!=TRANS_WRITE
){
4922 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
4925 /* It is illegal to drop a table if any cursors are open on the
4926 ** database. This is because in auto-vacuum mode the backend may
4927 ** need to move another root-page to fill a gap left by the deleted
4928 ** root page. If an open cursor was using this page a problem would
4932 return SQLITE_LOCKED
;
4935 rc
= getPage(pBt
, (Pgno
)iTable
, &pPage
);
4937 rc
= sqlite3BtreeClearTable(pBt
, iTable
);
4946 #ifdef SQLITE_OMIT_AUTOVACUUM
4947 rc
= freePage(pPage
);
4950 if( pBt
->autoVacuum
){
4952 rc
= sqlite3BtreeGetMeta(pBt
, 4, &maxRootPgno
);
4953 if( rc
!=SQLITE_OK
){
4958 if( iTable
==maxRootPgno
){
4959 /* If the table being dropped is the table with the largest root-page
4960 ** number in the database, put the root page on the free list.
4962 rc
= freePage(pPage
);
4964 if( rc
!=SQLITE_OK
){
4968 /* The table being dropped does not have the largest root-page
4969 ** number in the database. So move the page that does into the
4970 ** gap left by the deleted root-page.
4974 rc
= getPage(pBt
, maxRootPgno
, &pMove
);
4975 if( rc
!=SQLITE_OK
){
4978 rc
= relocatePage(pBt
, pMove
, PTRMAP_ROOTPAGE
, 0, iTable
);
4980 if( rc
!=SQLITE_OK
){
4983 rc
= getPage(pBt
, maxRootPgno
, &pMove
);
4984 if( rc
!=SQLITE_OK
){
4987 rc
= freePage(pMove
);
4989 if( rc
!=SQLITE_OK
){
4992 *piMoved
= maxRootPgno
;
4995 /* Set the new 'max-root-page' value in the database header. This
4996 ** is the old value less one, less one more if that happens to
4997 ** be a root-page number, less one again if that is the
4998 ** PENDING_BYTE_PAGE.
5001 if( maxRootPgno
==PENDING_BYTE_PAGE(pBt
) ){
5004 if( maxRootPgno
==PTRMAP_PAGENO(pBt
->usableSize
, maxRootPgno
) ){
5007 assert( maxRootPgno
!=PENDING_BYTE_PAGE(pBt
) );
5009 rc
= sqlite3BtreeUpdateMeta(pBt
, 4, maxRootPgno
);
5011 rc
= freePage(pPage
);
5016 /* If sqlite3BtreeDropTable was called on page 1. */
5017 zeroPage(pPage
, PTF_INTKEY
|PTF_LEAF
);
5025 ** Read the meta-information out of a database file. Meta[0]
5026 ** is the number of free pages currently in the database. Meta[1]
5027 ** through meta[15] are available for use by higher layers. Meta[0]
5028 ** is read-only, the others are read/write.
5030 ** The schema layer numbers meta values differently. At the schema
5031 ** layer (and the SetCookie and ReadCookie opcodes) the number of
5032 ** free pages is not visible. So Cookie[0] is the same as Meta[1].
5034 int sqlite3BtreeGetMeta(Btree
*pBt
, int idx
, u32
*pMeta
){
5038 assert( idx
>=0 && idx
<=15 );
5039 rc
= sqlite3pager_get(pBt
->pPager
, 1, (void**)&pP1
);
5041 *pMeta
= get4byte(&pP1
[36 + idx
*4]);
5042 sqlite3pager_unref(pP1
);
5044 /* If autovacuumed is disabled in this build but we are trying to
5045 ** access an autovacuumed database, then make the database readonly.
5047 #ifdef SQLITE_OMIT_AUTOVACUUM
5048 if( idx
==4 && *pMeta
>0 ) pBt
->readOnly
= 1;
5055 ** Write meta-information back into the database. Meta[0] is
5056 ** read-only and may not be written.
5058 int sqlite3BtreeUpdateMeta(Btree
*pBt
, int idx
, u32 iMeta
){
5061 assert( idx
>=1 && idx
<=15 );
5062 if( pBt
->inTrans
!=TRANS_WRITE
){
5063 return pBt
->readOnly
? SQLITE_READONLY
: SQLITE_ERROR
;
5065 assert( pBt
->pPage1
!=0 );
5066 pP1
= pBt
->pPage1
->aData
;
5067 rc
= sqlite3pager_write(pP1
);
5069 put4byte(&pP1
[36 + idx
*4], iMeta
);
5074 ** Return the flag byte at the beginning of the page that the cursor
5075 ** is currently pointing to.
5077 int sqlite3BtreeFlags(BtCursor
*pCur
){
5078 MemPage
*pPage
= pCur
->pPage
;
5079 return pPage
? pPage
->aData
[pPage
->hdrOffset
] : 0;
5084 ** Print a disassembly of the given page on standard output. This routine
5085 ** is used for debugging and testing only.
5087 static int btreePageDump(Btree
*pBt
, int pgno
, int recursive
, MemPage
*pParent
){
5096 unsigned char *data
;
5098 unsigned char payload
[20];
5100 rc
= getPage(pBt
, (Pgno
)pgno
, &pPage
);
5101 isInit
= pPage
->isInit
;
5102 if( pPage
->isInit
==0 ){
5103 initPage(pPage
, pParent
);
5108 hdr
= pPage
->hdrOffset
;
5109 data
= pPage
->aData
;
5111 pPage
->intKey
= (c
& (PTF_INTKEY
|PTF_LEAFDATA
))!=0;
5112 pPage
->zeroData
= (c
& PTF_ZERODATA
)!=0;
5113 pPage
->leafData
= (c
& PTF_LEAFDATA
)!=0;
5114 pPage
->leaf
= (c
& PTF_LEAF
)!=0;
5115 pPage
->hasData
= !(pPage
->zeroData
|| (!pPage
->leaf
&& pPage
->leafData
));
5116 nCell
= get2byte(&data
[hdr
+3]);
5117 sqlite3DebugPrintf("PAGE %d: flags=0x%02x frag=%d parent=%d\n", pgno
,
5118 data
[hdr
], data
[hdr
+7],
5119 (pPage
->isInit
&& pPage
->pParent
) ? pPage
->pParent
->pgno
: 0);
5120 assert( hdr
== (pgno
==1 ? 100 : 0) );
5121 idx
= hdr
+ 12 - pPage
->leaf
*4;
5122 for(i
=0; i
<nCell
; i
++){
5125 unsigned char *pCell
;
5129 addr
= get2byte(&data
[idx
+ 2*i
]);
5130 pCell
= &data
[addr
];
5131 parseCellPtr(pPage
, pCell
, &info
);
5133 sprintf(range
,"%d..%d", addr
, addr
+sz
-1);
5137 child
= get4byte(pCell
);
5140 if( !pPage
->intKey
) sz
+= info
.nKey
;
5141 if( sz
>sizeof(payload
)-1 ) sz
= sizeof(payload
)-1;
5142 memcpy(payload
, &pCell
[info
.nHeader
], sz
);
5143 for(j
=0; j
<sz
; j
++){
5144 if( payload
[j
]<0x20 || payload
[j
]>0x7f ) payload
[j
] = '.';
5148 "cell %2d: i=%-10s chld=%-4d nk=%-4lld nd=%-4d payload=%s\n",
5149 i
, range
, child
, info
.nKey
, info
.nData
, payload
5153 sqlite3DebugPrintf("right_child: %d\n", get4byte(&data
[hdr
+8]));
5157 idx
= get2byte(&data
[hdr
+1]);
5158 while( idx
>0 && idx
<pPage
->pBt
->usableSize
){
5159 int sz
= get2byte(&data
[idx
+2]);
5160 sprintf(range
,"%d..%d", idx
, idx
+sz
-1);
5162 sqlite3DebugPrintf("freeblock %2d: i=%-10s size=%-4d total=%d\n",
5163 i
, range
, sz
, nFree
);
5164 idx
= get2byte(&data
[idx
]);
5168 sqlite3DebugPrintf("ERROR: next freeblock index out of range: %d\n", idx
);
5170 if( recursive
&& !pPage
->leaf
){
5171 for(i
=0; i
<nCell
; i
++){
5172 unsigned char *pCell
= findCell(pPage
, i
);
5173 btreePageDump(pBt
, get4byte(pCell
), 1, pPage
);
5174 idx
= get2byte(pCell
);
5176 btreePageDump(pBt
, get4byte(&data
[hdr
+8]), 1, pPage
);
5178 pPage
->isInit
= isInit
;
5179 sqlite3pager_unref(data
);
5183 int sqlite3BtreePageDump(Btree
*pBt
, int pgno
, int recursive
){
5184 return btreePageDump(pBt
, pgno
, recursive
, 0);
5190 ** Fill aResult[] with information about the entry and page that the
5191 ** cursor is pointing to.
5193 ** aResult[0] = The page number
5194 ** aResult[1] = The entry number
5195 ** aResult[2] = Total number of entries on this page
5196 ** aResult[3] = Cell size (local payload + header)
5197 ** aResult[4] = Number of free bytes on this page
5198 ** aResult[5] = Number of free blocks on the page
5199 ** aResult[6] = Total payload size (local + overflow)
5200 ** aResult[7] = Header size in bytes
5201 ** aResult[8] = Local payload size
5202 ** aResult[9] = Parent page number
5204 ** This routine is used for testing and debugging only.
5206 int sqlite3BtreeCursorInfo(BtCursor
*pCur
, int *aResult
, int upCnt
){
5208 MemPage
*pPage
= pCur
->pPage
;
5211 pageIntegrity(pPage
);
5212 assert( pPage
->isInit
);
5213 getTempCursor(pCur
, &tmpCur
);
5215 moveToParent(&tmpCur
);
5217 pPage
= tmpCur
.pPage
;
5218 pageIntegrity(pPage
);
5219 aResult
[0] = sqlite3pager_pagenumber(pPage
->aData
);
5220 assert( aResult
[0]==pPage
->pgno
);
5221 aResult
[1] = tmpCur
.idx
;
5222 aResult
[2] = pPage
->nCell
;
5223 if( tmpCur
.idx
>=0 && tmpCur
.idx
<pPage
->nCell
){
5224 getCellInfo(&tmpCur
);
5225 aResult
[3] = tmpCur
.info
.nSize
;
5226 aResult
[6] = tmpCur
.info
.nData
;
5227 aResult
[7] = tmpCur
.info
.nHeader
;
5228 aResult
[8] = tmpCur
.info
.nLocal
;
5235 aResult
[4] = pPage
->nFree
;
5237 idx
= get2byte(&pPage
->aData
[pPage
->hdrOffset
+1]);
5238 while( idx
>0 && idx
<pPage
->pBt
->usableSize
){
5240 idx
= get2byte(&pPage
->aData
[idx
]);
5243 if( pPage
->pParent
==0 || isRootPage(pPage
) ){
5246 aResult
[9] = pPage
->pParent
->pgno
;
5248 releaseTempCursor(&tmpCur
);
5254 ** Return the pager associated with a BTree. This routine is used for
5255 ** testing and debugging only.
5257 Pager
*sqlite3BtreePager(Btree
*pBt
){
5262 ** This structure is passed around through all the sanity checking routines
5263 ** in order to keep track of some global state information.
5265 typedef struct IntegrityCk IntegrityCk
;
5266 struct IntegrityCk
{
5267 Btree
*pBt
; /* The tree being checked out */
5268 Pager
*pPager
; /* The associated pager. Also accessible by pBt->pPager */
5269 int nPage
; /* Number of pages in the database */
5270 int *anRef
; /* Number of times each page is referenced */
5271 STRPTR zErrMsg
; /* An error message. NULL of no errors seen. */
5274 #ifndef SQLITE_OMIT_INTEGRITY_CHECK
5276 ** Append a message to the error message string.
5278 static void checkAppendMsg(
5279 IntegrityCk
*pCheck
,
5281 CONST_STRPTR zFormat
,
5287 va_start(ap
, zFormat
);
5288 zMsg2
= sqlite3VMPrintf(zFormat
, ap
);
5290 if( zMsg1
==0 ) zMsg1
= "";
5291 if( pCheck
->zErrMsg
)
5293 STRPTR zOld
= pCheck
->zErrMsg
;
5294 pCheck
->zErrMsg
= NULL
;
5295 sqlite3SetString(&pCheck
->zErrMsg
, zOld
, "\n", zMsg1
, zMsg2
, NULL
);
5298 else sqlite3SetString(&pCheck
->zErrMsg
, zMsg1
, zMsg2
, NULL
);
5301 #endif /* SQLITE_OMIT_INTEGRITY_CHECK */
5303 #ifndef SQLITE_OMIT_INTEGRITY_CHECK
5305 ** Add 1 to the reference count for page iPage. If this is the second
5306 ** reference to the page, add an error message to pCheck->zErrMsg.
5307 ** Return 1 if there are 2 ore more references to the page and 0 if
5308 ** if this is the first reference to the page.
5310 ** Also check that the page number is in bounds.
5312 static int checkRef(IntegrityCk
*pCheck
, int iPage
, char *zContext
){
5313 if( iPage
==0 ) return 1;
5314 if( iPage
>pCheck
->nPage
|| iPage
<0 ){
5315 checkAppendMsg(pCheck
, zContext
, "invalid page number %d", iPage
);
5318 if( pCheck
->anRef
[iPage
]==1 ){
5319 checkAppendMsg(pCheck
, zContext
, "2nd reference to page %d", iPage
);
5322 return (pCheck
->anRef
[iPage
]++)>1;
5325 #ifndef SQLITE_OMIT_AUTOVACUUM
5327 ** Check that the entry in the pointer-map for page iChild maps to
5328 ** page iParent, pointer type ptrType. If not, append an error message
5331 static void checkPtrmap(
5332 IntegrityCk
*pCheck
, /* Integrity check context */
5333 Pgno iChild
, /* Child page number */
5334 u8 eType
, /* Expected pointer map type */
5335 Pgno iParent
, /* Expected pointer map parent page number */
5336 char *zContext
/* Context description (used for error msg) */
5342 rc
= ptrmapGet(pCheck
->pBt
, iChild
, &ePtrmapType
, &iPtrmapParent
);
5343 if( rc
!=SQLITE_OK
){
5344 checkAppendMsg(pCheck
, zContext
, "Failed to read ptrmap key=%d", iChild
);
5348 if( ePtrmapType
!=eType
|| iPtrmapParent
!=iParent
){
5349 checkAppendMsg(pCheck
, zContext
,
5350 "Bad ptr map entry key=%d expected=(%d,%d) got=(%d,%d)",
5351 iChild
, eType
, iParent
, ePtrmapType
, iPtrmapParent
);
5357 ** Check the integrity of the freelist or of an overflow page list.
5358 ** Verify that the number of pages on the list is N.
5360 static void checkList(
5361 IntegrityCk
*pCheck
, /* Integrity checking context */
5362 int isFreeList
, /* True for a freelist. False for overflow page list */
5363 int iPage
, /* Page number for first page in the list */
5364 int N
, /* Expected number of pages in the list */
5365 char *zContext
/* Context for error messages */
5371 unsigned char *pOvfl
;
5373 checkAppendMsg(pCheck
, zContext
,
5374 "%d of %d pages missing from overflow list starting at %d",
5375 N
+1, expected
, iFirst
);
5378 if( checkRef(pCheck
, iPage
, zContext
) ) break;
5379 if( sqlite3pager_get(pCheck
->pPager
, (Pgno
)iPage
, (void**)&pOvfl
) ){
5380 checkAppendMsg(pCheck
, zContext
, "failed to get page %d", iPage
);
5384 int n
= get4byte(&pOvfl
[4]);
5385 #ifndef SQLITE_OMIT_AUTOVACUUM
5386 if( pCheck
->pBt
->autoVacuum
){
5387 checkPtrmap(pCheck
, iPage
, PTRMAP_FREEPAGE
, 0, zContext
);
5390 if( n
>pCheck
->pBt
->usableSize
/4-8 ){
5391 checkAppendMsg(pCheck
, zContext
,
5392 "freelist leaf count too big on page %d", iPage
);
5396 Pgno iFreePage
= get4byte(&pOvfl
[8+i
*4]);
5397 #ifndef SQLITE_OMIT_AUTOVACUUM
5398 if( pCheck
->pBt
->autoVacuum
){
5399 checkPtrmap(pCheck
, iFreePage
, PTRMAP_FREEPAGE
, 0, zContext
);
5402 checkRef(pCheck
, iFreePage
, zContext
);
5407 #ifndef SQLITE_OMIT_AUTOVACUUM
5409 /* If this database supports auto-vacuum and iPage is not the last
5410 ** page in this overflow list, check that the pointer-map entry for
5411 ** the following page matches iPage.
5413 if( pCheck
->pBt
->autoVacuum
&& N
>0 ){
5414 i
= get4byte(pOvfl
);
5415 checkPtrmap(pCheck
, i
, PTRMAP_OVERFLOW2
, iPage
, zContext
);
5419 iPage
= get4byte(pOvfl
);
5420 sqlite3pager_unref(pOvfl
);
5423 #endif /* SQLITE_OMIT_INTEGRITY_CHECK */
5425 #ifndef SQLITE_OMIT_INTEGRITY_CHECK
5427 ** Do various sanity checks on a single page of a tree. Return
5428 ** the tree depth. Root pages return 0. Parents of root pages
5429 ** return 1, and so forth.
5431 ** These checks are done:
5433 ** 1. Make sure that cells and freeblocks do not overlap
5434 ** but combine to completely cover the page.
5435 ** NO 2. Make sure cell keys are in order.
5436 ** NO 3. Make sure no key is less than or equal to zLowerBound.
5437 ** NO 4. Make sure no key is greater than or equal to zUpperBound.
5438 ** 5. Check the integrity of overflow pages.
5439 ** 6. Recursively call checkTreePage on all children.
5440 ** 7. Verify that the depth of all children is the same.
5441 ** 8. Make sure this page is at least 33% full or else it is
5442 ** the root of the tree.
5444 static int checkTreePage(
5445 IntegrityCk
*pCheck
, /* Context for the sanity check */
5446 int iPage
, /* Page number of the page to check */
5447 MemPage
*pParent
, /* Parent page */
5448 char *zParentContext
, /* Parent context */
5449 char *zLowerBound
, /* All keys should be greater than this, if not NULL */
5450 int nLower
, /* Number of characters in zLowerBound */
5451 char *zUpperBound
, /* All keys should be less than this, if not NULL */
5452 int nUpper
/* Number of characters in zUpperBound */
5455 int i
, rc
, depth
, d2
, pgno
, cnt
;
5461 int maxLocal
, usableSize
;
5465 sprintf(zContext
, "Page %d: ", iPage
);
5467 /* Check that the page exists
5469 cur
.pBt
= pBt
= pCheck
->pBt
;
5470 usableSize
= pBt
->usableSize
;
5471 if( iPage
==0 ) return 0;
5472 if( checkRef(pCheck
, iPage
, zParentContext
) ) return 0;
5473 if( (rc
= getPage(pBt
, (Pgno
)iPage
, &pPage
))!=0 ){
5474 checkAppendMsg(pCheck
, zContext
,
5475 "unable to get the page. error code=%d", rc
);
5478 maxLocal
= pPage
->leafData
? pBt
->maxLeaf
: pBt
->maxLocal
;
5479 if( (rc
= initPage(pPage
, pParent
))!=0 ){
5480 checkAppendMsg(pCheck
, zContext
, "initPage() returns error code %d", rc
);
5485 /* Check out all the cells.
5489 for(i
=0; i
<pPage
->nCell
; i
++){
5494 /* Check payload overflow pages
5496 sprintf(zContext
, "On tree page %d cell %d: ", iPage
, i
);
5497 pCell
= findCell(pPage
,i
);
5498 parseCellPtr(pPage
, pCell
, &info
);
5500 if( !pPage
->intKey
) sz
+= info
.nKey
;
5501 if( sz
>info
.nLocal
){
5502 int nPage
= (sz
- info
.nLocal
+ usableSize
- 5)/(usableSize
- 4);
5503 Pgno pgnoOvfl
= get4byte(&pCell
[info
.iOverflow
]);
5504 #ifndef SQLITE_OMIT_AUTOVACUUM
5505 if( pBt
->autoVacuum
){
5506 checkPtrmap(pCheck
, pgnoOvfl
, PTRMAP_OVERFLOW1
, iPage
, zContext
);
5509 checkList(pCheck
, 0, pgnoOvfl
, nPage
, zContext
);
5512 /* Check sanity of left child page.
5515 pgno
= get4byte(pCell
);
5516 #ifndef SQLITE_OMIT_AUTOVACUUM
5517 if( pBt
->autoVacuum
){
5518 checkPtrmap(pCheck
, pgno
, PTRMAP_BTREE
, iPage
, zContext
);
5521 d2
= checkTreePage(pCheck
,pgno
,pPage
,zContext
,0,0,0,0);
5522 if( i
>0 && d2
!=depth
){
5523 checkAppendMsg(pCheck
, zContext
, "Child page depth differs");
5529 pgno
= get4byte(&pPage
->aData
[pPage
->hdrOffset
+8]);
5530 sprintf(zContext
, "On page %d at right child: ", iPage
);
5531 #ifndef SQLITE_OMIT_AUTOVACUUM
5532 if( pBt
->autoVacuum
){
5533 checkPtrmap(pCheck
, pgno
, PTRMAP_BTREE
, iPage
, 0);
5536 checkTreePage(pCheck
, pgno
, pPage
, zContext
,0,0,0,0);
5539 /* Check for complete coverage of the page
5541 data
= pPage
->aData
;
5542 hdr
= pPage
->hdrOffset
;
5543 hit
= sqliteMalloc( usableSize
);
5545 memset(hit
, 1, get2byte(&data
[hdr
+5]));
5546 nCell
= get2byte(&data
[hdr
+3]);
5547 cellStart
= hdr
+ 12 - 4*pPage
->leaf
;
5548 for(i
=0; i
<nCell
; i
++){
5549 int pc
= get2byte(&data
[cellStart
+i
*2]);
5550 int size
= cellSizePtr(pPage
, &data
[pc
]);
5552 if( (pc
+size
-1)>=usableSize
|| pc
<0 ){
5553 checkAppendMsg(pCheck
, 0,
5554 "Corruption detected in cell %d on page %d",i
,iPage
,0);
5556 for(j
=pc
+size
-1; j
>=pc
; j
--) hit
[j
]++;
5559 for(cnt
=0, i
=get2byte(&data
[hdr
+1]); i
>0 && i
<usableSize
&& cnt
<10000;
5561 int size
= get2byte(&data
[i
+2]);
5563 if( (i
+size
-1)>=usableSize
|| i
<0 ){
5564 checkAppendMsg(pCheck
, 0,
5565 "Corruption detected in cell %d on page %d",i
,iPage
,0);
5567 for(j
=i
+size
-1; j
>=i
; j
--) hit
[j
]++;
5569 i
= get2byte(&data
[i
]);
5571 for(i
=cnt
=0; i
<usableSize
; i
++){
5574 }else if( hit
[i
]>1 ){
5575 checkAppendMsg(pCheck
, 0,
5576 "Multiple uses for byte %d of page %d", i
, iPage
);
5580 if( cnt
!=data
[hdr
+7] ){
5581 checkAppendMsg(pCheck
, 0,
5582 "Fragmented space is %d byte reported as %d on page %d",
5583 cnt
, data
[hdr
+7], iPage
);
5591 #endif /* SQLITE_OMIT_INTEGRITY_CHECK */
5593 #ifndef SQLITE_OMIT_INTEGRITY_CHECK
5595 ** This routine does a complete check of the given BTree file. aRoot[] is
5596 ** an array of pages numbers were each page number is the root page of
5597 ** a table. nRoot is the number of entries in aRoot.
5599 ** If everything checks out, this routine returns NULL. If something is
5600 ** amiss, an error message is written into memory obtained from malloc()
5601 ** and a pointer to that error message is returned. The calling function
5602 ** is responsible for freeing the error message when it is done.
5604 char *sqlite3BtreeIntegrityCheck(Btree
*pBt
, int *aRoot
, int nRoot
){
5609 nRef
= *sqlite3pager_stats(pBt
->pPager
);
5610 if( lockBtreeWithRetry(pBt
)!=SQLITE_OK
){
5611 return sqliteStrDup("Unable to acquire a read lock on the database");
5614 sCheck
.pPager
= pBt
->pPager
;
5615 sCheck
.nPage
= sqlite3pager_pagecount(sCheck
.pPager
);
5616 if( sCheck
.nPage
==0 ){
5617 unlockBtreeIfUnused(pBt
);
5620 sCheck
.anRef
= sqliteMallocRaw( (sCheck
.nPage
+1)*sizeof(sCheck
.anRef
[0]) );
5621 if( !sCheck
.anRef
){
5622 unlockBtreeIfUnused(pBt
);
5623 return sqlite3MPrintf("Unable to malloc %d bytes",
5624 (sCheck
.nPage
+1)*sizeof(sCheck
.anRef
[0]));
5626 for(i
=0; i
<=sCheck
.nPage
; i
++){ sCheck
.anRef
[i
] = 0; }
5627 i
= PENDING_BYTE_PAGE(pBt
);
5628 if( i
<=sCheck
.nPage
){
5629 sCheck
.anRef
[i
] = 1;
5633 /* Check the integrity of the freelist
5635 checkList(&sCheck
, 1, get4byte(&pBt
->pPage1
->aData
[32]),
5636 get4byte(&pBt
->pPage1
->aData
[36]), "Main freelist: ");
5638 /* Check all the tables.
5640 for(i
=0; i
<nRoot
; i
++){
5641 if( aRoot
[i
]==0 ) continue;
5642 #ifndef SQLITE_OMIT_AUTOVACUUM
5643 if( pBt
->autoVacuum
&& aRoot
[i
]>1 ){
5644 checkPtrmap(&sCheck
, aRoot
[i
], PTRMAP_ROOTPAGE
, 0, 0);
5647 checkTreePage(&sCheck
, aRoot
[i
], 0, "List of tree roots: ", 0,0,0,0);
5650 /* Make sure every page in the file is referenced
5652 for(i
=1; i
<=sCheck
.nPage
; i
++){
5653 #ifdef SQLITE_OMIT_AUTOVACUUM
5654 if( sCheck
.anRef
[i
]==0 ){
5655 checkAppendMsg(&sCheck
, 0, "Page %d is never used", i
);
5658 /* If the database supports auto-vacuum, make sure no tables contain
5659 ** references to pointer-map pages.
5661 if( sCheck
.anRef
[i
]==0 &&
5662 (PTRMAP_PAGENO(pBt
->usableSize
, i
)!=i
|| !pBt
->autoVacuum
) ){
5663 checkAppendMsg(&sCheck
, 0, "Page %d is never used", i
);
5665 if( sCheck
.anRef
[i
]!=0 &&
5666 (PTRMAP_PAGENO(pBt
->usableSize
, i
)==i
&& pBt
->autoVacuum
) ){
5667 checkAppendMsg(&sCheck
, 0, "Pointer map page %d is referenced", i
);
5672 /* Make sure this analysis did not leave any unref() pages
5674 unlockBtreeIfUnused(pBt
);
5675 if( nRef
!= *sqlite3pager_stats(pBt
->pPager
) ){
5676 checkAppendMsg(&sCheck
, 0,
5677 "Outstanding page count goes from %d to %d during this analysis",
5678 nRef
, *sqlite3pager_stats(pBt
->pPager
)
5682 /* Clean up and report errors.
5684 sqliteFree(sCheck
.anRef
);
5685 return sCheck
.zErrMsg
;
5687 #endif /* SQLITE_OMIT_INTEGRITY_CHECK */
5690 ** Return the full pathname of the underlying database file.
5692 const char *sqlite3BtreeGetFilename(Btree
*pBt
){
5693 assert( pBt
->pPager
!=0 );
5694 return sqlite3pager_filename(pBt
->pPager
);
5698 ** Return the pathname of the directory that contains the database file.
5700 const char *sqlite3BtreeGetDirname(Btree
*pBt
){
5701 assert( pBt
->pPager
!=0 );
5702 return sqlite3pager_dirname(pBt
->pPager
);
5706 ** Return the pathname of the journal file for this database. The return
5707 ** value of this routine is the same regardless of whether the journal file
5708 ** has been created or not.
5710 const char *sqlite3BtreeGetJournalname(Btree
*pBt
){
5711 assert( pBt
->pPager
!=0 );
5712 return sqlite3pager_journalname(pBt
->pPager
);
5715 #ifndef SQLITE_OMIT_VACUUM
5717 ** Copy the complete content of pBtFrom into pBtTo. A transaction
5718 ** must be active for both files.
5720 ** The size of file pBtFrom may be reduced by this operation.
5721 ** If anything goes wrong, the transaction on pBtFrom is rolled back.
5723 int sqlite3BtreeCopyFile(Btree
*pBtTo
, Btree
*pBtFrom
){
5725 Pgno i
, nPage
, nToPage
;
5727 if( pBtTo
->inTrans
!=TRANS_WRITE
|| pBtFrom
->inTrans
!=TRANS_WRITE
){
5728 return SQLITE_ERROR
;
5730 if( pBtTo
->pCursor
) return SQLITE_BUSY
;
5731 nToPage
= sqlite3pager_pagecount(pBtTo
->pPager
);
5732 nPage
= sqlite3pager_pagecount(pBtFrom
->pPager
);
5733 for(i
=1; rc
==SQLITE_OK
&& i
<=nPage
; i
++){
5735 rc
= sqlite3pager_get(pBtFrom
->pPager
, i
, &pPage
);
5737 rc
= sqlite3pager_overwrite(pBtTo
->pPager
, i
, pPage
);
5739 sqlite3pager_unref(pPage
);
5741 for(i
=nPage
+1; rc
==SQLITE_OK
&& i
<=nToPage
; i
++){
5743 rc
= sqlite3pager_get(pBtTo
->pPager
, i
, &pPage
);
5745 rc
= sqlite3pager_write(pPage
);
5746 sqlite3pager_unref(pPage
);
5747 sqlite3pager_dont_write(pBtTo
->pPager
, i
);
5749 if( !rc
&& nPage
<nToPage
){
5750 rc
= sqlite3pager_truncate(pBtTo
->pPager
, nPage
);
5753 sqlite3BtreeRollback(pBtTo
);
5757 #endif /* SQLITE_OMIT_VACUUM */
5760 ** Return non-zero if a transaction is active.
5762 int sqlite3BtreeIsInTrans(Btree
*pBt
){
5763 return (pBt
&& (pBt
->inTrans
==TRANS_WRITE
));
5767 ** Return non-zero if a statement transaction is active.
5769 int sqlite3BtreeIsInStmt(Btree
*pBt
){
5770 return (pBt
&& pBt
->inStmt
);
5774 ** This call is a no-op if no write-transaction is currently active on pBt.
5776 ** Otherwise, sync the database file for the btree pBt. zMaster points to
5777 ** the name of a master journal file that should be written into the
5778 ** individual journal file, or is NULL, indicating no master journal file
5779 ** (single database transaction).
5781 ** When this is called, the master journal should already have been
5782 ** created, populated with this journal pointer and synced to disk.
5784 ** Once this is routine has returned, the only thing required to commit
5785 ** the write-transaction for this database file is to delete the journal.
5787 int sqlite3BtreeSync(Btree
*pBt
, const char *zMaster
){
5788 if( pBt
->inTrans
==TRANS_WRITE
){
5789 #ifndef SQLITE_OMIT_AUTOVACUUM
5791 if( pBt
->autoVacuum
){
5792 int rc
= autoVacuumCommit(pBt
, &nTrunc
);
5793 if( rc
!=SQLITE_OK
) return rc
;
5795 return sqlite3pager_sync(pBt
->pPager
, zMaster
, nTrunc
);
5797 return sqlite3pager_sync(pBt
->pPager
, zMaster
, 0);
5802 #ifndef SQLITE_OMIT_GLOBALRECOVER
5804 ** Reset the btree and underlying pager after a malloc() failure. Any
5805 ** transaction that was active when malloc() failed is rolled back.
5807 int sqlite3BtreeReset(Btree
*pBt
){
5808 if( pBt
->pCursor
) return SQLITE_BUSY
;
5809 pBt
->inTrans
= TRANS_NONE
;
5810 unlockBtreeIfUnused(pBt
);
5811 return sqlite3pager_reset(pBt
->pPager
);