{
  "format": "promptkb-evidence-brief",
  "version": 2,
  "title": "SQLite for a team tool: share the service, not the live file",
  "question": "For a team planning a shared record-keeping tool, when do SQLite’s published documents support a server-local SQLite database, and when do they favor a client/server database engine over directly sharing one file? Separate deployment topology, writer concurrency, journal mode and online backups. This is a scoped documentation decision aid, not a workload benchmark, application audit or endorsement.",
  "sources": [
    {
      "id": "S1",
      "title": "Appropriate Uses For SQLite",
      "locator": "https://www.sqlite.org/whentouse.html — retrieved 2026-11-04T12:20:21.371879+00:00; captured document, not release-pinned; reading-text SHA-256 7da46702f746b0fd702778f58abfbb488b689b9de264c032211471a660180271",
      "text": "Appropriate Uses For SQLite\nTable Of Contents\n1. Situations Where SQLite Works Well\n2. Situations Where A Client/Server RDBMS May Work Better\n3. Checklist For Choosing The Right Database Engine\nSQLite is not directly comparable to client/server SQL database engines such\nas MySQL, Oracle, PostgreSQL, or SQL Server since SQLite is trying to\nsolve a different problem.\nClient/server SQL database engines strive to implement a shared repository\nof enterprise data. They emphasize scalability, concurrency, centralization,\nand control.\nSQLite strives to provide local data storage for\nindividual applications and devices. SQLite emphasizes economy,\nefficiency, reliability, independence, and simplicity.\nSQLite does not compete with client/server databases.\nSQLite competes with fopen().\n1. Situations Where SQLite Works Well\nEmbedded devices and the internet of things\nBecause an SQLite database requires no administration,\nit works well in devices that must operate without expert human support.\nSQLite is a good fit for use in\ncellphones, set-top boxes, televisions, game consoles,\ncameras, watches, kitchen appliances, thermostats, automobiles,\nmachine tools, airplanes, remote sensors, drones, medical devices,\nand robots: the \"internet of things\".\nClient/server database engines are designed to live inside a\nlovingly-attended datacenter at the core of the network.\nSQLite works there too, but SQLite also thrives at the edge of the network,\nfending for itself while providing fast and\nreliable data services to applications that would otherwise\nhave dodgy connectivity.\nApplication file format\nSQLite is often used as the on-disk file format\nfor desktop applications such as version control systems,\nfinancial analysis tools, media cataloging and editing suites, CAD\npackages, record keeping programs, and so forth. The traditional\nFile/Open operation calls sqlite3_open() to attach to the database\nfile. Updates happen automatically as application content is revised\nso the File/Save menu option becomes superfluous. The File/Save_As\nmenu option can be implemented using the backup API.\nThere are many benefits to this approach, including improved\nperformance, reduced cost and complexity, and\nimproved reliability. See technical notes\n\"aff_short.html\" and\n\"appfileformat.html\" and\n\"fasterthanfs.html\" for more information.\nThis use case is closely related to the\ndata transfer format and\ndata container use cases below.\nWebsites\nSQLite works great as the database engine for most low to\nmedium traffic websites (which is to say, most websites).\nThe amount of web traffic that SQLite can handle depends\non how heavily the website uses its database. Generally\nspeaking, any site that gets fewer than 100K hits/day should work\nfine with SQLite.\nThe 100K hits/day figure is a conservative estimate, not a\nhard upper bound.\nSQLite has been demonstrated to work with 10 times that amount\nof traffic.\nThe SQLite website (https://sqlite.org/) uses SQLite itself,\nof course, and as of this writing (2015) it handles about 400K to 500K\nHTTP requests per day, about 15-20% of which are dynamic pages touching\nthe database. Dynamic content uses about 200 SQL statements per webpage.\nThis setup runs on a single VM that shares a physical server with 23 others\nand yet still keeps the load average below 0.1 most of the time.\nSee also: Hacker News discussion from 2022-12-13.\nData analysis\nPeople who understand SQL can employ the\nsqlite3 command-line shell (or various third-party\nSQLite access programs) to analyze large\ndatasets. Raw data can be imported from CSV files, then that\ndata can be sliced and diced to generate a myriad of summary\nreports. More complex analysis can be done using simple scripts written\nin Tcl or Python (both of which come with SQLite built-in) or in R or\nother languages using readily available adaptors.\nPossible uses include website log analysis, sports\nstatistics analysis, compilation of programming metrics, and\nanalysis of experimental results. Many bioinformatics researchers\nuse SQLite in this way.\nThe same thing can be done with an enterprise client/server\ndatabase, of course. The advantage of SQLite is\nthat it is easier to install and use and the resulting database\nis a single file that can be written to a USB memory stick\nor emailed to a colleague.\nCache for enterprise data\nMany applications use SQLite as a cache of relevant content from\nan enterprise RDBMS.\nThis reduces latency, since most queries now occur against the local\ncache and avoid a network round-trip. It also reduces the load\non the network and on the central database server. And in many cases,\nit means that the client-side application can continue operating during\nnetwork outages.\nServer-side database\nSystems designers\nreport success using SQLite as a data store on server applications\nrunning in the datacenter, or in other words, using SQLite as the underlying\nstorage engine for an application-specific database server.\nWith this pattern, the overall system is still client/server:\nclients send requests to the server and get back replies over the network.\nBut instead of sending generic SQL and getting back raw table content,\nthe client requests and server responses are high-level and\napplication-specific.\nThe server translates requests into multiple SQL queries, gathers the\nresults, does post-processing, filtering, and analysis, then constructs\na high-level reply containing only the essential information.\nDevelopers report that SQLite is often faster than a client/server\nSQL database engine in this scenario.\nDatabase requests are serialized by the server, so concurrency is not\nan issue. Concurrency is also improved by \"database sharding\":\nusing separate database files for different subdomains. For\nexample, the server might have a separate SQLite database for each\nuser, so that the server can handle hundreds or thousands of simultaneous\nconnections, but each SQLite database is only used by one connection.\nData transfer format\nBecause an SQLite database is a single compact file in a\nwell-defined cross-platform format, it is often used\nas a container for transferring content from one system to another.\nThe sender gathers content into an SQLite database file, transfers\nthat one file to the receiver, then the receiver uses SQL to extract\nthe content as needed.\nAn SQLite database facilitates data transfer between systems even\nwhen the endpoints have different word sizes and/or byte orders.\nThe data can be a complex mix of large binary blobs, text, and small\nnumeric or boolean values. The data format can be easily extended\nby adding new tables and/or columns, without breaking legacy receivers.\nThe SQL query language means that receivers are not required to parse\nthe entire transfer all at once, but can instead query the\nreceived content as needed. The data format is \"transparent\" in the\nsense that it is easily decoded for human viewing using\na variety of universally available, open-source tools, from multiple\nvendors.\nFile archive and/or data container\nThe SQLite Archive idea shows how\nSQLite can be used as a substitute for ZIP archives or Tarballs.\nAn archive of files stored in SQLite is only very slightly larger, and\nin some cases actually smaller, than the equivalent ZIP archive.\nAnd an SQLite archive features incremental and atomic updating\nand the ability to store much richer metadata.\nFossil version 2.5 and later offers\nSQLite Archive files as a download format, in addition\nto traditional tarball and ZIP archive.\nThe sqlite3.exe command-line shell version 3.22.0 and later will create,\nlist, or unpack an SQL archive using the\n.archive command.\nSQLite is a good solution for any situation that requires bundling\ndiverse content into a self-contained and self-describing package\nfor shipment across a network.\nContent is encoded in a\nwell-defined, cross-platform, and stable file format.\nThe encoding is efficient, and receivers can extract small subsets\nof the content without having to read and parse the entire file.\nSQL archives are useful as the distribution format for software\nor content updates that are broadcast to many clients. Variations\non this idea are used, for example, to transmit TV programming guides\nto set-top boxes and to send over-the-air updates to vehicle navigation\nsystems.\nReplacement for ad hoc disk files\nMany programs use\nfopen(),\nfread(), and\nfwrite() to create and\nmanage files of data in home-grown formats. SQLite works\nparticularly well as a\nreplacement for these ad hoc data files.\nContrary to intuition, SQLite can be faster than the filesystem\nfor reading and writing content to disk.\nInternal or temporary databases\nFor programs that have a lot of data that must be sifted and sorted\nin diverse ways, it is often easier and quicker to load the data into\nan in-memory SQLite database and use queries with joins and ORDER BY\nclauses to extract the data in the form and order needed rather than\nto try to code the same operations manually.\nUsing an SQL database internally in this way also gives the program\ngreater flexibility since new columns and indices can be added without\nhaving to recode every query.\nStand-in for an enterprise database during demos or testing\nClient applications typically use a generic database interface that allows\nconnections to various SQL database engines. It makes good sense to\ninclude SQLite in the mix of supported databases and to statically\nlink the SQLite engine in with the client. That way the client program\ncan be used standalone with an SQLite data file for testing or for\ndemonstrations.\nEducation and Training\nBecause it is simple to setup and use (installation is trivial: just\ncopy the sqlite3 or sqlite3.exe executable to the target machine\nand run it) SQLite makes a good database engine for use in teaching SQL.\nStudents can easily create as many databases as they like and can\nemail databases to the instructor for comments or grading. For more\nadvanced students who are interested in studying how an RDBMS is\nimplemented, the modular and well-commented and documented SQLite code\ncan serve as a good basis.\nExperimental SQL language extensions\nThe simple, modular design of SQLite makes it a good platform for\nprototyping new, experimental database language features or ideas.\n2. Situations Where A Client/Server RDBMS May Work Better\nClient/Server Applications\nIf there are many client programs sending SQL to the same\ndatabase over a network, then use a client/server database\nengine instead of SQLite. SQLite will work over a network filesystem,\nbut because of the latency associated with most network filesystems,\nperformance will not be great. Also, file locking logic is buggy in\nmany network filesystem implementations (on both Unix and Windows).\nIf file locking does not work correctly,\ntwo or more clients might try to modify the\nsame part of the same database at the same time, resulting in\ncorruption. Because this problem results from bugs in\nthe underlying filesystem implementation, there is nothing SQLite\ncan do to prevent it.\nA good rule of thumb is to avoid using SQLite\nin situations where the same database will be accessed directly\n(without an intervening application server) and simultaneously\nfrom many computers over a network.\nHigh-volume Websites\nSQLite will normally work fine as the database backend to a website.\nBut if the website is write-intensive or is so busy that it requires\nmultiple servers, then consider using an enterprise-class client/server\ndatabase engine instead of SQLite.\nVery large datasets\nAn SQLite database is limited in size to 281 terabytes\n(2^(48) bytes, 256 tibibytes).\nAnd even if it could handle larger databases, SQLite stores the entire\ndatabase in a single disk file and many filesystems limit the maximum\nsize of files to something less than this. So if you are contemplating\ndatabases of this magnitude, you would do well to consider using a\nclient/server database engine that spreads its content across multiple\ndisk files, and perhaps across multiple volumes.\nHigh Concurrency\nSQLite supports an unlimited number of simultaneous readers, but it\nwill only allow one writer at any instant in time.\nFor many situations, this is not a problem. Writers queue up. Each application\ndoes its database work quickly and moves on, and no lock lasts for more\nthan a few dozen milliseconds. But there are some applications that require\nmore concurrency, and those applications may need to seek a different\nsolution.\n3. Checklist For Choosing The Right Database Engine\nIs the data separated from the application by a network?\n→ choose client/server\nRelational database engines act as bandwidth-reducing data filters.\nSo it is best to keep the database engine and the data on\nthe same physical device so that the high-bandwidth engine-to-disk\nlink does not have to traverse the network, only the lower-bandwidth\napplication-to-engine link.\nBut SQLite is built into the application. So if the data is on a\nseparate device from the application, it is required that the higher\nbandwidth engine-to-disk link be across the network. This works, but\nit is suboptimal. Hence, it is usually better to select a client/server\ndatabase engine when the data is on a separate device from the\napplication.\nNota Bene:\nIn this rule, \"application\" means the code that issues SQL statements.\nIf the \"application\" is an application server and\nif the content resides on the same physical machine as the application server,\nthen SQLite might still be appropriate even though the end user is\nanother network hop away.\nMany concurrent writers? → choose client/server\nIf many threads and/or processes need to write the\ndatabase at the same instant (and they cannot queue up and take turns)\nthen it is best to select a database engine that supports that\ncapability, which always means a client/server database engine.\nSQLite only supports one writer at a time per database file.\nBut in most cases, a write transaction only takes milliseconds and\nso multiple writers can simply take turns. SQLite will handle\nmore write concurrency than many people suspect. Nevertheless,\nclient/server database systems, because they have a long-running\nserver process at hand to coordinate access, can usually handle\nfar more write concurrency than SQLite ever will.\nBig data? → choose client/server\nIf your data will grow to a size that you are uncomfortable\nor unable to fit into a single disk file, then you should select\na solution other than SQLite. SQLite supports databases up to\n281 terabytes in size, assuming you can find a disk drive and filesystem\nthat will support 281-terabyte files. Even so, when the size of the\ncontent looks like it might creep into the terabyte range, it would\nbe good to consider a centralized client/server database.\nOtherwise → choose SQLite!\nFor device-local storage with low writer concurrency and less than a\nterabyte of content, SQLite is almost always a better solution. SQLite\nis fast and reliable and it requires no configuration or maintenance.\nIt keeps things simple. SQLite \"just works\".\nThis page was last updated on 2025-05-31 13:08:22Z\n",
      "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
    },
    {
      "id": "S2",
      "title": "SQLite file locking — rollback-mode scope",
      "locator": "https://www.sqlite.org/lockingv3.html — retrieved 2026-11-04T12:20:21.459437+00:00; captured document, not release-pinned; reading-text SHA-256 2c78d8dada32d8d001dd41cb581cd68fade6a336ab1a06ded7ccf0c2ce099649",
      "text": "This document was originally created in early 2004 when SQLite version 2\nwas still in widespread use and was written to introduce\nthe new concepts of SQLite version 3 to readers who were already familiar\nwith SQLite version 2. But these days, most readers of this document have\nprobably never seen SQLite version 2 and are only familiar with SQLite\nversion 3. Nevertheless, this document continues to serve as an\nauthoritative reference to how database file locking works in SQLite\nversion 3.\nThe document only describes locking for the older rollback-mode\ntransaction mechanism. Locking for the newer write-ahead log or WAL mode\nis described separately.\n1.0 File Locking And Concurrency In SQLite Version 3\nSQLite Version 3.0.0 introduced a new locking and journaling\nmechanism designed to improve concurrency over SQLite version 2\nand to reduce the writer starvation\nproblem. The new mechanism also allows atomic commits of transactions\ninvolving multiple database files.\nThis document describes the new locking mechanism.\nThe intended audience is programmers who want to understand and/or modify\nthe pager code and reviewers working to verify the design\nof SQLite version 3.\n2.0 Overview\nLocking and concurrency control are handled by the\npager module.\nThe pager module is responsible for making SQLite \"ACID\" (Atomic,\nConsistent, Isolated, and Durable). The pager module makes sure changes\nhappen all at once, that either all changes occur or none of them do,\nthat two or more processes do not try to access the database\nin incompatible ways at the same time, and that once changes have been\nwritten they persist until explicitly deleted. The pager also provides\na memory cache of some of the contents of the disk file.\nThe pager is unconcerned\nwith the details of B-Trees, text encodings, indices, and so forth.\nFrom the point of view of the pager the database consists of\na single file of uniform-sized blocks. Each block is called a\n\"page\" and is usually 1024 bytes in size. The pages are numbered\nbeginning with 1. So the first 1024 bytes of the database are called\n\"page 1\" and the second 1024 bytes are call \"page 2\" and so forth. All\nother encoding details are handled by higher layers of the library.\nThe pager communicates with the operating system using one of several\nmodules\n(Examples:\nos_unix.c,\nos_win.c)\nthat provides a uniform abstraction for operating system services.\nThe pager module effectively controls access for separate threads, or\nseparate processes, or both. Throughout this document whenever the\nword \"process\" is written you may substitute the word \"thread\" without\nchanging the truth of the statement.\n3.0 Locking\nFrom the point of view of a single process, a database file\ncan be in one of five locking states:\nUNLOCKED\nNo locks are held on the database. The database may be neither read nor\nwritten. Any internally cached data is considered suspect and subject to\nverification against the database file before being used. Other\nprocesses can read or write the database as their own locking states\npermit. This is the default state.\nSHARED\nThe database may be read but not written. Any number of\nprocesses can hold SHARED locks at the same time, hence there can be\nmany simultaneous readers. But no other thread or process is allowed\nto write to the database file while one or more SHARED locks are active.\nRESERVED\nA RESERVED lock means that the process is planning on writing to the\ndatabase file at some point in the future but that it is currently just\nreading from the file. Only a single RESERVED lock may be active at one\ntime, though multiple SHARED locks can coexist with a single RESERVED lock.\nRESERVED differs from PENDING in that new SHARED locks can be acquired\nwhile there is a RESERVED lock.\nPENDING\nA PENDING lock means that the process holding the lock wants to write\nto the database as soon as possible and is just waiting on all current\nSHARED locks to clear so that it can get an EXCLUSIVE lock. No new\nSHARED locks are permitted against the database if\na PENDING lock is active, though existing SHARED locks are allowed to\ncontinue.\nEXCLUSIVE\nAn EXCLUSIVE lock is needed in order to write to the database file.\nOnly one EXCLUSIVE lock is allowed on the file and no other locks of\nany kind are allowed to coexist with an EXCLUSIVE lock. In order to\nmaximize concurrency, SQLite works to minimize the amount of time that\nEXCLUSIVE locks are held.\nThe operating system interface layer understands and tracks all five\nlocking states described above.\nThe pager module only tracks four of the five locking states.\nA PENDING lock is always just a temporary\nstepping stone on the path to an EXCLUSIVE lock and so the pager module\ndoes not track PENDING locks.\n4.0 The Rollback Journal\nWhen a process wants to change a database file (and it is not\nin WAL mode), it\nfirst records the original unchanged database content\nin a rollback journal. The rollback journal is an ordinary\ndisk file that is always located\nin the same directory or folder as the database file and has the\nsame name as the database file with the addition of a -journal\nsuffix. The rollback journal also records the initial\nsize of the database so that if the database file grows it can be truncated\nback to its original size on a rollback.\nIf SQLite is working with multiple databases at the same time\n(using the ATTACH command) then each database has its own rollback journal.\nBut there is also a separate aggregate journal\ncalled the super-journal.\nThe super-journal does not contain page data used for rolling back\nchanges. Instead the super-journal contains the names of the\nindividual database rollback journals for each of the ATTACHed databases.\nEach of the individual database rollback journals also contain the name\nof the super-journal.\nIf there are no ATTACHed databases (or if none of the ATTACHed database\nis participating in the current transaction) no super-journal is\ncreated and the normal rollback journal contains an empty string\nin the place normally reserved for recording the name of the\nsuper-journal.\nA rollback journal is said to be hot\nif it needs to be rolled back\nin order to restore the integrity of its database.\nA hot journal is created when a process is in the middle of a database\nupdate and a program or operating system crash or power failure prevents\nthe update from completing.\nHot journals are an exception condition.\nHot journals exist to recover from crashes and power failures.\nIf everything is working correctly\n(that is, if there are no crashes or power failures)\nyou will never get a hot journal.\nIf no super-journal is involved, then\na journal is hot if it exists and has a non-zero header\nand its corresponding database file\ndoes not have a RESERVED lock.\nIf a super-journal is named in the file journal, then the file journal\nis hot if its super-journal exists and there is no RESERVED\nlock on the corresponding database file.\nIt is important to understand when a journal is hot so the\npreceding rules will be repeated in bullets:\nA journal is hot if...\nIt exists, and\nIts size is greater than 512 bytes, and\nThe journal header is non-zero and well-formed, and\nIts super-journal exists or the super-journal name is an\nempty string, and\nThere is no RESERVED lock on the corresponding database file.\n4.1 Dealing with hot journals\nBefore reading from a database file, SQLite always checks to see if that\ndatabase file has a hot journal. If the file does have a hot journal, then\nthe journal is rolled back before the file is read. In this way, we ensure\nthat the database file is in a consistent state before it is read.\nWhen a process wants to read from a database file, it follows\nthe following sequence of steps:\nOpen the database file and obtain a SHARED lock. If the SHARED lock\ncannot be obtained, fail immediately and return SQLITE_BUSY.\nCheck to see if the database file has a hot journal. If the file\ndoes not have a hot journal, we are done. Return immediately.\nIf there is a hot journal, that journal must be rolled back by\nthe subsequent steps of this algorithm.\nAcquire a PENDING lock then an EXCLUSIVE lock on the database file.\n(Note: Do not acquire a RESERVED lock because that would make\nother processes think the journal was no longer hot.) If we\nfail to acquire these locks it means another process\nis already trying to do the rollback. In that case,\ndrop all locks, close the database, and return SQLITE_BUSY.\nRead the journal file and roll back the changes.\nWait for the rolled back changes to be written onto\npersistent storage. This protects the integrity of the database\nin case another power failure or crash occurs.\nDelete the journal file (or truncate the journal to zero bytes in\nlength if PRAGMA journal_mode=TRUNCATE is\nset, or zero the journal header if\nPRAGMA journal_mode=PERSIST is set).\nDelete the super-journal file if it is safe to do so.\nThis step is optional. It is here only to prevent stale\nsuper-journals from cluttering up the disk drive.\nSee the discussion below for details.\nDrop the EXCLUSIVE and PENDING locks but retain the SHARED lock.\nAfter the algorithm above completes successfully, it is safe to\nread from the database file. Once all reading has completed, the\nSHARED lock is dropped.\n4.2 Deleting stale super-journals\nA stale super-journal is a super-journal that is no longer being\nused for anything. There is no requirement that stale super-journals\nbe deleted. The only reason for doing so is to free up disk space.\nA super-journal is stale if no individual file journals are pointing\nto it. To figure out if a super-journal is stale, we first read the\nsuper-journal to obtain the names of all of its file journals. Then\nwe check each of those file journals. If any of the file journals named\nin the super-journal exists and points back to the super-journal, then\nthe super-journal is not stale. If all file journals are either missing\nor refer to other super-journals or no super-journal at all, then the\nsuper-journal we are testing is stale and can be safely deleted.\n5.0 Writing to a database file\nTo write to a database, a process must first acquire a SHARED lock\nas described above (possibly rolling back incomplete changes if there\nis a hot journal).\nAfter a SHARED lock is obtained, a RESERVED lock must be acquired.\nThe RESERVED lock signals that the process intends to write to the\ndatabase at some point in the future. Only one process at a time\ncan hold a RESERVED lock. But other processes can continue to read\nthe database while the RESERVED lock is held.\nIf the process that wants to write is unable to obtain a RESERVED\nlock, it must mean that another process already has a RESERVED lock.\nIn that case, the write attempt fails and returns SQLITE_BUSY.\nAfter obtaining a RESERVED lock, the process that wants to write\ncreates a rollback journal. The header of the journal is initialized\nwith the original size of the database file. Space in the journal header\nis also reserved for a super-journal name, though the super-journal\nname is initially empty.\nBefore making changes to any page of the database, the process writes\nthe original content of that page into the rollback journal. Changes\nto pages are held in memory at first and are not written to the disk.\nThe original database file remains unaltered, which means that other\nprocesses can continue to read the database.\nEventually, the writing process will want to update the database\nfile, either because its memory cache has filled up or because it is\nready to commit its changes. Before this happens, the writer must\nmake sure no other process is reading the database and that the rollback\njournal data is safely on the disk surface so that it can be used to\nrollback incomplete changes in the event of a power failure.\nThe steps are as follows:\nMake sure all rollback journal data has actually been written to\nthe surface of the disk (and is not just being held in the operating\nsystem's or disk controllers cache) so that if a power failure occurs\nthe data will still be there after power is restored.\nObtain a PENDING lock and then an EXCLUSIVE lock on the database file.\nIf other processes still have SHARED locks, the writer might have\nto wait until those SHARED locks clear before it is able to obtain\nan EXCLUSIVE lock.\nWrite all page modifications currently held in memory out to the\noriginal database disk file.\nIf the reason for writing to the database file is because the memory\ncache was full, then the writer will not commit right away. Instead,\nthe writer might continue to make changes to other pages. Before\nsubsequent changes are written to the database file, the rollback\njournal must be flushed to disk again. Note also that the EXCLUSIVE\nlock that the writer obtained in order to write to the database initially\nmust be held until all changes are committed. That means that no other\nprocesses are able to access the database from the\ntime the memory cache first spills to disk until the transaction\ncommits.\nWhen a writer is ready to commit its changes, it executes the following\nsteps:\nObtain an EXCLUSIVE lock on the database file and\nmake sure all memory changes have been written to the database file\nusing the algorithm of steps 1-3 above.\nFlush all database file changes to the disk. Wait for those changes\nto actually be written onto the disk surface.\nDelete the journal file. (Or if the PRAGMA journal_mode is TRUNCATE or\nPERSIST, truncate the journal file or zero the header of the journal file,\nrespectively.) This is the instant when the changes are\ncommitted. Prior to deleting the journal file, if a power failure\nor crash occurs, the next process to open the database will see that\nit has a hot journal and will roll the changes back.\nAfter the journal is deleted, there will no longer be a hot journal\nand the changes will persist.\nDrop the EXCLUSIVE and PENDING locks from the database file.\nAs soon as the PENDING lock is released from the database file, other\nprocesses can begin reading the database again. In the current implementation,\nthe RESERVED lock is also released, but that is not essential for\ncorrect operation.\nIf a transaction involves multiple databases, then a more complex\ncommit sequence is used, as follows:\nMake sure all individual database files have an EXCLUSIVE lock and a\nvalid journal.\nCreate a super-journal. The name of the super-journal is arbitrary.\n(The current implementation appends random suffixes to the name of the\nmain database file until it finds a name that does not previously exist.)\nFill the super-journal with the names of all the individual journals\nand flush its contents to disk.\nWrite the name of the super-journal into\nall individual journals (in space set aside for that purpose in the\nheaders of the individual journals) and flush the contents of the\nindividual journals to disk and wait for those changes to reach the\ndisk surface.\nFlush all database file changes to the disk. Wait for those changes\nto actually be written onto the disk surface.\nDelete the super-journal file. This is the instant when the changes are\ncommitted. Prior to deleting the super-journal file, if a power failure\nor crash occurs, the individual file journals will be considered hot\nand will be rolled back by the next process that\nattempts to read them. After the super-journal has been deleted,\nthe file journals will no longer be considered hot and the changes\nwill persist.\nDelete all individual journal files.\nDrop the EXCLUSIVE and PENDING locks from all database files.\n5.1 Writer starvation\nIn SQLite version 2, if many processes are reading from the database,\nit might be the case that there is never a time when there are\nno active readers. And if there is always at least one read lock on the\ndatabase, no process would ever be able to make changes to the database\nbecause it would be impossible to acquire a write lock. This situation\nis called writer starvation.\nSQLite version 3 seeks to avoid writer starvation through the use of\nthe PENDING lock. The PENDING lock allows existing readers to continue\nbut prevents new readers from connecting to the database. So when a\nprocess wants to write a busy database, it can set a PENDING lock which\nwill prevent new readers from coming in. Assuming existing readers do\neventually complete, all SHARED locks will eventually clear and the\nwriter will be given a chance to make its changes.\n6.0 How To Corrupt Your Database Files\nThe pager module is very robust but it can be subverted.\nThis section attempts to identify and explain the risks.\n(See also the Things That Can Go Wrong section of the article\non Atomic Commit.\nClearly, a hardware or operating system fault that introduces incorrect data\ninto the middle of the database file or journal will cause problems.\nLikewise,\nif a rogue process opens a database file or journal and writes malformed\ndata into the middle of it, then the database will become corrupt.\nThere is not much that can be done about these kinds of problems\nso they are given no further attention.\nSQLite uses POSIX advisory locks to implement locking on Unix. On\nWindows it uses the LockFile(), LockFileEx(), and UnlockFile() system\ncalls. SQLite assumes that these system calls all work as advertised. If\nthat is not the case, then database corruption can result. One should\nnote that POSIX advisory locking is known to be buggy or even unimplemented\non many NFS implementations (including recent versions of Mac OS X)\nand that there are reports of locking problems\nfor network filesystems under Windows. Your best defense is to not\nuse SQLite for files on a network filesystem.\nSQLite uses the fsync() system call to flush data to the disk under Unix and\nit uses the FlushFileBuffers() to do the same under Windows. Once again,\nSQLite assumes that these operating system services function as advertised.\nBut it has been reported that fsync() and FlushFileBuffers() do not always\nwork correctly, especially with some network filesystems or inexpensive IDE disks.\nApparently some manufacturers of IDE disks have controller chips that report\nthat data has reached the disk surface when in fact the data is still\nin volatile cache memory in the disk drive electronics. There are also\nreports that Windows sometimes chooses to ignore FlushFileBuffers() for\nunspecified reasons. The author cannot verify any of these reports.\nBut if they are true, it means that database corruption is a possibility\nfollowing an unexpected power loss. These are hardware and/or operating\nsystem bugs that SQLite is unable to defend against.\nIf a Linux ext3\nfilesystem is mounted without the \"barrier=1\" option\nin the /etc/fstab\nand the disk drive write cache is enabled\nthen filesystem corruption can occur following a power loss or OS crash.\nWhether or not corruption can occur depends on the details of the disk control\nhardware; corruption is more likely with inexpensive consumer-grade disks\nand less of a problem for enterprise-class storage devices with advanced\nfeatures such as non-volatile write caches.\nVarious ext3 experts\nconfirm this behavior.\nWe are told that most Linux distributions do not use barrier=1 and do\nnot disable the write cache so most\nLinux distributions are vulnerable to this problem. Note that this is an\noperating system and hardware issue and that there is nothing that SQLite\ncan do to work around it.\nOther database engines have also run into this same problem.\nIf a crash or power failure occurs and results in a hot journal but that\njournal is deleted, the next process to open the database will not\nknow that it contains changes that need to be rolled back. The rollback\nwill not occur and the database will be left in an inconsistent state.\nRollback journals might be deleted for any number of reasons:\nAn administrator might be cleaning up after an OS crash or power failure,\nsee the journal file, think it is junk, and delete it.\nSomeone (or some process) might rename the database file but fail to\nalso rename its associated journal.\nIf the database file has aliases (hard or soft links) and the file\nis opened by a different alias than the one used to create the journal,\nthen the journal will not be found. To avoid this problem, you should\nnot create links to SQLite database files.\nFilesystem corruption following a power failure might cause the\njournal to be renamed or deleted.\nThe last (fourth) bullet above merits additional comment. When SQLite creates\na journal file on Unix, it opens the directory that contains that file and\ncalls fsync() on the directory, in an effort to push the directory information\nto disk. But suppose some other process is adding or removing unrelated\nfiles to the directory that contains the database and journal at the\nmoment of a power failure. The supposedly unrelated actions of this other\nprocess might result in the journal file being dropped from the directory and\nmoved into \"lost+found\". This is an unlikely scenario, but it could happen.\nThe best defenses are to use a journaling filesystem or to keep the\ndatabase and journal in a directory by themselves.\nFor a commit involving multiple databases and a super-journal, if the\nvarious databases were on different disk volumes and a power failure occurs\nduring the commit, then when the machine comes back up the disks might\nbe remounted with different names. Or some disks might not be mounted\nat all. When this happens the individual file journals and the\nsuper-journal might not be able to find each other. The worst outcome from\nthis scenario is that the commit ceases to be atomic.\nSome databases might be rolled back and others might not.\nAll databases will continue to be self-consistent.\nTo defend against this problem, keep all databases\non the same disk volume and/or remount disks using exactly the same names\nafter a power failure.\n7.0 Transaction Control At The SQL Level\nThe changes to locking and concurrency control in SQLite version 3 also\nintroduce some subtle changes in the way transactions work at the SQL\nlanguage level.\nBy default, SQLite version 3 operates in autocommit mode.\nIn autocommit mode,\nall changes to the database are committed as soon as all operations associated\nwith the current database connection complete.\nThe SQL command \"BEGIN TRANSACTION\" (the TRANSACTION keyword\nis optional) is used to take SQLite out of autocommit mode.\nNote that the BEGIN command does not acquire any locks on the database.\nAfter a BEGIN command, a SHARED lock will be acquired when the first\nSELECT statement is executed. A RESERVED lock will be acquired when\nthe first INSERT, UPDATE, or DELETE statement is executed. No EXCLUSIVE\nlock is acquired until either the memory cache fills up and must\nbe spilled to disk or until the transaction commits. In this way,\nthe system delays blocking read access to the file until the\nlast possible moment.\nThe SQL command \"COMMIT\" does not actually commit the changes to\ndisk. It just turns autocommit back on. Then, at the conclusion of\nthe command, the regular autocommit logic takes over and causes the\nactual commit to disk to occur.\nThe SQL command \"ROLLBACK\" also operates by turning autocommit back on,\nbut it also sets a flag that tells the autocommit logic to rollback rather\nthan commit.\nIf the SQL COMMIT command turns autocommit on and the autocommit logic\nthen tries to commit change but fails because some other process is holding\na SHARED lock, then autocommit is turned back off automatically. This\nallows the user to retry the COMMIT at a later time after the SHARED lock\nhas had an opportunity to clear.\nThis page was last updated on 2025-05-31 13:08:22Z\n",
      "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
    },
    {
      "id": "S3",
      "title": "SQLite Write-Ahead Logging",
      "locator": "https://www.sqlite.org/wal.html — retrieved 2026-11-04T12:20:21.562425+00:00; captured document, not release-pinned; reading-text SHA-256 3a5d7be15c9c8c9542318722bd294cc7b2289d5771811cbe7e4eb655069c003f",
      "text": "Write-Ahead Logging\nTable Of Contents\n1. Overview\n2. How WAL Works\n2.1. Checkpointing\n2.2. Concurrency\n2.3. Performance Considerations\n3. Activating And Configuring WAL Mode\n3.1. Automatic Checkpoint\n3.2. Application-Initiated Checkpoints\n3.3. Persistence of WAL mode\n4. The WAL File\n5. Read-Only Databases\n6. Avoiding Excessively Large WAL Files\n7. Implementation Of Shared-Memory For The WAL-Index\n8. Use of WAL Without Shared-Memory\n9. Sometimes Queries Return SQLITE_BUSY In WAL Mode\n10. Backwards Compatibility\n11. The WAL-Reset Bug\n11.1. Bug Details\n11.2. Low Probability Of Occurrence\n1. Overview\nThe default method by which SQLite implements\natomic commit and rollback is a rollback journal.\nBeginning with version 3.7.0 (2010-07-21), a new \"Write-Ahead Log\" option\n(hereafter referred to as \"WAL\") is available.\nThere are advantages and disadvantages to using WAL instead of\na rollback journal. Advantages include:\nWAL is significantly faster in most scenarios.\nWAL provides more concurrency as readers do not block writers and\na writer does not block readers. Reading and writing can proceed\nconcurrently.\nDisk I/O operations tends to be more sequential using WAL.\nWAL uses many fewer fsync() operations and is thus less vulnerable to\nproblems on systems where the fsync() system call is broken.\nBut there are also disadvantages:\nAll processes using a database must be on the same host computer;\nWAL does not work over a network filesystem. This is because WAL requires\nall processes to share a small amount of memory and processes on\nseparate host machines obviously cannot share memory with each other.\nTransactions that involve changes against multiple ATTACHed\ndatabases are atomic for each individual database, but are not\natomic across all databases as a set.\nIt is not possible to change the page_size after entering WAL\nmode, either on an empty database or by using VACUUM or by restoring\nfrom a backup using the backup API. You must be in a rollback journal\nmode to change the page size.\n[SOURCE STRIKETHROUGH START] It is not possible to open read-only WAL databases.\nThe opening process must have write privileges for \"-shm\"\nwal-index shared memory file associated with the database, if that\nfile exists, or else write access on the directory containing\nthe database file if the \"-shm\" file does not exist. [SOURCE STRIKETHROUGH END]\nBeginning with version 3.22.0 (2018-01-22), a read-only\nWAL-mode database file can be opened if\nthe -shm and -wal files\nalready exist or those files can be created or the\ndatabase is immutable.\nWAL might be very slightly slower (perhaps 1% or 2% slower)\nthan the traditional rollback-journal approach\nin applications that do mostly reads and seldom write.\nThere is an additional quasi-persistent \"-wal\" file and\n\"-shm\" shared memory file associated with each\ndatabase, which can make SQLite less appealing for use as an\napplication file-format.\nThere is the extra operation of checkpointing which, though automatic\nby default, is still something that application developers need to\nbe mindful of.\n[SOURCE STRIKETHROUGH START] WAL works best with smaller transactions. WAL does\nnot work well for very large transactions. For transactions larger than\nabout 100 megabytes, traditional rollback journal modes will likely\nbe faster. For transactions in excess of a gigabyte, WAL mode may\nfail with an I/O or disk-full error.\nIt is recommended that one of the rollback journal modes be used for\ntransactions larger than a few dozen megabytes. [SOURCE STRIKETHROUGH END]\nBeginning with version 3.11.0 (2016-02-15),\nWAL mode works as efficiently with\nlarge transactions as does rollback mode.\n2. How WAL Works\nThe traditional rollback journal works by writing a copy of the\noriginal unchanged database content into a separate rollback journal file\nand then writing changes directly into the database file. In the\nevent of a crash or ROLLBACK, the original content contained in the\nrollback journal is played back into the database file to\nrevert the database file to its original state. The COMMIT occurs\nwhen the rollback journal is deleted.\nThe WAL approach inverts this. The original content is preserved\nin the database file and the changes are appended into a separate\nWAL file. A COMMIT occurs when a special record indicating a commit\nis appended to the WAL. Thus a COMMIT can happen without ever writing\nto the original database, which allows readers to continue operating\nfrom the original unaltered database while changes are simultaneously being\ncommitted into the WAL. Multiple transactions can be appended to the\nend of a single WAL file.\n2.1. Checkpointing\nOf course, one wants to eventually transfer all the transactions that\nare appended in the WAL file back into the original database. Moving\nthe WAL file transactions back into the database is called a\n\"checkpoint\".\nAnother way to think about the difference between rollback and\nwrite-ahead log is that in the rollback-journal\napproach, there are two primitive operations, reading and writing,\nwhereas with a write-ahead log\nthere are now three primitive operations: reading, writing, and\ncheckpointing.\nBy default, SQLite does a checkpoint automatically when the WAL file\nreaches a threshold size of 1000 pages. (The\nSQLITE_DEFAULT_WAL_AUTOCHECKPOINT compile-time option can be used to\nspecify a different default.) Applications using WAL do\nnot have to do anything in order for these checkpoints to occur.\nBut if they want to, applications can adjust the automatic checkpoint\nthreshold. Or they can turn off the automatic checkpoints and run\ncheckpoints during idle moments or in a separate thread or process.\n2.2. Concurrency\nWhen a read operation begins on a WAL-mode database, it first\nremembers the location of the last valid commit record in the WAL.\nCall this point the \"end mark\". Because the WAL can be growing and\nadding new commit records while various readers connect to the database,\neach reader can potentially have its own end mark. But for any\nparticular reader, the end mark is unchanged for the duration of the\ntransaction, thus ensuring that a single read transaction only sees\nthe database content as it existed at a single point in time.\nWhen a reader needs a page of content, it first checks the WAL to\nsee if that page appears there, and if so it pulls in the last copy\nof the page that occurs in the WAL prior to the reader's end mark.\nIf no copy of the page exists in the WAL prior to the reader's end mark,\nthen the page is read from the original database file. Readers can\nexist in separate processes, so to avoid forcing every reader to scan\nthe entire WAL looking for pages (the WAL file can grow to\nmultiple megabytes, depending on how often checkpoints are run), a\ndata structure called the \"wal-index\" is maintained in shared memory\nwhich helps readers locate pages in the WAL quickly and with a minimum\nof I/O. The wal-index greatly improves the performance of readers,\nbut the use of shared memory means that all readers must exist on the\nsame machine. This is why the write-ahead log implementation will not\nwork on a network filesystem.\nWriters merely append new content to the end of the WAL file.\nBecause writers do nothing that would interfere with the actions of\nreaders, writers and readers can run at the same time. However,\nsince there is only one WAL file, there can only be one writer at\na time.\nA checkpoint operation takes content from the WAL file\nand transfers it back into the original database file.\nA checkpoint can run concurrently with readers, however the checkpoint\nmust stop when it reaches a page in the WAL that is past the end mark\nof any current reader. The checkpoint has to stop at that point because\notherwise it might overwrite part of the database file that the reader\nis actively using. The checkpoint remembers (in the wal-index) how far\nit got and will resume transferring content from the WAL to the database\nfrom where it left off on the next invocation.\nThus a long-running read transaction can prevent a checkpointer from\nmaking progress. But presumably every read transaction will eventually\nend and the checkpointer will be able to continue.\nWhenever a write operation occurs, the writer checks how much progress\nthe checkpointer has made, and if the entire WAL has been transferred into\nthe database and synced and if no readers are making use of the WAL, then\nthe writer will rewind the WAL back to the beginning and start putting new\ntransactions at the beginning of the WAL. This mechanism prevents a WAL\nfile from growing without bound.\n2.3. Performance Considerations\nWrite transactions are very fast since they only involve writing\nthe content once (versus twice for rollback-journal transactions)\nand because the writes are all sequential. Further, syncing the\ncontent to the disk is not required, as long as the application is\nwilling to sacrifice durability following a power loss or hard reboot.\n(Writers sync the WAL on every transaction commit if\nPRAGMA synchronous is set to FULL but omit this sync if\nPRAGMA synchronous is set to NORMAL.)\nOn the other hand, read performance deteriorates as the WAL file\ngrows in size since each reader must check the WAL file for the content\nand the time needed to check the WAL file is proportional\nto the size of the WAL file. The wal-index helps find content\nin the WAL file much faster, but performance still falls off with\nincreasing WAL file size. Hence, to maintain good read performance\nit is important to keep the WAL file size down by\nrunning checkpoints at regular intervals.\nCheckpointing does require sync operations in order to avoid\nthe possibility of database corruption following a power loss\nor hard reboot. The WAL must be synced to persistent storage\nprior to moving content from the WAL into the database and the\ndatabase file must be synced prior to resetting the WAL.\nCheckpoint also requires more seeking.\nThe checkpointer makes an effort to\ndo as many sequential page writes to the database as it can (the pages\nare transferred from WAL to database in ascending order) but even\nthen there will typically be many seek operations interspersed among\nthe page writes. These factors combine to make checkpoints slower than\nwrite transactions.\nThe default strategy is to allow successive write transactions to\ngrow the WAL until the WAL becomes about 1000 pages in size, then to\nrun a checkpoint operation for each subsequent COMMIT until the WAL\nis reset to be smaller than 1000 pages. By default, the checkpoint will be\nrun automatically by the same thread that does the COMMIT that pushes\nthe WAL over its size limit. This has the effect of causing most\nCOMMIT operations to be very fast but an occasional COMMIT (those that trigger\na checkpoint) to be much slower. If that effect is undesirable, then\nthe application can disable automatic checkpointing and run the\nperiodic checkpoints in a separate thread, or separate process.\n(Links to commands and interfaces to accomplish this are\nshown below.)\nNote that with PRAGMA synchronous set to NORMAL, the checkpoint\nis the only operation to issue an I/O barrier or sync operation\n(fsync() on unix or FlushFileBuffers() on windows). If an application\ntherefore runs checkpoint in a separate thread or process, the main\nthread or process that is doing database queries and updates will never\nblock on a sync operation. This helps to prevent \"latch-up\" in applications\nrunning on a busy disk drive. The downside to\nthis configuration is that transactions are no longer durable and\nmight rollback following a power failure or hard reset.\nNotice too that there is a tradeoff between average read performance\nand average write performance. To maximize the read performance,\none wants to keep the WAL as small as possible and hence run checkpoints\nfrequently, perhaps as often as every COMMIT. To maximize\nwrite performance, one wants to amortize the cost of each checkpoint\nover as many writes as possible, meaning that one wants to run checkpoints\ninfrequently and let the WAL grow as large as possible before each\ncheckpoint. The decision of how often to run checkpoints may therefore\nvary from one application to another depending on the relative read\nand write performance requirements of the application.\nThe default strategy is to run a checkpoint once the WAL\nreaches 1000 pages and this strategy seems to work well in test applications on\nworkstations, but other strategies might work better on different\nplatforms or for different workloads.\n3. Activating And Configuring WAL Mode\nAn SQLite database connection defaults to\njournal_mode=DELETE. To convert to WAL mode, use the\nfollowing pragma:\nPRAGMA journal_mode=WAL;\nThe journal_mode pragma returns a string which is the new journal mode.\nOn success, the pragma will return the string \"wal\". If\nthe conversion to WAL could not be completed (for example, if the VFS\ndoes not support the necessary shared-memory primitives) then the\njournaling mode will be unchanged and the string returned from the\nprimitive will be the prior journaling mode (for example \"delete\").\n3.1. Automatic Checkpoint\nBy default, SQLite will automatically checkpoint whenever a COMMIT\noccurs that causes the WAL file to be 1000 pages or more in size, or when the\nlast database connection on a database file closes. The default\nconfiguration is intended to work well for most applications.\nBut programs that want more control can force a checkpoint\nusing the wal_checkpoint pragma or by calling the\nsqlite3_wal_checkpoint() C interface. The automatic checkpoint\nthreshold can be changed or automatic checkpointing can be completely\ndisabled using the wal_autocheckpoint pragma or by calling the\nsqlite3_wal_autocheckpoint() C interface. A program can also\nuse sqlite3_wal_hook() to register a callback to be invoked whenever\nany transaction commits to the WAL. This callback can then invoke\nsqlite3_wal_checkpoint() or sqlite3_wal_checkpoint_v2() based on whatever\ncriteria it thinks is appropriate. (The automatic checkpoint mechanism\nis implemented as a simple wrapper around sqlite3_wal_hook().)\n3.2. Application-Initiated Checkpoints\nAn application can initiate a checkpoint using any writable database\nconnection on the database simply by invoking\nsqlite3_wal_checkpoint() or sqlite3_wal_checkpoint_v2().\nThere are three subtypes of checkpoints that vary in their aggressiveness:\nPASSIVE, FULL, and RESTART. The default checkpoint style is PASSIVE, which\ndoes as much work as it can without interfering with other database\nconnections, and which might not run to completion if there are\nconcurrent readers or writers.\nAll checkpoints initiated by sqlite3_wal_checkpoint() and\nby the automatic checkpoint mechanism are PASSIVE. FULL and RESTART\ncheckpoints try harder to run the checkpoint to completion and can only\nbe initiated by a call to sqlite3_wal_checkpoint_v2(). See the\nsqlite3_wal_checkpoint_v2() documentation for additional information\non FULL and RESET checkpoints.\n3.3. Persistence of WAL mode\nUnlike the other journaling modes,\nPRAGMA journal_mode=WAL is\npersistent. If a process sets WAL mode, then closes and reopens the\ndatabase, the database will come back in WAL mode. In contrast, if\na process sets (for example) PRAGMA journal_mode=TRUNCATE and then closes and\nreopens the database will come back up in the default rollback mode of\nDELETE rather than the previous TRUNCATE setting.\nThe persistence of WAL mode means that applications can be converted\nto using SQLite in WAL mode without making any changes to the application\nitself. One has merely to run \"PRAGMA journal_mode=WAL;\" on the\ndatabase file(s) using the command-line shell or other utility, then\nrestart the application.\nThe WAL journal mode will be set on all\nconnections to the same database file if it is set on any one connection.\n4. The WAL File\nWhile a database connection is open on a WAL-mode database, SQLite\nmaintains an extra journal file called a \"Write Ahead Log\" or \"WAL File\".\nThe name of this file on disk is usually the name of the database file\nwith an extra \"-wal\" suffix, though different naming rules may\napply if SQLite is compiled with SQLITE_ENABLE_8_3_NAMES.\nThe WAL file exists for as long as any database connection has the\ndatabase open. Usually, the WAL file is deleted automatically when the\nlast connection to the database closes. However, if the last process to\nhave the database open exits without cleanly\nshutting down the database connection, or if the\nSQLITE_FCNTL_PERSIST_WAL file control is used, then the WAL file\nmight be retained on disk after all connections to the database have\nbeen closed. The WAL file is part of the persistent state of the\ndatabase and should be kept with the database if the database is copied\nor moved. If a database file is separated from its WAL file, then\ntransactions that were previously committed to the database might be lost,\nor the database file might become corrupted.\nThe only safe way to remove a WAL file is\nto open the database file using one of the sqlite3_open() interfaces\nthen immediately close the database using sqlite3_close().\nThe WAL file format is precisely defined and is cross-platform.\n5. Read-Only Databases\nOlder versions of SQLite could not read a WAL-mode database that was\nread-only. In other words, write access was required in order to read a\nWAL-mode database. This constraint was relaxed beginning with\nSQLite version 3.22.0 (2018-01-22).\nOn newer versions of SQLite,\na WAL-mode database on read-only media, or a WAL-mode database that lacks\nwrite permission, can still be read as long as one or more of the following\nconditions are met:\nThe -shm and -wal files already exist and are readable.\nThere is write permission on the directory containing the database so\nthat the -shm and -wal files can be created.\nThe database connection is opened using the\nimmutable query parameter.\nEven though it is possible to open a read-only WAL-mode database,\nit is good practice to convert the database to\nPRAGMA journal_mode=DELETE prior to burning an\nSQLite database image onto read-only media.\n6. Avoiding Excessively Large WAL Files\nIn normal cases, new content is appended to the WAL file until the\nWAL file accumulates about 1000 pages (and is thus about 4MB\nin size) at which point a checkpoint is automatically run and the WAL file\nis recycled. The checkpoint does not normally truncate the WAL file\n(unless the journal_size_limit pragma is set). Instead, it merely\ncauses SQLite to start overwriting the WAL file from the beginning.\nThis is done because it is normally faster to overwrite an existing file\nthan to append. When the last connection to a database closes, that\nconnection does one last checkpoint and then deletes the WAL and its\nassociated shared-memory file, to clean up the disk.\nSo in the vast majority of cases, applications need not worry about\nthe WAL file at all. SQLite will automatically take care of it. But\nit is possible to get SQLite into a state where the WAL file will grow\nwithout bound, causing excess disk space usage and slow query speeds.\nThe following bullets enumerate some of the ways that this can happen\nand how to avoid them.\nDisabling the automatic checkpoint mechanism.\nIn its default configuration, SQLite will checkpoint the WAL file at the\nconclusion of any transaction when the WAL file is more than 1000 pages\nlong. However, compile-time and run-time options exist that can disable\nor defer this automatic checkpoint. If an application disables the\nautomatic checkpoint, then there is nothing to prevent the WAL file\nfrom growing excessively.\nCheckpoint starvation.\nA checkpoint is only able to run to completion, and reset the WAL file,\nif there are no other database connections using the WAL file. If another\nconnection has a read transaction open,\nthen the checkpoint cannot reset the WAL file because\ndoing so might delete content out from under the reader.\nThe checkpoint will do as much work as it can without upsetting the\nreader, but it cannot run to completion.\nThe checkpoint will start up again where it left off after the next\nwrite transaction. This repeats until some checkpoint is able to complete.\nHowever, if a database has many concurrent overlapping readers\nand there is always at least one active reader, then\nno checkpoints will be able to complete\nand hence the WAL file will grow without bound.\nThis scenario can be avoided by ensuring that there are \"reader gaps\":\ntimes when no processes are reading from the\ndatabase and that checkpoints are attempted during those times.\nIn applications with many concurrent readers, one might also consider\nrunning manual checkpoints with the SQLITE_CHECKPOINT_RESTART or\nSQLITE_CHECKPOINT_TRUNCATE option which will ensure that the checkpoint\nruns to completion before returning. The disadvantage of using\nSQLITE_CHECKPOINT_RESTART and SQLITE_CHECKPOINT_TRUNCATE is that\nreaders might block while the checkpoint is running.\nVery large write transactions.\nA checkpoint can only complete when no other transactions are running,\nwhich means the WAL file cannot be reset in the middle of a write\ntransaction. So a large change to a large database\nmight result in a large WAL file. The WAL file will be checkpointed\nonce the write transaction completes (assuming there are no other readers\nblocking it) but in the meantime, the file can grow very big.\nAs of SQLite version 3.11.0 (2016-02-15),\nthe WAL file for a single transaction\nshould be proportional in size to the transaction itself. Pages that\nare changed by the transaction should only be written into the WAL file\nonce. However, with older versions of SQLite, the same page might be\nwritten into the WAL file multiple times if the transaction grows larger\nthan the page cache.\n7. Implementation Of Shared-Memory For The WAL-Index\nThe wal-index is implemented using an ordinary file that is\nmmapped for robustness. Early (pre-release) implementations of WAL mode\nstored the wal-index in volatile shared-memory, such as files created in\n/dev/shm on Linux or /tmp on other unix systems. The problem\nwith that approach is that processes with a different root directory\n(changed via chroot)\nwill see different files and hence use different shared memory areas,\nleading to database corruption. Other methods for creating nameless\nshared memory blocks are not portable across the various flavors of\nunix. And we could not find any method to create nameless shared\nmemory blocks on windows. The only way we have found to guarantee\nthat all processes accessing the same database file use the same shared\nmemory is to create the shared memory by mmapping a file in the same\ndirectory as the database itself.\nUsing an ordinary disk file to provide shared memory has the\ndisadvantage that it might actually do unnecessary disk I/O by\nwriting the shared memory to disk. However, the developers do not\nthink this is a major concern since the wal-index rarely exceeds\n32 KiB in size and is never synced. Furthermore, the wal-index\nbacking file is deleted when the last database connection disconnects,\nwhich often prevents any real disk I/O from ever happening.\nSpecialized applications for which the default implementation of\nshared memory is unacceptable can devise alternative methods via a\ncustom VFS.\nFor example, if it is known that a particular database\nwill only be accessed by threads within a single process, the wal-index\ncan be implemented using heap memory instead of true shared memory.\n8. Use of WAL Without Shared-Memory\nBeginning in SQLite version 3.7.4 (2010-12-07),\nWAL databases can be created, read, and\nwritten even if shared memory is unavailable as long as the\nlocking_mode is set to EXCLUSIVE before the first attempted access.\nIn other words, a process can interact with\na WAL database without using shared memory if that\nprocess is guaranteed to be the only process accessing the database.\nThis feature allows WAL databases to be created, read, and written\nby legacy VFSes that lack the \"version 2\" shared-memory\nmethods xShmMap, xShmLock, xShmBarrier, and xShmUnmap on the\nsqlite3_io_methods object.\nIf EXCLUSIVE locking mode\nis set prior to the first WAL-mode\ndatabase access, then SQLite never attempts to call any of the\nshared-memory methods and hence no shared-memory\nwal-index is ever created.\nIn that case, the database connection remains in EXCLUSIVE mode\nas long as the journal mode is WAL; attempts to change the locking\nmode using \"PRAGMA locking_mode=NORMAL;\" are no-ops.\nThe only way to change out of EXCLUSIVE locking mode is to first\nchange out of WAL journal mode.\nIf NORMAL locking mode is in effect for the first WAL-mode database\naccess, then the shared-memory wal-index is created. This means that the\nunderlying VFS must support the \"version 2\" shared-memory.\nIf the VFS does not support shared-memory methods, then the attempt to\nopen a database that is already in WAL mode, or the attempt to convert a\ndatabase into WAL mode, will fail.\nAs long as exactly one connection is using a shared-memory wal-index,\nthe locking mode can be changed freely between NORMAL and EXCLUSIVE.\nIt is only when the shared-memory wal-index is omitted, when the locking\nmode is EXCLUSIVE prior to the first WAL-mode database access, that the\nlocking mode is stuck in EXCLUSIVE.\n9. Sometimes Queries Return SQLITE_BUSY In WAL Mode\nThe second advantage of WAL-mode is that\nwriters do not block readers and readers do not block writers.\nThis is mostly true.\nBut there are some obscure cases where a query against a WAL-mode\ndatabase can return SQLITE_BUSY, so applications should be prepared\nfor that happenstance.\nCases where a query against a WAL-mode database can return SQLITE_BUSY\ninclude the following:\nIf another database connection has the database mode open\nin exclusive locking mode then all queries against the\ndatabase will return SQLITE_BUSY. Both Chrome and Firefox open their\ndatabase files in exclusive locking mode, so attempts to read Chrome or\nFirefox databases while the applications are running will run into this\nproblem, for example.\nWhen the last connection to a particular database is closing,\nthat connection will acquire an exclusive lock for a short time while\nit cleans up the WAL and shared-memory files. If a separate attempt\nis made to open and query the database while the first connection is\nstill in the middle of its cleanup process, the second connection\nmight get an SQLITE_BUSY error.\nIf the last connection to a database crashed, then the first new\nconnection to open the database will start a recovery process. An\nexclusive lock is held during recovery. So if a third database connection\ntries to jump in and query while the second connection is running recovery,\nthe third connection will get an SQLITE_BUSY error.\n10. Backwards Compatibility\nThe database file format is unchanged for WAL mode. However, the\nWAL file and the wal-index are new concepts and so older versions of\nSQLite will not know\nhow to recover a crashed SQLite database that was operating in WAL mode\nwhen the crash occurred.\nTo prevent older versions of SQLite (prior to version 3.7.0, 2010-07-22)\nfrom trying to recover\na WAL-mode database (and making matters worse) the database file format\nversion numbers (bytes 18 and 19 in the database header)\nare increased from 1 to 2 in WAL mode.\nThus, if an older version of SQLite attempts to connect to an SQLite\ndatabase that is operating in WAL mode, it will report an error along\nthe lines of \"file is encrypted or is not a database\".\nOne can explicitly change out of WAL mode using a pragma such as\nthis:\nPRAGMA journal_mode=DELETE;\nDeliberately changing out of WAL mode changes the database file format\nversion numbers back to 1 so that older versions of SQLite can once again\naccess the database file.\n11. The WAL-Reset Bug\nOn 2026-03-03, one of the SQLite developers (Dan) found and fixed\na bug that could, in rare cases, lead to database corruption. We call\nthis the \"WAL-reset bug\".\nKey points:\nThe bug is likely present in all version of SQLite from\n3.7.0 (2010-07-21) through 3.51.2 (2026-01-09). It is\nfixed in version 3.51.3 (2026-03-13) and later. Backports of the fix\nare available for some earlier releases:\n3.44.6 and\n3.50.7.\nThe bug only affects databases in WAL mode when there are two or more\ndatabase connections open on the same file, in separate threads or processes,\nand when those two connections attempt to write or checkpoint at the\nsame instant.\nThe bug is a data race with tight timing constraints and that requires\nan unusual usage pattern. It is unlikely\nto occur in common use. The developers were unable to reproduce\nthe bug organically and had to add special testing logic to SQLite\nthat deliberately triggers the circumstances of the the bug in order\nto verify that the issue has been fixed.\n11.1. Bug Details\nThis is what happens:\nOne connection does a checkpoint. This first checkpoint must be\ncomplete. In other words, the checkpoint must successfully copy all\ncontent from the WAL file back into the database and leave the WAL file\nin a state where it can potentially be reset.\nShortly after the first checkpoint completes, a second checkpoint\nis started.\nWhile the second checkpoint from step 2 is starting up, another database\nconnection commits a transaction that resets the WAL file and writes new\ncontent into the beginning of the WAL file.\nDue to a data race, the second checkpoint from step 2 does not realize\nthat the WAL file has been reset by the transaction commit in step 3.\nThe second checkpoint leaves a field in the header of the WAL-Index set\nincorrectly. That field indicates that part of the WAL file has already\nbeen checkpointed, when in fact it has not been.\nAdditional transactions are committed to increase the number of pages\nin the WAL file to be more than were present for the first checkpoint\nfrom step 1.\nLater when a third checkpoint occurs, the third checkpoint skips all\nor part of the transaction that was written in step 3. Thus parts of\nthe transaction from step 3 never reach the database file, and the\ndatabase file goes corrupt.\n11.2. Low Probability Of Occurrence\nIn order for this bug to happen, many details must align at just\nthe right moment. So much so that the SQLite developers were unable\nto reproduce the bug organically in the lab. The only way the developers\nhave been able to cause the malfunction was to modify the SQLite sources\nto invoke a callback controlled by sqlite3_test_control()\nthat enables a test program or script to trigger the write transaction\nof step 3 at just the right moment during the second checkpoint.\nWithout that code hack, the problem has never been observed during\ndevelopment and testing.\nUPDATE: 2026-08-24 → Phil Eaton has devised\na reproducer that does not involve using the special testing code hack.\nSee Another look at SQLite's WAL-Reset bug\nfor details.\nThis bug, though rare, does have serious consequences, and so\napplication developers should upgrade to a version of SQLite\nthat fixes the problem.\nHowever, this is not an emergency.\nBased on available telemetry, the occurrence rate of this problem in the\nwild appears to be less than or equal to the expected occurrence rate of\nSSD malfunctions and/or cosmic-ray hits. So even if you are running an\nunpatched version of SQLite, and unless you are doing something really\nunusual, you are unlikely to ever encounter this problem.\nThis page was last updated on 2026-08-25 19:42:39Z\n",
      "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
    },
    {
      "id": "S4",
      "title": "SQLite Online Backup API",
      "locator": "https://www.sqlite.org/backup.html — retrieved 2026-11-04T12:20:21.652115+00:00; captured document, not release-pinned; reading-text SHA-256 247324b2ef9147ec5137ce7b2e083cf7d7ad4eb289b5e625cea40f8400116c10",
      "text": "SQLite Backup API\nTable Of Contents\n1. Using the SQLite Online Backup API\n1.1. Other Backup Techniques\n2. Example 1: Loading and Saving In-Memory Databases\n2.1. Error handling\n2.2. Possible Enhancements\n3. Example 2: Online Backup of a Running Database\n3.1. File and Database Connection Locking\n3.2. backup_remaining() and backup_pagecount()\n1. Using the SQLite Online Backup API\nHistorically, backups (copies) of SQLite databases have been created\nusing the following method:\nEstablish a shared lock on the database file using the SQLite API (i.e.\nthe shell tool).\nCopy the database file using an external tool (for example the unix 'cp'\nutility or the DOS 'copy' command).\nRelinquish the shared lock on the database file obtained in step 1.\nThis procedure works well in many scenarios and is usually very\nfast. However, this technique has the following shortcomings:\nAny database clients wishing to write to the database file while a\nbackup is being created must wait until the shared lock is\nrelinquished.\nIt cannot be used to copy data to or from in-memory databases.\nIf a power failure or operating system failure occurs while copying\nthe database file the backup database may be corrupted following\nsystem recovery.\nThe Online Backup API was created to\naddress these concerns. The online backup API allows the contents of\none database to be copied into another database file, replacing any\noriginal contents of the target database. The copy operation may be\ndone incrementally, in which case the source database does not need\nto be locked for the duration of the copy, only for the brief periods\nof time when it is actually being read from. This allows other database\nusers to continue without excessive delays while a backup of an online\ndatabase is made.\nThe effect of completing the backup call sequence is to make the\ndestination a bit-wise identical copy of the source database as it\nwas when the copying commenced. (The destination becomes a \"snapshot.\")\nThe online backup API is documented here.\nThe remainder of this page contains two C language examples illustrating\ncommon uses of the API and discussions thereof. Reading these examples\nis no substitute for reading the API documentation!\n1.1. Other Backup Techniques\nThe Online Backup API is the original method for making backups of a\nlive SQLite databases. Other more recent techniques for accomplishing\nthe same thing include:\nThe VACUUM INTO command will make a vacuumed copy of a live\nSQLite database into a separate file.\nThe sqlite3_rsync program makes a copy of a live SQLite database\nto or from a remote system using an SSH connection.\n2. Example 1: Loading and Saving In-Memory Databases\n/*\n** This function is used to load the contents of a database file on disk\n** into the \"main\" database of open database connection pInMemory, or\n** to save the current contents of the database opened by pInMemory into\n** a database file on disk. pInMemory is probably an in-memory database,\n** but this function will also work fine if it is not.\n**\n** Parameter zFilename points to a nul-terminated string containing the\n** name of the database file on disk to load from or save to. If parameter\n** isSave is non-zero, then the contents of the file zFilename are\n** overwritten with the contents of the database opened by pInMemory. If\n** parameter isSave is zero, then the contents of the database opened by\n** pInMemory are replaced by data loaded from the file zFilename.\n**\n** If the operation is successful, SQLITE_OK is returned. Otherwise, if\n** an error occurs, an SQLite error code is returned.\n*/\nint loadOrSaveDb(sqlite3 *pInMemory, const char *zFilename, int isSave){\nint rc; /* Function return code */\nsqlite3 *pFile; /* Database connection opened on zFilename */\nsqlite3_backup *pBackup; /* Backup object used to copy data */\nsqlite3 *pTo; /* Database to copy to (pFile or pInMemory) */\nsqlite3 *pFrom; /* Database to copy from (pFile or pInMemory) */\n/* Open the database file identified by zFilename. Exit early if this fails\n** for any reason. */\nrc = sqlite3_open(zFilename, &pFile);\nif( rc==SQLITE_OK ){\n/* If this is a 'load' operation (isSave==0), then data is copied\n** from the database file just opened to database pInMemory.\n** Otherwise, if this is a 'save' operation (isSave==1), then data\n** is copied from pInMemory to pFile. Set the variables pFrom and\n** pTo accordingly. */\npFrom = (isSave ? pInMemory : pFile);\npTo = (isSave ? pFile : pInMemory);\n/* Set up the backup procedure to copy from the \"main\" database of\n** connection pFile to the main database of connection pInMemory.\n** If something goes wrong, pBackup will be set to NULL and an error\n** code and message left in connection pTo.\n**\n** If the backup object is successfully created, call backup_step()\n** to copy data from pFile to pInMemory. Then call backup_finish()\n** to release resources associated with the pBackup object. If an\n** error occurred, then an error code and message will be left in\n** connection pTo. If no error occurred, then the error code belonging\n** to pTo is set to SQLITE_OK.\n*/\npBackup = sqlite3_backup_init(pTo, \"main\", pFrom, \"main\");\nif( pBackup ){\n(void)sqlite3_backup_step(pBackup, -1);\n(void)sqlite3_backup_finish(pBackup);\n}\nrc = sqlite3_errcode(pTo);\n}\n/* Close the database connection opened on database file zFilename\n** and return the result of this function. */\n(void)sqlite3_close(pFile);\nreturn rc;\n}\nThe C function above demonstrates one of the simplest,\nand most common, uses of the backup API: loading and saving the contents\nof an in-memory database to a file on disk. The backup API is used as\nfollows in this example:\nFunction sqlite3_backup_init() is called to create an sqlite3_backup\nobject to copy data between the two databases (either from a file and\ninto the in-memory database, or vice-versa).\nFunction sqlite3_backup_step() is called with a parameter of\n-1 to copy the entire source database to the destination.\nFunction sqlite3_backup_finish() is called to clean up resources\nallocated by sqlite3_backup_init().\n2.1. Error handling\nIf an error occurs in any of the three main backup API routines\nthen the error code and message are attached to\nthe destination database connection.\nAdditionally, if\nsqlite3_backup_step() encounters an error, then the error code is returned\nby both the sqlite3_backup_step() call itself, and by the subsequent call\nto sqlite3_backup_finish(). So a call to sqlite3_backup_finish()\ndoes not overwrite an error code stored in the destination\ndatabase connection by sqlite3_backup_step(). This feature is used\nin the example code to reduce the amount of error handling required.\nThe return values of the sqlite3_backup_step() and sqlite3_backup_finish()\ncalls are ignored and the error code indicating the success or failure of\nthe copy operation is collected from the destination database connection\nafterward.\n2.2. Possible Enhancements\nThe implementation of this function could be enhanced in at least two ways:\nFailing to obtain the lock on database file zFilename (an SQLITE_BUSY\nerror) could be handled, and\nCases where the page-sizes of database pInMemory and zFilename are\ndifferent could be handled better.\nSince database zFilename is a file on disk, then it may be accessed\nexternally by another process. This means that when the call to\nsqlite3_backup_step() attempts to read from or write data to it, it may\nfail to obtain the required file lock. If this happens, this implementation\nwill fail, returning SQLITE_BUSY immediately. The solution would be to\nregister a busy-handler callback or\ntimeout with database connection pFile\nusing sqlite3_busy_handler() or sqlite3_busy_timeout()\nas soon as it is opened. If it fails to obtain a required lock immediately,\nsqlite3_backup_step() uses any registered busy-handler callback or timeout\nin the same way as sqlite3_step() or sqlite3_exec() does.\nUsually, it does not matter if the page-sizes of the source database and the\ndestination database are different before the contents of the destination\nare overwritten. The page-size of the destination database is simply changed\nas part of the backup operation. The exception is if the destination database\nhappens to be an in-memory database. In this case, if the page sizes\nare not the same at the start of the backup operation, then the operation\nfails with an SQLITE_READONLY error. Unfortunately, this could occur when\nloading a database image from a file into an in-memory database using\nfunction loadOrSaveDb().\nHowever, if in-memory database pInMemory has just been opened (and is\ntherefore completely empty) before being passed to function loadOrSaveDb(),\nthen it is still possible to change its page size using an SQLite \"PRAGMA\npage_size\" command. Function loadOrSaveDb() could detect this case, and\nattempt to set the page-size of the in-memory database to the page-size\nof database zFilename before invoking the online backup API functions.\n3. Example 2: Online Backup of a Running Database\n/*\n** Perform an online backup of database pDb to the database file named\n** by zFilename. This function copies 5 database pages from pDb to\n** zFilename, then unlocks pDb and sleeps for 250 ms, then repeats the\n** process until the entire database is backed up.\n**\n** The third argument passed to this function must be a pointer to a progress\n** function. After each set of 5 pages is backed up, the progress function\n** is invoked with two integer parameters: the number of pages left to\n** copy, and the total number of pages in the source file. This information\n** may be used, for example, to update a GUI progress bar.\n**\n** While this function is running, another thread may use the database pDb, or\n** another process may access the underlying database file via a separate\n** connection.\n**\n** If the backup process is successfully completed, SQLITE_OK is returned.\n** Otherwise, if an error occurs, an SQLite error code is returned.\n*/\nint backupDb(\nsqlite3 *pDb, /* Database to back up */\nconst char *zFilename, /* Name of file to back up to */\nvoid(*xProgress)(int, int) /* Progress function to invoke */\n){\nint rc; /* Function return code */\nsqlite3 *pFile; /* Database connection opened on zFilename */\nsqlite3_backup *pBackup; /* Backup handle used to copy data */\n/* Open the database file identified by zFilename. */\nrc = sqlite3_open(zFilename, &pFile);\nif( rc==SQLITE_OK ){\n/* Open the sqlite3_backup object used to accomplish the transfer */\npBackup = sqlite3_backup_init(pFile, \"main\", pDb, \"main\");\nif( pBackup ){\n/* Each iteration of this loop copies 5 database pages from database\n** pDb to the backup database. If the return value of backup_step()\n** indicates that there are still further pages to copy, sleep for\n** 250 ms before repeating. */\ndo {\nrc = sqlite3_backup_step(pBackup, 5);\nxProgress(\nsqlite3_backup_remaining(pBackup),\nsqlite3_backup_pagecount(pBackup)\n);\nif( rc==SQLITE_OK || rc==SQLITE_BUSY || rc==SQLITE_LOCKED ){\nsqlite3_sleep(250);\n}\n} while( rc==SQLITE_OK || rc==SQLITE_BUSY || rc==SQLITE_LOCKED );\n/* Release resources allocated by backup_init(). */\n(void)sqlite3_backup_finish(pBackup);\n}\nrc = sqlite3_errcode(pFile);\n}\n/* Close the database connection opened on database file zFilename\n** and return the result of this function. */\n(void)sqlite3_close(pFile);\nreturn rc;\n}\nThe function presented in the previous example copies the entire source\ndatabase in one call to sqlite3_backup_step(). This requires holding a\nread-lock on the source database file for the duration of the operation,\npreventing any other database user from writing to the database. It also\nholds the mutex associated with database pInMemory throughout the copy,\npreventing any other thread from using it. The C function in this section,\ndesigned to be called by a background thread or process for creating a\nbackup of an online database, avoids these problems using the following\napproach:\nFunction sqlite3_backup_init() is called to create an sqlite3_backup\nobject to copy data from database pDb to the backup database file\nidentified by zFilename.\nFunction sqlite3_backup_step() is called with a parameter of 5 to\ncopy 5 pages of database pDb to the backup database (file zFilename).\nIf there are still more pages to copy from database pDb, then the\nfunction sleeps for 250 milliseconds (using the sqlite3_sleep()\nutility) and then returns to step 2.\nFunction sqlite3_backup_finish() is called to clean up resources\nallocated by sqlite3_backup_init().\n3.1. File and Database Connection Locking\nDuring the 250 ms sleep in step 3 above, no read-lock is held on the database\nfile and the mutex associated with pDb is not held. This allows other threads\nto use database connection pDb and other connections to write to the\nunderlying database file.\nIf another thread or process writes to the source database while this\nfunction is sleeping, then SQLite detects this and usually restarts the\nbackup process when sqlite3_backup_step() is next called. There is one\nexception to this rule: If the source database is not an in-memory database,\nand the write is performed from within the same process as the backup\noperation and uses the same database handle (pDb), then the destination\ndatabase (the one opened using connection pFile) is automatically updated\nalong with the source. The backup process may then be continued after the\nsqlite3_sleep() call returns as if nothing had happened.\nWhether or not the backup process is restarted as a result of writes to\nthe source database mid-backup, the user can be sure that when the backup\noperation is completed the backup database contains a consistent and\nup-to-date snapshot of the original. However:\nWrites to an in-memory source database, or writes to a file-based\nsource database by an external process or thread using a\ndatabase connection other than pDb are significantly more expensive\nthan writes made to a file-based source database using pDb (as the\nentire backup operation must be restarted in the former two cases).\nIf the backup process is restarted frequently enough it may never\nrun to completion and the backupDb() function may never return.\n3.2. backup_remaining() and backup_pagecount()\nThe backupDb() function uses the sqlite3_backup_remaining() and\nsqlite3_backup_pagecount() functions to report its progress via the\nuser-supplied xProgress() callback. Function sqlite3_backup_remaining()\nreturns the number of pages left to copy and sqlite3_backup_pagecount()\nreturns the total number of pages in the source database (in this case\nthe database opened by pDb). So the percentage completion of the process\nmay be calculated as:\nCompletion = 100% * (pagecount() - remaining()) / pagecount()\nThe sqlite3_backup_remaining() and sqlite3_backup_pagecount() APIs report\nvalues stored by the previous call to sqlite3_backup_step(), they do not\nactually inspect the source database file. This means that if the source\ndatabase is written to by another thread or process after the call to\nsqlite3_backup_step() returns but before the values returned by\nsqlite3_backup_remaining() and sqlite3_backup_pagecount() are used, the\nvalues may be technically incorrect. This is not usually a problem.\nThis page was last updated on 2025-11-13 07:12:58Z\n",
      "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
    }
  ],
  "response": "{\n  \"claims\": [\n    {\n      \"id\": \"C1\",\n      \"text\": \"Do not treat a shared network file as equivalent to an application server. SQLite’s guidance says to avoid direct, simultaneous access to the same database from many computers across a network, without an intervening application server.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"A good rule of thumb is to avoid using SQLite\\nin situations where the same database will be accessed directly\\n(without an intervening application server) and simultaneously\\nfrom many computers over a network.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C2\",\n      \"text\": \"A networked team application can still be a candidate for SQLite when its SQL-issuing application server and database content live on the same machine. The end user being another network hop away is not, by itself, the same as putting the database file across that hop.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"In this rule, \\\"application\\\" means the code that issues SQL statements.\\nIf the \\\"application\\\" is an application server and\\nif the content resides on the same physical machine as the application server,\\nthen SQLite might still be appropriate even though the end user is\\nanother network hop away.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C3\",\n      \"text\": \"Plan for one writer at a time per database file. If the workload needs simultaneous writes that cannot queue and take turns, SQLite’s checklist favors a client/server database engine. WAL also retains a single writer; improved read/write overlap is not concurrent writing.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"If many threads and/or processes need to write the\\ndatabase at the same instant (and they cannot queue up and take turns)\\nthen it is best to select a database engine that supports that\\ncapability, which always means a client/server database engine.\\nSQLite only supports one writer at a time per database file.\"\n        },\n        {\n          \"source\": \"S3\",\n          \"quote\": \"since there is only one WAL file, there can only be one writer at\\na time.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C4\",\n      \"text\": \"Read locking advice in its journal-mode context. The file-locking document explicitly covers the older rollback mechanism, while the WAL guide describes readers and a writer proceeding concurrently. Do not apply the rollback document’s read/write-lock rules as a blanket description of WAL.\",\n      \"citations\": [\n        {\n          \"source\": \"S2\",\n          \"quote\": \"The document only describes locking for the older rollback-mode\\ntransaction mechanism.\"\n        },\n        {\n          \"source\": \"S3\",\n          \"quote\": \"WAL provides more concurrency as readers do not block writers and\\na writer does not block readers. Reading and writing can proceed\\nconcurrently.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C5\",\n      \"text\": \"Turning on WAL is not a workaround for sharing the live database across host machines. The WAL guide requires processes using the database to be on the same host and explicitly says WAL does not work over a network filesystem.\",\n      \"citations\": [\n        {\n          \"source\": \"S3\",\n          \"quote\": \"All processes using a database must be on the same host computer;\\nWAL does not work over a network filesystem. This is because WAL requires\\nall processes to share a small amount of memory and processes on\\nseparate host machines obviously cannot share memory with each other.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C6\",\n      \"text\": \"For online backups, evaluate a database-aware method rather than assuming a live file copy is enough. The Online Backup API can copy incrementally, taking brief source read locks instead of holding a lock for the whole copy. The guide warns that reading its examples is not a substitute for reading the API documentation.\",\n      \"citations\": [\n        {\n          \"source\": \"S4\",\n          \"quote\": \"The copy operation may be\\ndone incrementally, in which case the source database does not need\\nto be locked for the duration of the copy, only for the brief periods\\nof time when it is actually being read from. This allows other database\\nusers to continue without excessive delays while a backup of an online\\ndatabase is made.\"\n        },\n        {\n          \"source\": \"S4\",\n          \"quote\": \"Reading these examples\\nis no substitute for reading the API documentation!\"\n        }\n      ]\n    }\n  ],\n  \"questions\": [\n    \"Where will the code issuing SQL run: on one application host, or on multiple machines opening the same file directly? Draw that boundary before choosing an engine.\",\n    \"What are the write arrival rate, transaction durations and acceptable queue latency for the actual workload? These documents and this review do not measure them.\",\n    \"Which SQLite version, journal mode and filesystem will be used, and how will representative behavior be tested? Captured documentation is not a configuration audit.\",\n    \"How will a completed backup be verified, restored and kept off the failed host? An API capability is not an observed restore or disaster-recovery guarantee.\"\n  ]\n}",
  "notes": "Same-agent editorial review of four real SQLite documentation captures. No model call, workload measurement, engine change, filesystem test or production restore. The capture adapter was corrected before this work was built: prefatory rollback/WAL scope is retained, and obsolete struck-through WAL statements are explicitly marked rather than presented as current text. Original retrieval times and full HTML hashes remain; old plain-text conversion is preserved separately as transformation evidence, not the source used here. All six selected factual claims require source/version/context review; question prompts identify missing deployment evidence, not measured failures. Initial claim order follows source-reading order. The organized version changes presentation only; it does not prove improved comprehension. SQLite public-domain notice is retained in every source’s rights field.",
  "snapshots": [
    {
      "label": "Flat source-reading-order review",
      "savedAt": "2026-11-04T12:28:06.654Z",
      "evidence": {
        "format": "promptkb-evidence-brief",
        "version": 1,
        "title": "SQLite for a team tool: share the service, not the live file",
        "question": "For a team planning a shared record-keeping tool, when do SQLite’s published documents support a server-local SQLite database, and when do they favor a client/server database engine over directly sharing one file? Separate deployment topology, writer concurrency, journal mode and online backups. This is a scoped documentation decision aid, not a workload benchmark, application audit or endorsement.",
        "sources": [
          {
            "id": "S1",
            "title": "Appropriate Uses For SQLite",
            "locator": "https://www.sqlite.org/whentouse.html — retrieved 2026-11-04T12:20:21.371879+00:00; captured document, not release-pinned; reading-text SHA-256 7da46702f746b0fd702778f58abfbb488b689b9de264c032211471a660180271",
            "text": "Appropriate Uses For SQLite\nTable Of Contents\n1. Situations Where SQLite Works Well\n2. Situations Where A Client/Server RDBMS May Work Better\n3. Checklist For Choosing The Right Database Engine\nSQLite is not directly comparable to client/server SQL database engines such\nas MySQL, Oracle, PostgreSQL, or SQL Server since SQLite is trying to\nsolve a different problem.\nClient/server SQL database engines strive to implement a shared repository\nof enterprise data. They emphasize scalability, concurrency, centralization,\nand control.\nSQLite strives to provide local data storage for\nindividual applications and devices. SQLite emphasizes economy,\nefficiency, reliability, independence, and simplicity.\nSQLite does not compete with client/server databases.\nSQLite competes with fopen().\n1. Situations Where SQLite Works Well\nEmbedded devices and the internet of things\nBecause an SQLite database requires no administration,\nit works well in devices that must operate without expert human support.\nSQLite is a good fit for use in\ncellphones, set-top boxes, televisions, game consoles,\ncameras, watches, kitchen appliances, thermostats, automobiles,\nmachine tools, airplanes, remote sensors, drones, medical devices,\nand robots: the \"internet of things\".\nClient/server database engines are designed to live inside a\nlovingly-attended datacenter at the core of the network.\nSQLite works there too, but SQLite also thrives at the edge of the network,\nfending for itself while providing fast and\nreliable data services to applications that would otherwise\nhave dodgy connectivity.\nApplication file format\nSQLite is often used as the on-disk file format\nfor desktop applications such as version control systems,\nfinancial analysis tools, media cataloging and editing suites, CAD\npackages, record keeping programs, and so forth. The traditional\nFile/Open operation calls sqlite3_open() to attach to the database\nfile. Updates happen automatically as application content is revised\nso the File/Save menu option becomes superfluous. The File/Save_As\nmenu option can be implemented using the backup API.\nThere are many benefits to this approach, including improved\nperformance, reduced cost and complexity, and\nimproved reliability. See technical notes\n\"aff_short.html\" and\n\"appfileformat.html\" and\n\"fasterthanfs.html\" for more information.\nThis use case is closely related to the\ndata transfer format and\ndata container use cases below.\nWebsites\nSQLite works great as the database engine for most low to\nmedium traffic websites (which is to say, most websites).\nThe amount of web traffic that SQLite can handle depends\non how heavily the website uses its database. Generally\nspeaking, any site that gets fewer than 100K hits/day should work\nfine with SQLite.\nThe 100K hits/day figure is a conservative estimate, not a\nhard upper bound.\nSQLite has been demonstrated to work with 10 times that amount\nof traffic.\nThe SQLite website (https://sqlite.org/) uses SQLite itself,\nof course, and as of this writing (2015) it handles about 400K to 500K\nHTTP requests per day, about 15-20% of which are dynamic pages touching\nthe database. Dynamic content uses about 200 SQL statements per webpage.\nThis setup runs on a single VM that shares a physical server with 23 others\nand yet still keeps the load average below 0.1 most of the time.\nSee also: Hacker News discussion from 2022-12-13.\nData analysis\nPeople who understand SQL can employ the\nsqlite3 command-line shell (or various third-party\nSQLite access programs) to analyze large\ndatasets. Raw data can be imported from CSV files, then that\ndata can be sliced and diced to generate a myriad of summary\nreports. More complex analysis can be done using simple scripts written\nin Tcl or Python (both of which come with SQLite built-in) or in R or\nother languages using readily available adaptors.\nPossible uses include website log analysis, sports\nstatistics analysis, compilation of programming metrics, and\nanalysis of experimental results. Many bioinformatics researchers\nuse SQLite in this way.\nThe same thing can be done with an enterprise client/server\ndatabase, of course. The advantage of SQLite is\nthat it is easier to install and use and the resulting database\nis a single file that can be written to a USB memory stick\nor emailed to a colleague.\nCache for enterprise data\nMany applications use SQLite as a cache of relevant content from\nan enterprise RDBMS.\nThis reduces latency, since most queries now occur against the local\ncache and avoid a network round-trip. It also reduces the load\non the network and on the central database server. And in many cases,\nit means that the client-side application can continue operating during\nnetwork outages.\nServer-side database\nSystems designers\nreport success using SQLite as a data store on server applications\nrunning in the datacenter, or in other words, using SQLite as the underlying\nstorage engine for an application-specific database server.\nWith this pattern, the overall system is still client/server:\nclients send requests to the server and get back replies over the network.\nBut instead of sending generic SQL and getting back raw table content,\nthe client requests and server responses are high-level and\napplication-specific.\nThe server translates requests into multiple SQL queries, gathers the\nresults, does post-processing, filtering, and analysis, then constructs\na high-level reply containing only the essential information.\nDevelopers report that SQLite is often faster than a client/server\nSQL database engine in this scenario.\nDatabase requests are serialized by the server, so concurrency is not\nan issue. Concurrency is also improved by \"database sharding\":\nusing separate database files for different subdomains. For\nexample, the server might have a separate SQLite database for each\nuser, so that the server can handle hundreds or thousands of simultaneous\nconnections, but each SQLite database is only used by one connection.\nData transfer format\nBecause an SQLite database is a single compact file in a\nwell-defined cross-platform format, it is often used\nas a container for transferring content from one system to another.\nThe sender gathers content into an SQLite database file, transfers\nthat one file to the receiver, then the receiver uses SQL to extract\nthe content as needed.\nAn SQLite database facilitates data transfer between systems even\nwhen the endpoints have different word sizes and/or byte orders.\nThe data can be a complex mix of large binary blobs, text, and small\nnumeric or boolean values. The data format can be easily extended\nby adding new tables and/or columns, without breaking legacy receivers.\nThe SQL query language means that receivers are not required to parse\nthe entire transfer all at once, but can instead query the\nreceived content as needed. The data format is \"transparent\" in the\nsense that it is easily decoded for human viewing using\na variety of universally available, open-source tools, from multiple\nvendors.\nFile archive and/or data container\nThe SQLite Archive idea shows how\nSQLite can be used as a substitute for ZIP archives or Tarballs.\nAn archive of files stored in SQLite is only very slightly larger, and\nin some cases actually smaller, than the equivalent ZIP archive.\nAnd an SQLite archive features incremental and atomic updating\nand the ability to store much richer metadata.\nFossil version 2.5 and later offers\nSQLite Archive files as a download format, in addition\nto traditional tarball and ZIP archive.\nThe sqlite3.exe command-line shell version 3.22.0 and later will create,\nlist, or unpack an SQL archive using the\n.archive command.\nSQLite is a good solution for any situation that requires bundling\ndiverse content into a self-contained and self-describing package\nfor shipment across a network.\nContent is encoded in a\nwell-defined, cross-platform, and stable file format.\nThe encoding is efficient, and receivers can extract small subsets\nof the content without having to read and parse the entire file.\nSQL archives are useful as the distribution format for software\nor content updates that are broadcast to many clients. Variations\non this idea are used, for example, to transmit TV programming guides\nto set-top boxes and to send over-the-air updates to vehicle navigation\nsystems.\nReplacement for ad hoc disk files\nMany programs use\nfopen(),\nfread(), and\nfwrite() to create and\nmanage files of data in home-grown formats. SQLite works\nparticularly well as a\nreplacement for these ad hoc data files.\nContrary to intuition, SQLite can be faster than the filesystem\nfor reading and writing content to disk.\nInternal or temporary databases\nFor programs that have a lot of data that must be sifted and sorted\nin diverse ways, it is often easier and quicker to load the data into\nan in-memory SQLite database and use queries with joins and ORDER BY\nclauses to extract the data in the form and order needed rather than\nto try to code the same operations manually.\nUsing an SQL database internally in this way also gives the program\ngreater flexibility since new columns and indices can be added without\nhaving to recode every query.\nStand-in for an enterprise database during demos or testing\nClient applications typically use a generic database interface that allows\nconnections to various SQL database engines. It makes good sense to\ninclude SQLite in the mix of supported databases and to statically\nlink the SQLite engine in with the client. That way the client program\ncan be used standalone with an SQLite data file for testing or for\ndemonstrations.\nEducation and Training\nBecause it is simple to setup and use (installation is trivial: just\ncopy the sqlite3 or sqlite3.exe executable to the target machine\nand run it) SQLite makes a good database engine for use in teaching SQL.\nStudents can easily create as many databases as they like and can\nemail databases to the instructor for comments or grading. For more\nadvanced students who are interested in studying how an RDBMS is\nimplemented, the modular and well-commented and documented SQLite code\ncan serve as a good basis.\nExperimental SQL language extensions\nThe simple, modular design of SQLite makes it a good platform for\nprototyping new, experimental database language features or ideas.\n2. Situations Where A Client/Server RDBMS May Work Better\nClient/Server Applications\nIf there are many client programs sending SQL to the same\ndatabase over a network, then use a client/server database\nengine instead of SQLite. SQLite will work over a network filesystem,\nbut because of the latency associated with most network filesystems,\nperformance will not be great. Also, file locking logic is buggy in\nmany network filesystem implementations (on both Unix and Windows).\nIf file locking does not work correctly,\ntwo or more clients might try to modify the\nsame part of the same database at the same time, resulting in\ncorruption. Because this problem results from bugs in\nthe underlying filesystem implementation, there is nothing SQLite\ncan do to prevent it.\nA good rule of thumb is to avoid using SQLite\nin situations where the same database will be accessed directly\n(without an intervening application server) and simultaneously\nfrom many computers over a network.\nHigh-volume Websites\nSQLite will normally work fine as the database backend to a website.\nBut if the website is write-intensive or is so busy that it requires\nmultiple servers, then consider using an enterprise-class client/server\ndatabase engine instead of SQLite.\nVery large datasets\nAn SQLite database is limited in size to 281 terabytes\n(2^(48) bytes, 256 tibibytes).\nAnd even if it could handle larger databases, SQLite stores the entire\ndatabase in a single disk file and many filesystems limit the maximum\nsize of files to something less than this. So if you are contemplating\ndatabases of this magnitude, you would do well to consider using a\nclient/server database engine that spreads its content across multiple\ndisk files, and perhaps across multiple volumes.\nHigh Concurrency\nSQLite supports an unlimited number of simultaneous readers, but it\nwill only allow one writer at any instant in time.\nFor many situations, this is not a problem. Writers queue up. Each application\ndoes its database work quickly and moves on, and no lock lasts for more\nthan a few dozen milliseconds. But there are some applications that require\nmore concurrency, and those applications may need to seek a different\nsolution.\n3. Checklist For Choosing The Right Database Engine\nIs the data separated from the application by a network?\n→ choose client/server\nRelational database engines act as bandwidth-reducing data filters.\nSo it is best to keep the database engine and the data on\nthe same physical device so that the high-bandwidth engine-to-disk\nlink does not have to traverse the network, only the lower-bandwidth\napplication-to-engine link.\nBut SQLite is built into the application. So if the data is on a\nseparate device from the application, it is required that the higher\nbandwidth engine-to-disk link be across the network. This works, but\nit is suboptimal. Hence, it is usually better to select a client/server\ndatabase engine when the data is on a separate device from the\napplication.\nNota Bene:\nIn this rule, \"application\" means the code that issues SQL statements.\nIf the \"application\" is an application server and\nif the content resides on the same physical machine as the application server,\nthen SQLite might still be appropriate even though the end user is\nanother network hop away.\nMany concurrent writers? → choose client/server\nIf many threads and/or processes need to write the\ndatabase at the same instant (and they cannot queue up and take turns)\nthen it is best to select a database engine that supports that\ncapability, which always means a client/server database engine.\nSQLite only supports one writer at a time per database file.\nBut in most cases, a write transaction only takes milliseconds and\nso multiple writers can simply take turns. SQLite will handle\nmore write concurrency than many people suspect. Nevertheless,\nclient/server database systems, because they have a long-running\nserver process at hand to coordinate access, can usually handle\nfar more write concurrency than SQLite ever will.\nBig data? → choose client/server\nIf your data will grow to a size that you are uncomfortable\nor unable to fit into a single disk file, then you should select\na solution other than SQLite. SQLite supports databases up to\n281 terabytes in size, assuming you can find a disk drive and filesystem\nthat will support 281-terabyte files. Even so, when the size of the\ncontent looks like it might creep into the terabyte range, it would\nbe good to consider a centralized client/server database.\nOtherwise → choose SQLite!\nFor device-local storage with low writer concurrency and less than a\nterabyte of content, SQLite is almost always a better solution. SQLite\nis fast and reliable and it requires no configuration or maintenance.\nIt keeps things simple. SQLite \"just works\".\nThis page was last updated on 2025-05-31 13:08:22Z\n",
            "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
          },
          {
            "id": "S2",
            "title": "SQLite file locking — rollback-mode scope",
            "locator": "https://www.sqlite.org/lockingv3.html — retrieved 2026-11-04T12:20:21.459437+00:00; captured document, not release-pinned; reading-text SHA-256 2c78d8dada32d8d001dd41cb581cd68fade6a336ab1a06ded7ccf0c2ce099649",
            "text": "This document was originally created in early 2004 when SQLite version 2\nwas still in widespread use and was written to introduce\nthe new concepts of SQLite version 3 to readers who were already familiar\nwith SQLite version 2. But these days, most readers of this document have\nprobably never seen SQLite version 2 and are only familiar with SQLite\nversion 3. Nevertheless, this document continues to serve as an\nauthoritative reference to how database file locking works in SQLite\nversion 3.\nThe document only describes locking for the older rollback-mode\ntransaction mechanism. Locking for the newer write-ahead log or WAL mode\nis described separately.\n1.0 File Locking And Concurrency In SQLite Version 3\nSQLite Version 3.0.0 introduced a new locking and journaling\nmechanism designed to improve concurrency over SQLite version 2\nand to reduce the writer starvation\nproblem. The new mechanism also allows atomic commits of transactions\ninvolving multiple database files.\nThis document describes the new locking mechanism.\nThe intended audience is programmers who want to understand and/or modify\nthe pager code and reviewers working to verify the design\nof SQLite version 3.\n2.0 Overview\nLocking and concurrency control are handled by the\npager module.\nThe pager module is responsible for making SQLite \"ACID\" (Atomic,\nConsistent, Isolated, and Durable). The pager module makes sure changes\nhappen all at once, that either all changes occur or none of them do,\nthat two or more processes do not try to access the database\nin incompatible ways at the same time, and that once changes have been\nwritten they persist until explicitly deleted. The pager also provides\na memory cache of some of the contents of the disk file.\nThe pager is unconcerned\nwith the details of B-Trees, text encodings, indices, and so forth.\nFrom the point of view of the pager the database consists of\na single file of uniform-sized blocks. Each block is called a\n\"page\" and is usually 1024 bytes in size. The pages are numbered\nbeginning with 1. So the first 1024 bytes of the database are called\n\"page 1\" and the second 1024 bytes are call \"page 2\" and so forth. All\nother encoding details are handled by higher layers of the library.\nThe pager communicates with the operating system using one of several\nmodules\n(Examples:\nos_unix.c,\nos_win.c)\nthat provides a uniform abstraction for operating system services.\nThe pager module effectively controls access for separate threads, or\nseparate processes, or both. Throughout this document whenever the\nword \"process\" is written you may substitute the word \"thread\" without\nchanging the truth of the statement.\n3.0 Locking\nFrom the point of view of a single process, a database file\ncan be in one of five locking states:\nUNLOCKED\nNo locks are held on the database. The database may be neither read nor\nwritten. Any internally cached data is considered suspect and subject to\nverification against the database file before being used. Other\nprocesses can read or write the database as their own locking states\npermit. This is the default state.\nSHARED\nThe database may be read but not written. Any number of\nprocesses can hold SHARED locks at the same time, hence there can be\nmany simultaneous readers. But no other thread or process is allowed\nto write to the database file while one or more SHARED locks are active.\nRESERVED\nA RESERVED lock means that the process is planning on writing to the\ndatabase file at some point in the future but that it is currently just\nreading from the file. Only a single RESERVED lock may be active at one\ntime, though multiple SHARED locks can coexist with a single RESERVED lock.\nRESERVED differs from PENDING in that new SHARED locks can be acquired\nwhile there is a RESERVED lock.\nPENDING\nA PENDING lock means that the process holding the lock wants to write\nto the database as soon as possible and is just waiting on all current\nSHARED locks to clear so that it can get an EXCLUSIVE lock. No new\nSHARED locks are permitted against the database if\na PENDING lock is active, though existing SHARED locks are allowed to\ncontinue.\nEXCLUSIVE\nAn EXCLUSIVE lock is needed in order to write to the database file.\nOnly one EXCLUSIVE lock is allowed on the file and no other locks of\nany kind are allowed to coexist with an EXCLUSIVE lock. In order to\nmaximize concurrency, SQLite works to minimize the amount of time that\nEXCLUSIVE locks are held.\nThe operating system interface layer understands and tracks all five\nlocking states described above.\nThe pager module only tracks four of the five locking states.\nA PENDING lock is always just a temporary\nstepping stone on the path to an EXCLUSIVE lock and so the pager module\ndoes not track PENDING locks.\n4.0 The Rollback Journal\nWhen a process wants to change a database file (and it is not\nin WAL mode), it\nfirst records the original unchanged database content\nin a rollback journal. The rollback journal is an ordinary\ndisk file that is always located\nin the same directory or folder as the database file and has the\nsame name as the database file with the addition of a -journal\nsuffix. The rollback journal also records the initial\nsize of the database so that if the database file grows it can be truncated\nback to its original size on a rollback.\nIf SQLite is working with multiple databases at the same time\n(using the ATTACH command) then each database has its own rollback journal.\nBut there is also a separate aggregate journal\ncalled the super-journal.\nThe super-journal does not contain page data used for rolling back\nchanges. Instead the super-journal contains the names of the\nindividual database rollback journals for each of the ATTACHed databases.\nEach of the individual database rollback journals also contain the name\nof the super-journal.\nIf there are no ATTACHed databases (or if none of the ATTACHed database\nis participating in the current transaction) no super-journal is\ncreated and the normal rollback journal contains an empty string\nin the place normally reserved for recording the name of the\nsuper-journal.\nA rollback journal is said to be hot\nif it needs to be rolled back\nin order to restore the integrity of its database.\nA hot journal is created when a process is in the middle of a database\nupdate and a program or operating system crash or power failure prevents\nthe update from completing.\nHot journals are an exception condition.\nHot journals exist to recover from crashes and power failures.\nIf everything is working correctly\n(that is, if there are no crashes or power failures)\nyou will never get a hot journal.\nIf no super-journal is involved, then\na journal is hot if it exists and has a non-zero header\nand its corresponding database file\ndoes not have a RESERVED lock.\nIf a super-journal is named in the file journal, then the file journal\nis hot if its super-journal exists and there is no RESERVED\nlock on the corresponding database file.\nIt is important to understand when a journal is hot so the\npreceding rules will be repeated in bullets:\nA journal is hot if...\nIt exists, and\nIts size is greater than 512 bytes, and\nThe journal header is non-zero and well-formed, and\nIts super-journal exists or the super-journal name is an\nempty string, and\nThere is no RESERVED lock on the corresponding database file.\n4.1 Dealing with hot journals\nBefore reading from a database file, SQLite always checks to see if that\ndatabase file has a hot journal. If the file does have a hot journal, then\nthe journal is rolled back before the file is read. In this way, we ensure\nthat the database file is in a consistent state before it is read.\nWhen a process wants to read from a database file, it follows\nthe following sequence of steps:\nOpen the database file and obtain a SHARED lock. If the SHARED lock\ncannot be obtained, fail immediately and return SQLITE_BUSY.\nCheck to see if the database file has a hot journal. If the file\ndoes not have a hot journal, we are done. Return immediately.\nIf there is a hot journal, that journal must be rolled back by\nthe subsequent steps of this algorithm.\nAcquire a PENDING lock then an EXCLUSIVE lock on the database file.\n(Note: Do not acquire a RESERVED lock because that would make\nother processes think the journal was no longer hot.) If we\nfail to acquire these locks it means another process\nis already trying to do the rollback. In that case,\ndrop all locks, close the database, and return SQLITE_BUSY.\nRead the journal file and roll back the changes.\nWait for the rolled back changes to be written onto\npersistent storage. This protects the integrity of the database\nin case another power failure or crash occurs.\nDelete the journal file (or truncate the journal to zero bytes in\nlength if PRAGMA journal_mode=TRUNCATE is\nset, or zero the journal header if\nPRAGMA journal_mode=PERSIST is set).\nDelete the super-journal file if it is safe to do so.\nThis step is optional. It is here only to prevent stale\nsuper-journals from cluttering up the disk drive.\nSee the discussion below for details.\nDrop the EXCLUSIVE and PENDING locks but retain the SHARED lock.\nAfter the algorithm above completes successfully, it is safe to\nread from the database file. Once all reading has completed, the\nSHARED lock is dropped.\n4.2 Deleting stale super-journals\nA stale super-journal is a super-journal that is no longer being\nused for anything. There is no requirement that stale super-journals\nbe deleted. The only reason for doing so is to free up disk space.\nA super-journal is stale if no individual file journals are pointing\nto it. To figure out if a super-journal is stale, we first read the\nsuper-journal to obtain the names of all of its file journals. Then\nwe check each of those file journals. If any of the file journals named\nin the super-journal exists and points back to the super-journal, then\nthe super-journal is not stale. If all file journals are either missing\nor refer to other super-journals or no super-journal at all, then the\nsuper-journal we are testing is stale and can be safely deleted.\n5.0 Writing to a database file\nTo write to a database, a process must first acquire a SHARED lock\nas described above (possibly rolling back incomplete changes if there\nis a hot journal).\nAfter a SHARED lock is obtained, a RESERVED lock must be acquired.\nThe RESERVED lock signals that the process intends to write to the\ndatabase at some point in the future. Only one process at a time\ncan hold a RESERVED lock. But other processes can continue to read\nthe database while the RESERVED lock is held.\nIf the process that wants to write is unable to obtain a RESERVED\nlock, it must mean that another process already has a RESERVED lock.\nIn that case, the write attempt fails and returns SQLITE_BUSY.\nAfter obtaining a RESERVED lock, the process that wants to write\ncreates a rollback journal. The header of the journal is initialized\nwith the original size of the database file. Space in the journal header\nis also reserved for a super-journal name, though the super-journal\nname is initially empty.\nBefore making changes to any page of the database, the process writes\nthe original content of that page into the rollback journal. Changes\nto pages are held in memory at first and are not written to the disk.\nThe original database file remains unaltered, which means that other\nprocesses can continue to read the database.\nEventually, the writing process will want to update the database\nfile, either because its memory cache has filled up or because it is\nready to commit its changes. Before this happens, the writer must\nmake sure no other process is reading the database and that the rollback\njournal data is safely on the disk surface so that it can be used to\nrollback incomplete changes in the event of a power failure.\nThe steps are as follows:\nMake sure all rollback journal data has actually been written to\nthe surface of the disk (and is not just being held in the operating\nsystem's or disk controllers cache) so that if a power failure occurs\nthe data will still be there after power is restored.\nObtain a PENDING lock and then an EXCLUSIVE lock on the database file.\nIf other processes still have SHARED locks, the writer might have\nto wait until those SHARED locks clear before it is able to obtain\nan EXCLUSIVE lock.\nWrite all page modifications currently held in memory out to the\noriginal database disk file.\nIf the reason for writing to the database file is because the memory\ncache was full, then the writer will not commit right away. Instead,\nthe writer might continue to make changes to other pages. Before\nsubsequent changes are written to the database file, the rollback\njournal must be flushed to disk again. Note also that the EXCLUSIVE\nlock that the writer obtained in order to write to the database initially\nmust be held until all changes are committed. That means that no other\nprocesses are able to access the database from the\ntime the memory cache first spills to disk until the transaction\ncommits.\nWhen a writer is ready to commit its changes, it executes the following\nsteps:\nObtain an EXCLUSIVE lock on the database file and\nmake sure all memory changes have been written to the database file\nusing the algorithm of steps 1-3 above.\nFlush all database file changes to the disk. Wait for those changes\nto actually be written onto the disk surface.\nDelete the journal file. (Or if the PRAGMA journal_mode is TRUNCATE or\nPERSIST, truncate the journal file or zero the header of the journal file,\nrespectively.) This is the instant when the changes are\ncommitted. Prior to deleting the journal file, if a power failure\nor crash occurs, the next process to open the database will see that\nit has a hot journal and will roll the changes back.\nAfter the journal is deleted, there will no longer be a hot journal\nand the changes will persist.\nDrop the EXCLUSIVE and PENDING locks from the database file.\nAs soon as the PENDING lock is released from the database file, other\nprocesses can begin reading the database again. In the current implementation,\nthe RESERVED lock is also released, but that is not essential for\ncorrect operation.\nIf a transaction involves multiple databases, then a more complex\ncommit sequence is used, as follows:\nMake sure all individual database files have an EXCLUSIVE lock and a\nvalid journal.\nCreate a super-journal. The name of the super-journal is arbitrary.\n(The current implementation appends random suffixes to the name of the\nmain database file until it finds a name that does not previously exist.)\nFill the super-journal with the names of all the individual journals\nand flush its contents to disk.\nWrite the name of the super-journal into\nall individual journals (in space set aside for that purpose in the\nheaders of the individual journals) and flush the contents of the\nindividual journals to disk and wait for those changes to reach the\ndisk surface.\nFlush all database file changes to the disk. Wait for those changes\nto actually be written onto the disk surface.\nDelete the super-journal file. This is the instant when the changes are\ncommitted. Prior to deleting the super-journal file, if a power failure\nor crash occurs, the individual file journals will be considered hot\nand will be rolled back by the next process that\nattempts to read them. After the super-journal has been deleted,\nthe file journals will no longer be considered hot and the changes\nwill persist.\nDelete all individual journal files.\nDrop the EXCLUSIVE and PENDING locks from all database files.\n5.1 Writer starvation\nIn SQLite version 2, if many processes are reading from the database,\nit might be the case that there is never a time when there are\nno active readers. And if there is always at least one read lock on the\ndatabase, no process would ever be able to make changes to the database\nbecause it would be impossible to acquire a write lock. This situation\nis called writer starvation.\nSQLite version 3 seeks to avoid writer starvation through the use of\nthe PENDING lock. The PENDING lock allows existing readers to continue\nbut prevents new readers from connecting to the database. So when a\nprocess wants to write a busy database, it can set a PENDING lock which\nwill prevent new readers from coming in. Assuming existing readers do\neventually complete, all SHARED locks will eventually clear and the\nwriter will be given a chance to make its changes.\n6.0 How To Corrupt Your Database Files\nThe pager module is very robust but it can be subverted.\nThis section attempts to identify and explain the risks.\n(See also the Things That Can Go Wrong section of the article\non Atomic Commit.\nClearly, a hardware or operating system fault that introduces incorrect data\ninto the middle of the database file or journal will cause problems.\nLikewise,\nif a rogue process opens a database file or journal and writes malformed\ndata into the middle of it, then the database will become corrupt.\nThere is not much that can be done about these kinds of problems\nso they are given no further attention.\nSQLite uses POSIX advisory locks to implement locking on Unix. On\nWindows it uses the LockFile(), LockFileEx(), and UnlockFile() system\ncalls. SQLite assumes that these system calls all work as advertised. If\nthat is not the case, then database corruption can result. One should\nnote that POSIX advisory locking is known to be buggy or even unimplemented\non many NFS implementations (including recent versions of Mac OS X)\nand that there are reports of locking problems\nfor network filesystems under Windows. Your best defense is to not\nuse SQLite for files on a network filesystem.\nSQLite uses the fsync() system call to flush data to the disk under Unix and\nit uses the FlushFileBuffers() to do the same under Windows. Once again,\nSQLite assumes that these operating system services function as advertised.\nBut it has been reported that fsync() and FlushFileBuffers() do not always\nwork correctly, especially with some network filesystems or inexpensive IDE disks.\nApparently some manufacturers of IDE disks have controller chips that report\nthat data has reached the disk surface when in fact the data is still\nin volatile cache memory in the disk drive electronics. There are also\nreports that Windows sometimes chooses to ignore FlushFileBuffers() for\nunspecified reasons. The author cannot verify any of these reports.\nBut if they are true, it means that database corruption is a possibility\nfollowing an unexpected power loss. These are hardware and/or operating\nsystem bugs that SQLite is unable to defend against.\nIf a Linux ext3\nfilesystem is mounted without the \"barrier=1\" option\nin the /etc/fstab\nand the disk drive write cache is enabled\nthen filesystem corruption can occur following a power loss or OS crash.\nWhether or not corruption can occur depends on the details of the disk control\nhardware; corruption is more likely with inexpensive consumer-grade disks\nand less of a problem for enterprise-class storage devices with advanced\nfeatures such as non-volatile write caches.\nVarious ext3 experts\nconfirm this behavior.\nWe are told that most Linux distributions do not use barrier=1 and do\nnot disable the write cache so most\nLinux distributions are vulnerable to this problem. Note that this is an\noperating system and hardware issue and that there is nothing that SQLite\ncan do to work around it.\nOther database engines have also run into this same problem.\nIf a crash or power failure occurs and results in a hot journal but that\njournal is deleted, the next process to open the database will not\nknow that it contains changes that need to be rolled back. The rollback\nwill not occur and the database will be left in an inconsistent state.\nRollback journals might be deleted for any number of reasons:\nAn administrator might be cleaning up after an OS crash or power failure,\nsee the journal file, think it is junk, and delete it.\nSomeone (or some process) might rename the database file but fail to\nalso rename its associated journal.\nIf the database file has aliases (hard or soft links) and the file\nis opened by a different alias than the one used to create the journal,\nthen the journal will not be found. To avoid this problem, you should\nnot create links to SQLite database files.\nFilesystem corruption following a power failure might cause the\njournal to be renamed or deleted.\nThe last (fourth) bullet above merits additional comment. When SQLite creates\na journal file on Unix, it opens the directory that contains that file and\ncalls fsync() on the directory, in an effort to push the directory information\nto disk. But suppose some other process is adding or removing unrelated\nfiles to the directory that contains the database and journal at the\nmoment of a power failure. The supposedly unrelated actions of this other\nprocess might result in the journal file being dropped from the directory and\nmoved into \"lost+found\". This is an unlikely scenario, but it could happen.\nThe best defenses are to use a journaling filesystem or to keep the\ndatabase and journal in a directory by themselves.\nFor a commit involving multiple databases and a super-journal, if the\nvarious databases were on different disk volumes and a power failure occurs\nduring the commit, then when the machine comes back up the disks might\nbe remounted with different names. Or some disks might not be mounted\nat all. When this happens the individual file journals and the\nsuper-journal might not be able to find each other. The worst outcome from\nthis scenario is that the commit ceases to be atomic.\nSome databases might be rolled back and others might not.\nAll databases will continue to be self-consistent.\nTo defend against this problem, keep all databases\non the same disk volume and/or remount disks using exactly the same names\nafter a power failure.\n7.0 Transaction Control At The SQL Level\nThe changes to locking and concurrency control in SQLite version 3 also\nintroduce some subtle changes in the way transactions work at the SQL\nlanguage level.\nBy default, SQLite version 3 operates in autocommit mode.\nIn autocommit mode,\nall changes to the database are committed as soon as all operations associated\nwith the current database connection complete.\nThe SQL command \"BEGIN TRANSACTION\" (the TRANSACTION keyword\nis optional) is used to take SQLite out of autocommit mode.\nNote that the BEGIN command does not acquire any locks on the database.\nAfter a BEGIN command, a SHARED lock will be acquired when the first\nSELECT statement is executed. A RESERVED lock will be acquired when\nthe first INSERT, UPDATE, or DELETE statement is executed. No EXCLUSIVE\nlock is acquired until either the memory cache fills up and must\nbe spilled to disk or until the transaction commits. In this way,\nthe system delays blocking read access to the file until the\nlast possible moment.\nThe SQL command \"COMMIT\" does not actually commit the changes to\ndisk. It just turns autocommit back on. Then, at the conclusion of\nthe command, the regular autocommit logic takes over and causes the\nactual commit to disk to occur.\nThe SQL command \"ROLLBACK\" also operates by turning autocommit back on,\nbut it also sets a flag that tells the autocommit logic to rollback rather\nthan commit.\nIf the SQL COMMIT command turns autocommit on and the autocommit logic\nthen tries to commit change but fails because some other process is holding\na SHARED lock, then autocommit is turned back off automatically. This\nallows the user to retry the COMMIT at a later time after the SHARED lock\nhas had an opportunity to clear.\nThis page was last updated on 2025-05-31 13:08:22Z\n",
            "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
          },
          {
            "id": "S3",
            "title": "SQLite Write-Ahead Logging",
            "locator": "https://www.sqlite.org/wal.html — retrieved 2026-11-04T12:20:21.562425+00:00; captured document, not release-pinned; reading-text SHA-256 3a5d7be15c9c8c9542318722bd294cc7b2289d5771811cbe7e4eb655069c003f",
            "text": "Write-Ahead Logging\nTable Of Contents\n1. Overview\n2. How WAL Works\n2.1. Checkpointing\n2.2. Concurrency\n2.3. Performance Considerations\n3. Activating And Configuring WAL Mode\n3.1. Automatic Checkpoint\n3.2. Application-Initiated Checkpoints\n3.3. Persistence of WAL mode\n4. The WAL File\n5. Read-Only Databases\n6. Avoiding Excessively Large WAL Files\n7. Implementation Of Shared-Memory For The WAL-Index\n8. Use of WAL Without Shared-Memory\n9. Sometimes Queries Return SQLITE_BUSY In WAL Mode\n10. Backwards Compatibility\n11. The WAL-Reset Bug\n11.1. Bug Details\n11.2. Low Probability Of Occurrence\n1. Overview\nThe default method by which SQLite implements\natomic commit and rollback is a rollback journal.\nBeginning with version 3.7.0 (2010-07-21), a new \"Write-Ahead Log\" option\n(hereafter referred to as \"WAL\") is available.\nThere are advantages and disadvantages to using WAL instead of\na rollback journal. Advantages include:\nWAL is significantly faster in most scenarios.\nWAL provides more concurrency as readers do not block writers and\na writer does not block readers. Reading and writing can proceed\nconcurrently.\nDisk I/O operations tends to be more sequential using WAL.\nWAL uses many fewer fsync() operations and is thus less vulnerable to\nproblems on systems where the fsync() system call is broken.\nBut there are also disadvantages:\nAll processes using a database must be on the same host computer;\nWAL does not work over a network filesystem. This is because WAL requires\nall processes to share a small amount of memory and processes on\nseparate host machines obviously cannot share memory with each other.\nTransactions that involve changes against multiple ATTACHed\ndatabases are atomic for each individual database, but are not\natomic across all databases as a set.\nIt is not possible to change the page_size after entering WAL\nmode, either on an empty database or by using VACUUM or by restoring\nfrom a backup using the backup API. You must be in a rollback journal\nmode to change the page size.\n[SOURCE STRIKETHROUGH START] It is not possible to open read-only WAL databases.\nThe opening process must have write privileges for \"-shm\"\nwal-index shared memory file associated with the database, if that\nfile exists, or else write access on the directory containing\nthe database file if the \"-shm\" file does not exist. [SOURCE STRIKETHROUGH END]\nBeginning with version 3.22.0 (2018-01-22), a read-only\nWAL-mode database file can be opened if\nthe -shm and -wal files\nalready exist or those files can be created or the\ndatabase is immutable.\nWAL might be very slightly slower (perhaps 1% or 2% slower)\nthan the traditional rollback-journal approach\nin applications that do mostly reads and seldom write.\nThere is an additional quasi-persistent \"-wal\" file and\n\"-shm\" shared memory file associated with each\ndatabase, which can make SQLite less appealing for use as an\napplication file-format.\nThere is the extra operation of checkpointing which, though automatic\nby default, is still something that application developers need to\nbe mindful of.\n[SOURCE STRIKETHROUGH START] WAL works best with smaller transactions. WAL does\nnot work well for very large transactions. For transactions larger than\nabout 100 megabytes, traditional rollback journal modes will likely\nbe faster. For transactions in excess of a gigabyte, WAL mode may\nfail with an I/O or disk-full error.\nIt is recommended that one of the rollback journal modes be used for\ntransactions larger than a few dozen megabytes. [SOURCE STRIKETHROUGH END]\nBeginning with version 3.11.0 (2016-02-15),\nWAL mode works as efficiently with\nlarge transactions as does rollback mode.\n2. How WAL Works\nThe traditional rollback journal works by writing a copy of the\noriginal unchanged database content into a separate rollback journal file\nand then writing changes directly into the database file. In the\nevent of a crash or ROLLBACK, the original content contained in the\nrollback journal is played back into the database file to\nrevert the database file to its original state. The COMMIT occurs\nwhen the rollback journal is deleted.\nThe WAL approach inverts this. The original content is preserved\nin the database file and the changes are appended into a separate\nWAL file. A COMMIT occurs when a special record indicating a commit\nis appended to the WAL. Thus a COMMIT can happen without ever writing\nto the original database, which allows readers to continue operating\nfrom the original unaltered database while changes are simultaneously being\ncommitted into the WAL. Multiple transactions can be appended to the\nend of a single WAL file.\n2.1. Checkpointing\nOf course, one wants to eventually transfer all the transactions that\nare appended in the WAL file back into the original database. Moving\nthe WAL file transactions back into the database is called a\n\"checkpoint\".\nAnother way to think about the difference between rollback and\nwrite-ahead log is that in the rollback-journal\napproach, there are two primitive operations, reading and writing,\nwhereas with a write-ahead log\nthere are now three primitive operations: reading, writing, and\ncheckpointing.\nBy default, SQLite does a checkpoint automatically when the WAL file\nreaches a threshold size of 1000 pages. (The\nSQLITE_DEFAULT_WAL_AUTOCHECKPOINT compile-time option can be used to\nspecify a different default.) Applications using WAL do\nnot have to do anything in order for these checkpoints to occur.\nBut if they want to, applications can adjust the automatic checkpoint\nthreshold. Or they can turn off the automatic checkpoints and run\ncheckpoints during idle moments or in a separate thread or process.\n2.2. Concurrency\nWhen a read operation begins on a WAL-mode database, it first\nremembers the location of the last valid commit record in the WAL.\nCall this point the \"end mark\". Because the WAL can be growing and\nadding new commit records while various readers connect to the database,\neach reader can potentially have its own end mark. But for any\nparticular reader, the end mark is unchanged for the duration of the\ntransaction, thus ensuring that a single read transaction only sees\nthe database content as it existed at a single point in time.\nWhen a reader needs a page of content, it first checks the WAL to\nsee if that page appears there, and if so it pulls in the last copy\nof the page that occurs in the WAL prior to the reader's end mark.\nIf no copy of the page exists in the WAL prior to the reader's end mark,\nthen the page is read from the original database file. Readers can\nexist in separate processes, so to avoid forcing every reader to scan\nthe entire WAL looking for pages (the WAL file can grow to\nmultiple megabytes, depending on how often checkpoints are run), a\ndata structure called the \"wal-index\" is maintained in shared memory\nwhich helps readers locate pages in the WAL quickly and with a minimum\nof I/O. The wal-index greatly improves the performance of readers,\nbut the use of shared memory means that all readers must exist on the\nsame machine. This is why the write-ahead log implementation will not\nwork on a network filesystem.\nWriters merely append new content to the end of the WAL file.\nBecause writers do nothing that would interfere with the actions of\nreaders, writers and readers can run at the same time. However,\nsince there is only one WAL file, there can only be one writer at\na time.\nA checkpoint operation takes content from the WAL file\nand transfers it back into the original database file.\nA checkpoint can run concurrently with readers, however the checkpoint\nmust stop when it reaches a page in the WAL that is past the end mark\nof any current reader. The checkpoint has to stop at that point because\notherwise it might overwrite part of the database file that the reader\nis actively using. The checkpoint remembers (in the wal-index) how far\nit got and will resume transferring content from the WAL to the database\nfrom where it left off on the next invocation.\nThus a long-running read transaction can prevent a checkpointer from\nmaking progress. But presumably every read transaction will eventually\nend and the checkpointer will be able to continue.\nWhenever a write operation occurs, the writer checks how much progress\nthe checkpointer has made, and if the entire WAL has been transferred into\nthe database and synced and if no readers are making use of the WAL, then\nthe writer will rewind the WAL back to the beginning and start putting new\ntransactions at the beginning of the WAL. This mechanism prevents a WAL\nfile from growing without bound.\n2.3. Performance Considerations\nWrite transactions are very fast since they only involve writing\nthe content once (versus twice for rollback-journal transactions)\nand because the writes are all sequential. Further, syncing the\ncontent to the disk is not required, as long as the application is\nwilling to sacrifice durability following a power loss or hard reboot.\n(Writers sync the WAL on every transaction commit if\nPRAGMA synchronous is set to FULL but omit this sync if\nPRAGMA synchronous is set to NORMAL.)\nOn the other hand, read performance deteriorates as the WAL file\ngrows in size since each reader must check the WAL file for the content\nand the time needed to check the WAL file is proportional\nto the size of the WAL file. The wal-index helps find content\nin the WAL file much faster, but performance still falls off with\nincreasing WAL file size. Hence, to maintain good read performance\nit is important to keep the WAL file size down by\nrunning checkpoints at regular intervals.\nCheckpointing does require sync operations in order to avoid\nthe possibility of database corruption following a power loss\nor hard reboot. The WAL must be synced to persistent storage\nprior to moving content from the WAL into the database and the\ndatabase file must be synced prior to resetting the WAL.\nCheckpoint also requires more seeking.\nThe checkpointer makes an effort to\ndo as many sequential page writes to the database as it can (the pages\nare transferred from WAL to database in ascending order) but even\nthen there will typically be many seek operations interspersed among\nthe page writes. These factors combine to make checkpoints slower than\nwrite transactions.\nThe default strategy is to allow successive write transactions to\ngrow the WAL until the WAL becomes about 1000 pages in size, then to\nrun a checkpoint operation for each subsequent COMMIT until the WAL\nis reset to be smaller than 1000 pages. By default, the checkpoint will be\nrun automatically by the same thread that does the COMMIT that pushes\nthe WAL over its size limit. This has the effect of causing most\nCOMMIT operations to be very fast but an occasional COMMIT (those that trigger\na checkpoint) to be much slower. If that effect is undesirable, then\nthe application can disable automatic checkpointing and run the\nperiodic checkpoints in a separate thread, or separate process.\n(Links to commands and interfaces to accomplish this are\nshown below.)\nNote that with PRAGMA synchronous set to NORMAL, the checkpoint\nis the only operation to issue an I/O barrier or sync operation\n(fsync() on unix or FlushFileBuffers() on windows). If an application\ntherefore runs checkpoint in a separate thread or process, the main\nthread or process that is doing database queries and updates will never\nblock on a sync operation. This helps to prevent \"latch-up\" in applications\nrunning on a busy disk drive. The downside to\nthis configuration is that transactions are no longer durable and\nmight rollback following a power failure or hard reset.\nNotice too that there is a tradeoff between average read performance\nand average write performance. To maximize the read performance,\none wants to keep the WAL as small as possible and hence run checkpoints\nfrequently, perhaps as often as every COMMIT. To maximize\nwrite performance, one wants to amortize the cost of each checkpoint\nover as many writes as possible, meaning that one wants to run checkpoints\ninfrequently and let the WAL grow as large as possible before each\ncheckpoint. The decision of how often to run checkpoints may therefore\nvary from one application to another depending on the relative read\nand write performance requirements of the application.\nThe default strategy is to run a checkpoint once the WAL\nreaches 1000 pages and this strategy seems to work well in test applications on\nworkstations, but other strategies might work better on different\nplatforms or for different workloads.\n3. Activating And Configuring WAL Mode\nAn SQLite database connection defaults to\njournal_mode=DELETE. To convert to WAL mode, use the\nfollowing pragma:\nPRAGMA journal_mode=WAL;\nThe journal_mode pragma returns a string which is the new journal mode.\nOn success, the pragma will return the string \"wal\". If\nthe conversion to WAL could not be completed (for example, if the VFS\ndoes not support the necessary shared-memory primitives) then the\njournaling mode will be unchanged and the string returned from the\nprimitive will be the prior journaling mode (for example \"delete\").\n3.1. Automatic Checkpoint\nBy default, SQLite will automatically checkpoint whenever a COMMIT\noccurs that causes the WAL file to be 1000 pages or more in size, or when the\nlast database connection on a database file closes. The default\nconfiguration is intended to work well for most applications.\nBut programs that want more control can force a checkpoint\nusing the wal_checkpoint pragma or by calling the\nsqlite3_wal_checkpoint() C interface. The automatic checkpoint\nthreshold can be changed or automatic checkpointing can be completely\ndisabled using the wal_autocheckpoint pragma or by calling the\nsqlite3_wal_autocheckpoint() C interface. A program can also\nuse sqlite3_wal_hook() to register a callback to be invoked whenever\nany transaction commits to the WAL. This callback can then invoke\nsqlite3_wal_checkpoint() or sqlite3_wal_checkpoint_v2() based on whatever\ncriteria it thinks is appropriate. (The automatic checkpoint mechanism\nis implemented as a simple wrapper around sqlite3_wal_hook().)\n3.2. Application-Initiated Checkpoints\nAn application can initiate a checkpoint using any writable database\nconnection on the database simply by invoking\nsqlite3_wal_checkpoint() or sqlite3_wal_checkpoint_v2().\nThere are three subtypes of checkpoints that vary in their aggressiveness:\nPASSIVE, FULL, and RESTART. The default checkpoint style is PASSIVE, which\ndoes as much work as it can without interfering with other database\nconnections, and which might not run to completion if there are\nconcurrent readers or writers.\nAll checkpoints initiated by sqlite3_wal_checkpoint() and\nby the automatic checkpoint mechanism are PASSIVE. FULL and RESTART\ncheckpoints try harder to run the checkpoint to completion and can only\nbe initiated by a call to sqlite3_wal_checkpoint_v2(). See the\nsqlite3_wal_checkpoint_v2() documentation for additional information\non FULL and RESET checkpoints.\n3.3. Persistence of WAL mode\nUnlike the other journaling modes,\nPRAGMA journal_mode=WAL is\npersistent. If a process sets WAL mode, then closes and reopens the\ndatabase, the database will come back in WAL mode. In contrast, if\na process sets (for example) PRAGMA journal_mode=TRUNCATE and then closes and\nreopens the database will come back up in the default rollback mode of\nDELETE rather than the previous TRUNCATE setting.\nThe persistence of WAL mode means that applications can be converted\nto using SQLite in WAL mode without making any changes to the application\nitself. One has merely to run \"PRAGMA journal_mode=WAL;\" on the\ndatabase file(s) using the command-line shell or other utility, then\nrestart the application.\nThe WAL journal mode will be set on all\nconnections to the same database file if it is set on any one connection.\n4. The WAL File\nWhile a database connection is open on a WAL-mode database, SQLite\nmaintains an extra journal file called a \"Write Ahead Log\" or \"WAL File\".\nThe name of this file on disk is usually the name of the database file\nwith an extra \"-wal\" suffix, though different naming rules may\napply if SQLite is compiled with SQLITE_ENABLE_8_3_NAMES.\nThe WAL file exists for as long as any database connection has the\ndatabase open. Usually, the WAL file is deleted automatically when the\nlast connection to the database closes. However, if the last process to\nhave the database open exits without cleanly\nshutting down the database connection, or if the\nSQLITE_FCNTL_PERSIST_WAL file control is used, then the WAL file\nmight be retained on disk after all connections to the database have\nbeen closed. The WAL file is part of the persistent state of the\ndatabase and should be kept with the database if the database is copied\nor moved. If a database file is separated from its WAL file, then\ntransactions that were previously committed to the database might be lost,\nor the database file might become corrupted.\nThe only safe way to remove a WAL file is\nto open the database file using one of the sqlite3_open() interfaces\nthen immediately close the database using sqlite3_close().\nThe WAL file format is precisely defined and is cross-platform.\n5. Read-Only Databases\nOlder versions of SQLite could not read a WAL-mode database that was\nread-only. In other words, write access was required in order to read a\nWAL-mode database. This constraint was relaxed beginning with\nSQLite version 3.22.0 (2018-01-22).\nOn newer versions of SQLite,\na WAL-mode database on read-only media, or a WAL-mode database that lacks\nwrite permission, can still be read as long as one or more of the following\nconditions are met:\nThe -shm and -wal files already exist and are readable.\nThere is write permission on the directory containing the database so\nthat the -shm and -wal files can be created.\nThe database connection is opened using the\nimmutable query parameter.\nEven though it is possible to open a read-only WAL-mode database,\nit is good practice to convert the database to\nPRAGMA journal_mode=DELETE prior to burning an\nSQLite database image onto read-only media.\n6. Avoiding Excessively Large WAL Files\nIn normal cases, new content is appended to the WAL file until the\nWAL file accumulates about 1000 pages (and is thus about 4MB\nin size) at which point a checkpoint is automatically run and the WAL file\nis recycled. The checkpoint does not normally truncate the WAL file\n(unless the journal_size_limit pragma is set). Instead, it merely\ncauses SQLite to start overwriting the WAL file from the beginning.\nThis is done because it is normally faster to overwrite an existing file\nthan to append. When the last connection to a database closes, that\nconnection does one last checkpoint and then deletes the WAL and its\nassociated shared-memory file, to clean up the disk.\nSo in the vast majority of cases, applications need not worry about\nthe WAL file at all. SQLite will automatically take care of it. But\nit is possible to get SQLite into a state where the WAL file will grow\nwithout bound, causing excess disk space usage and slow query speeds.\nThe following bullets enumerate some of the ways that this can happen\nand how to avoid them.\nDisabling the automatic checkpoint mechanism.\nIn its default configuration, SQLite will checkpoint the WAL file at the\nconclusion of any transaction when the WAL file is more than 1000 pages\nlong. However, compile-time and run-time options exist that can disable\nor defer this automatic checkpoint. If an application disables the\nautomatic checkpoint, then there is nothing to prevent the WAL file\nfrom growing excessively.\nCheckpoint starvation.\nA checkpoint is only able to run to completion, and reset the WAL file,\nif there are no other database connections using the WAL file. If another\nconnection has a read transaction open,\nthen the checkpoint cannot reset the WAL file because\ndoing so might delete content out from under the reader.\nThe checkpoint will do as much work as it can without upsetting the\nreader, but it cannot run to completion.\nThe checkpoint will start up again where it left off after the next\nwrite transaction. This repeats until some checkpoint is able to complete.\nHowever, if a database has many concurrent overlapping readers\nand there is always at least one active reader, then\nno checkpoints will be able to complete\nand hence the WAL file will grow without bound.\nThis scenario can be avoided by ensuring that there are \"reader gaps\":\ntimes when no processes are reading from the\ndatabase and that checkpoints are attempted during those times.\nIn applications with many concurrent readers, one might also consider\nrunning manual checkpoints with the SQLITE_CHECKPOINT_RESTART or\nSQLITE_CHECKPOINT_TRUNCATE option which will ensure that the checkpoint\nruns to completion before returning. The disadvantage of using\nSQLITE_CHECKPOINT_RESTART and SQLITE_CHECKPOINT_TRUNCATE is that\nreaders might block while the checkpoint is running.\nVery large write transactions.\nA checkpoint can only complete when no other transactions are running,\nwhich means the WAL file cannot be reset in the middle of a write\ntransaction. So a large change to a large database\nmight result in a large WAL file. The WAL file will be checkpointed\nonce the write transaction completes (assuming there are no other readers\nblocking it) but in the meantime, the file can grow very big.\nAs of SQLite version 3.11.0 (2016-02-15),\nthe WAL file for a single transaction\nshould be proportional in size to the transaction itself. Pages that\nare changed by the transaction should only be written into the WAL file\nonce. However, with older versions of SQLite, the same page might be\nwritten into the WAL file multiple times if the transaction grows larger\nthan the page cache.\n7. Implementation Of Shared-Memory For The WAL-Index\nThe wal-index is implemented using an ordinary file that is\nmmapped for robustness. Early (pre-release) implementations of WAL mode\nstored the wal-index in volatile shared-memory, such as files created in\n/dev/shm on Linux or /tmp on other unix systems. The problem\nwith that approach is that processes with a different root directory\n(changed via chroot)\nwill see different files and hence use different shared memory areas,\nleading to database corruption. Other methods for creating nameless\nshared memory blocks are not portable across the various flavors of\nunix. And we could not find any method to create nameless shared\nmemory blocks on windows. The only way we have found to guarantee\nthat all processes accessing the same database file use the same shared\nmemory is to create the shared memory by mmapping a file in the same\ndirectory as the database itself.\nUsing an ordinary disk file to provide shared memory has the\ndisadvantage that it might actually do unnecessary disk I/O by\nwriting the shared memory to disk. However, the developers do not\nthink this is a major concern since the wal-index rarely exceeds\n32 KiB in size and is never synced. Furthermore, the wal-index\nbacking file is deleted when the last database connection disconnects,\nwhich often prevents any real disk I/O from ever happening.\nSpecialized applications for which the default implementation of\nshared memory is unacceptable can devise alternative methods via a\ncustom VFS.\nFor example, if it is known that a particular database\nwill only be accessed by threads within a single process, the wal-index\ncan be implemented using heap memory instead of true shared memory.\n8. Use of WAL Without Shared-Memory\nBeginning in SQLite version 3.7.4 (2010-12-07),\nWAL databases can be created, read, and\nwritten even if shared memory is unavailable as long as the\nlocking_mode is set to EXCLUSIVE before the first attempted access.\nIn other words, a process can interact with\na WAL database without using shared memory if that\nprocess is guaranteed to be the only process accessing the database.\nThis feature allows WAL databases to be created, read, and written\nby legacy VFSes that lack the \"version 2\" shared-memory\nmethods xShmMap, xShmLock, xShmBarrier, and xShmUnmap on the\nsqlite3_io_methods object.\nIf EXCLUSIVE locking mode\nis set prior to the first WAL-mode\ndatabase access, then SQLite never attempts to call any of the\nshared-memory methods and hence no shared-memory\nwal-index is ever created.\nIn that case, the database connection remains in EXCLUSIVE mode\nas long as the journal mode is WAL; attempts to change the locking\nmode using \"PRAGMA locking_mode=NORMAL;\" are no-ops.\nThe only way to change out of EXCLUSIVE locking mode is to first\nchange out of WAL journal mode.\nIf NORMAL locking mode is in effect for the first WAL-mode database\naccess, then the shared-memory wal-index is created. This means that the\nunderlying VFS must support the \"version 2\" shared-memory.\nIf the VFS does not support shared-memory methods, then the attempt to\nopen a database that is already in WAL mode, or the attempt to convert a\ndatabase into WAL mode, will fail.\nAs long as exactly one connection is using a shared-memory wal-index,\nthe locking mode can be changed freely between NORMAL and EXCLUSIVE.\nIt is only when the shared-memory wal-index is omitted, when the locking\nmode is EXCLUSIVE prior to the first WAL-mode database access, that the\nlocking mode is stuck in EXCLUSIVE.\n9. Sometimes Queries Return SQLITE_BUSY In WAL Mode\nThe second advantage of WAL-mode is that\nwriters do not block readers and readers do not block writers.\nThis is mostly true.\nBut there are some obscure cases where a query against a WAL-mode\ndatabase can return SQLITE_BUSY, so applications should be prepared\nfor that happenstance.\nCases where a query against a WAL-mode database can return SQLITE_BUSY\ninclude the following:\nIf another database connection has the database mode open\nin exclusive locking mode then all queries against the\ndatabase will return SQLITE_BUSY. Both Chrome and Firefox open their\ndatabase files in exclusive locking mode, so attempts to read Chrome or\nFirefox databases while the applications are running will run into this\nproblem, for example.\nWhen the last connection to a particular database is closing,\nthat connection will acquire an exclusive lock for a short time while\nit cleans up the WAL and shared-memory files. If a separate attempt\nis made to open and query the database while the first connection is\nstill in the middle of its cleanup process, the second connection\nmight get an SQLITE_BUSY error.\nIf the last connection to a database crashed, then the first new\nconnection to open the database will start a recovery process. An\nexclusive lock is held during recovery. So if a third database connection\ntries to jump in and query while the second connection is running recovery,\nthe third connection will get an SQLITE_BUSY error.\n10. Backwards Compatibility\nThe database file format is unchanged for WAL mode. However, the\nWAL file and the wal-index are new concepts and so older versions of\nSQLite will not know\nhow to recover a crashed SQLite database that was operating in WAL mode\nwhen the crash occurred.\nTo prevent older versions of SQLite (prior to version 3.7.0, 2010-07-22)\nfrom trying to recover\na WAL-mode database (and making matters worse) the database file format\nversion numbers (bytes 18 and 19 in the database header)\nare increased from 1 to 2 in WAL mode.\nThus, if an older version of SQLite attempts to connect to an SQLite\ndatabase that is operating in WAL mode, it will report an error along\nthe lines of \"file is encrypted or is not a database\".\nOne can explicitly change out of WAL mode using a pragma such as\nthis:\nPRAGMA journal_mode=DELETE;\nDeliberately changing out of WAL mode changes the database file format\nversion numbers back to 1 so that older versions of SQLite can once again\naccess the database file.\n11. The WAL-Reset Bug\nOn 2026-03-03, one of the SQLite developers (Dan) found and fixed\na bug that could, in rare cases, lead to database corruption. We call\nthis the \"WAL-reset bug\".\nKey points:\nThe bug is likely present in all version of SQLite from\n3.7.0 (2010-07-21) through 3.51.2 (2026-01-09). It is\nfixed in version 3.51.3 (2026-03-13) and later. Backports of the fix\nare available for some earlier releases:\n3.44.6 and\n3.50.7.\nThe bug only affects databases in WAL mode when there are two or more\ndatabase connections open on the same file, in separate threads or processes,\nand when those two connections attempt to write or checkpoint at the\nsame instant.\nThe bug is a data race with tight timing constraints and that requires\nan unusual usage pattern. It is unlikely\nto occur in common use. The developers were unable to reproduce\nthe bug organically and had to add special testing logic to SQLite\nthat deliberately triggers the circumstances of the the bug in order\nto verify that the issue has been fixed.\n11.1. Bug Details\nThis is what happens:\nOne connection does a checkpoint. This first checkpoint must be\ncomplete. In other words, the checkpoint must successfully copy all\ncontent from the WAL file back into the database and leave the WAL file\nin a state where it can potentially be reset.\nShortly after the first checkpoint completes, a second checkpoint\nis started.\nWhile the second checkpoint from step 2 is starting up, another database\nconnection commits a transaction that resets the WAL file and writes new\ncontent into the beginning of the WAL file.\nDue to a data race, the second checkpoint from step 2 does not realize\nthat the WAL file has been reset by the transaction commit in step 3.\nThe second checkpoint leaves a field in the header of the WAL-Index set\nincorrectly. That field indicates that part of the WAL file has already\nbeen checkpointed, when in fact it has not been.\nAdditional transactions are committed to increase the number of pages\nin the WAL file to be more than were present for the first checkpoint\nfrom step 1.\nLater when a third checkpoint occurs, the third checkpoint skips all\nor part of the transaction that was written in step 3. Thus parts of\nthe transaction from step 3 never reach the database file, and the\ndatabase file goes corrupt.\n11.2. Low Probability Of Occurrence\nIn order for this bug to happen, many details must align at just\nthe right moment. So much so that the SQLite developers were unable\nto reproduce the bug organically in the lab. The only way the developers\nhave been able to cause the malfunction was to modify the SQLite sources\nto invoke a callback controlled by sqlite3_test_control()\nthat enables a test program or script to trigger the write transaction\nof step 3 at just the right moment during the second checkpoint.\nWithout that code hack, the problem has never been observed during\ndevelopment and testing.\nUPDATE: 2026-08-24 → Phil Eaton has devised\na reproducer that does not involve using the special testing code hack.\nSee Another look at SQLite's WAL-Reset bug\nfor details.\nThis bug, though rare, does have serious consequences, and so\napplication developers should upgrade to a version of SQLite\nthat fixes the problem.\nHowever, this is not an emergency.\nBased on available telemetry, the occurrence rate of this problem in the\nwild appears to be less than or equal to the expected occurrence rate of\nSSD malfunctions and/or cosmic-ray hits. So even if you are running an\nunpatched version of SQLite, and unless you are doing something really\nunusual, you are unlikely to ever encounter this problem.\nThis page was last updated on 2026-08-25 19:42:39Z\n",
            "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
          },
          {
            "id": "S4",
            "title": "SQLite Online Backup API",
            "locator": "https://www.sqlite.org/backup.html — retrieved 2026-11-04T12:20:21.652115+00:00; captured document, not release-pinned; reading-text SHA-256 247324b2ef9147ec5137ce7b2e083cf7d7ad4eb289b5e625cea40f8400116c10",
            "text": "SQLite Backup API\nTable Of Contents\n1. Using the SQLite Online Backup API\n1.1. Other Backup Techniques\n2. Example 1: Loading and Saving In-Memory Databases\n2.1. Error handling\n2.2. Possible Enhancements\n3. Example 2: Online Backup of a Running Database\n3.1. File and Database Connection Locking\n3.2. backup_remaining() and backup_pagecount()\n1. Using the SQLite Online Backup API\nHistorically, backups (copies) of SQLite databases have been created\nusing the following method:\nEstablish a shared lock on the database file using the SQLite API (i.e.\nthe shell tool).\nCopy the database file using an external tool (for example the unix 'cp'\nutility or the DOS 'copy' command).\nRelinquish the shared lock on the database file obtained in step 1.\nThis procedure works well in many scenarios and is usually very\nfast. However, this technique has the following shortcomings:\nAny database clients wishing to write to the database file while a\nbackup is being created must wait until the shared lock is\nrelinquished.\nIt cannot be used to copy data to or from in-memory databases.\nIf a power failure or operating system failure occurs while copying\nthe database file the backup database may be corrupted following\nsystem recovery.\nThe Online Backup API was created to\naddress these concerns. The online backup API allows the contents of\none database to be copied into another database file, replacing any\noriginal contents of the target database. The copy operation may be\ndone incrementally, in which case the source database does not need\nto be locked for the duration of the copy, only for the brief periods\nof time when it is actually being read from. This allows other database\nusers to continue without excessive delays while a backup of an online\ndatabase is made.\nThe effect of completing the backup call sequence is to make the\ndestination a bit-wise identical copy of the source database as it\nwas when the copying commenced. (The destination becomes a \"snapshot.\")\nThe online backup API is documented here.\nThe remainder of this page contains two C language examples illustrating\ncommon uses of the API and discussions thereof. Reading these examples\nis no substitute for reading the API documentation!\n1.1. Other Backup Techniques\nThe Online Backup API is the original method for making backups of a\nlive SQLite databases. Other more recent techniques for accomplishing\nthe same thing include:\nThe VACUUM INTO command will make a vacuumed copy of a live\nSQLite database into a separate file.\nThe sqlite3_rsync program makes a copy of a live SQLite database\nto or from a remote system using an SSH connection.\n2. Example 1: Loading and Saving In-Memory Databases\n/*\n** This function is used to load the contents of a database file on disk\n** into the \"main\" database of open database connection pInMemory, or\n** to save the current contents of the database opened by pInMemory into\n** a database file on disk. pInMemory is probably an in-memory database,\n** but this function will also work fine if it is not.\n**\n** Parameter zFilename points to a nul-terminated string containing the\n** name of the database file on disk to load from or save to. If parameter\n** isSave is non-zero, then the contents of the file zFilename are\n** overwritten with the contents of the database opened by pInMemory. If\n** parameter isSave is zero, then the contents of the database opened by\n** pInMemory are replaced by data loaded from the file zFilename.\n**\n** If the operation is successful, SQLITE_OK is returned. Otherwise, if\n** an error occurs, an SQLite error code is returned.\n*/\nint loadOrSaveDb(sqlite3 *pInMemory, const char *zFilename, int isSave){\nint rc; /* Function return code */\nsqlite3 *pFile; /* Database connection opened on zFilename */\nsqlite3_backup *pBackup; /* Backup object used to copy data */\nsqlite3 *pTo; /* Database to copy to (pFile or pInMemory) */\nsqlite3 *pFrom; /* Database to copy from (pFile or pInMemory) */\n/* Open the database file identified by zFilename. Exit early if this fails\n** for any reason. */\nrc = sqlite3_open(zFilename, &pFile);\nif( rc==SQLITE_OK ){\n/* If this is a 'load' operation (isSave==0), then data is copied\n** from the database file just opened to database pInMemory.\n** Otherwise, if this is a 'save' operation (isSave==1), then data\n** is copied from pInMemory to pFile. Set the variables pFrom and\n** pTo accordingly. */\npFrom = (isSave ? pInMemory : pFile);\npTo = (isSave ? pFile : pInMemory);\n/* Set up the backup procedure to copy from the \"main\" database of\n** connection pFile to the main database of connection pInMemory.\n** If something goes wrong, pBackup will be set to NULL and an error\n** code and message left in connection pTo.\n**\n** If the backup object is successfully created, call backup_step()\n** to copy data from pFile to pInMemory. Then call backup_finish()\n** to release resources associated with the pBackup object. If an\n** error occurred, then an error code and message will be left in\n** connection pTo. If no error occurred, then the error code belonging\n** to pTo is set to SQLITE_OK.\n*/\npBackup = sqlite3_backup_init(pTo, \"main\", pFrom, \"main\");\nif( pBackup ){\n(void)sqlite3_backup_step(pBackup, -1);\n(void)sqlite3_backup_finish(pBackup);\n}\nrc = sqlite3_errcode(pTo);\n}\n/* Close the database connection opened on database file zFilename\n** and return the result of this function. */\n(void)sqlite3_close(pFile);\nreturn rc;\n}\nThe C function above demonstrates one of the simplest,\nand most common, uses of the backup API: loading and saving the contents\nof an in-memory database to a file on disk. The backup API is used as\nfollows in this example:\nFunction sqlite3_backup_init() is called to create an sqlite3_backup\nobject to copy data between the two databases (either from a file and\ninto the in-memory database, or vice-versa).\nFunction sqlite3_backup_step() is called with a parameter of\n-1 to copy the entire source database to the destination.\nFunction sqlite3_backup_finish() is called to clean up resources\nallocated by sqlite3_backup_init().\n2.1. Error handling\nIf an error occurs in any of the three main backup API routines\nthen the error code and message are attached to\nthe destination database connection.\nAdditionally, if\nsqlite3_backup_step() encounters an error, then the error code is returned\nby both the sqlite3_backup_step() call itself, and by the subsequent call\nto sqlite3_backup_finish(). So a call to sqlite3_backup_finish()\ndoes not overwrite an error code stored in the destination\ndatabase connection by sqlite3_backup_step(). This feature is used\nin the example code to reduce the amount of error handling required.\nThe return values of the sqlite3_backup_step() and sqlite3_backup_finish()\ncalls are ignored and the error code indicating the success or failure of\nthe copy operation is collected from the destination database connection\nafterward.\n2.2. Possible Enhancements\nThe implementation of this function could be enhanced in at least two ways:\nFailing to obtain the lock on database file zFilename (an SQLITE_BUSY\nerror) could be handled, and\nCases where the page-sizes of database pInMemory and zFilename are\ndifferent could be handled better.\nSince database zFilename is a file on disk, then it may be accessed\nexternally by another process. This means that when the call to\nsqlite3_backup_step() attempts to read from or write data to it, it may\nfail to obtain the required file lock. If this happens, this implementation\nwill fail, returning SQLITE_BUSY immediately. The solution would be to\nregister a busy-handler callback or\ntimeout with database connection pFile\nusing sqlite3_busy_handler() or sqlite3_busy_timeout()\nas soon as it is opened. If it fails to obtain a required lock immediately,\nsqlite3_backup_step() uses any registered busy-handler callback or timeout\nin the same way as sqlite3_step() or sqlite3_exec() does.\nUsually, it does not matter if the page-sizes of the source database and the\ndestination database are different before the contents of the destination\nare overwritten. The page-size of the destination database is simply changed\nas part of the backup operation. The exception is if the destination database\nhappens to be an in-memory database. In this case, if the page sizes\nare not the same at the start of the backup operation, then the operation\nfails with an SQLITE_READONLY error. Unfortunately, this could occur when\nloading a database image from a file into an in-memory database using\nfunction loadOrSaveDb().\nHowever, if in-memory database pInMemory has just been opened (and is\ntherefore completely empty) before being passed to function loadOrSaveDb(),\nthen it is still possible to change its page size using an SQLite \"PRAGMA\npage_size\" command. Function loadOrSaveDb() could detect this case, and\nattempt to set the page-size of the in-memory database to the page-size\nof database zFilename before invoking the online backup API functions.\n3. Example 2: Online Backup of a Running Database\n/*\n** Perform an online backup of database pDb to the database file named\n** by zFilename. This function copies 5 database pages from pDb to\n** zFilename, then unlocks pDb and sleeps for 250 ms, then repeats the\n** process until the entire database is backed up.\n**\n** The third argument passed to this function must be a pointer to a progress\n** function. After each set of 5 pages is backed up, the progress function\n** is invoked with two integer parameters: the number of pages left to\n** copy, and the total number of pages in the source file. This information\n** may be used, for example, to update a GUI progress bar.\n**\n** While this function is running, another thread may use the database pDb, or\n** another process may access the underlying database file via a separate\n** connection.\n**\n** If the backup process is successfully completed, SQLITE_OK is returned.\n** Otherwise, if an error occurs, an SQLite error code is returned.\n*/\nint backupDb(\nsqlite3 *pDb, /* Database to back up */\nconst char *zFilename, /* Name of file to back up to */\nvoid(*xProgress)(int, int) /* Progress function to invoke */\n){\nint rc; /* Function return code */\nsqlite3 *pFile; /* Database connection opened on zFilename */\nsqlite3_backup *pBackup; /* Backup handle used to copy data */\n/* Open the database file identified by zFilename. */\nrc = sqlite3_open(zFilename, &pFile);\nif( rc==SQLITE_OK ){\n/* Open the sqlite3_backup object used to accomplish the transfer */\npBackup = sqlite3_backup_init(pFile, \"main\", pDb, \"main\");\nif( pBackup ){\n/* Each iteration of this loop copies 5 database pages from database\n** pDb to the backup database. If the return value of backup_step()\n** indicates that there are still further pages to copy, sleep for\n** 250 ms before repeating. */\ndo {\nrc = sqlite3_backup_step(pBackup, 5);\nxProgress(\nsqlite3_backup_remaining(pBackup),\nsqlite3_backup_pagecount(pBackup)\n);\nif( rc==SQLITE_OK || rc==SQLITE_BUSY || rc==SQLITE_LOCKED ){\nsqlite3_sleep(250);\n}\n} while( rc==SQLITE_OK || rc==SQLITE_BUSY || rc==SQLITE_LOCKED );\n/* Release resources allocated by backup_init(). */\n(void)sqlite3_backup_finish(pBackup);\n}\nrc = sqlite3_errcode(pFile);\n}\n/* Close the database connection opened on database file zFilename\n** and return the result of this function. */\n(void)sqlite3_close(pFile);\nreturn rc;\n}\nThe function presented in the previous example copies the entire source\ndatabase in one call to sqlite3_backup_step(). This requires holding a\nread-lock on the source database file for the duration of the operation,\npreventing any other database user from writing to the database. It also\nholds the mutex associated with database pInMemory throughout the copy,\npreventing any other thread from using it. The C function in this section,\ndesigned to be called by a background thread or process for creating a\nbackup of an online database, avoids these problems using the following\napproach:\nFunction sqlite3_backup_init() is called to create an sqlite3_backup\nobject to copy data from database pDb to the backup database file\nidentified by zFilename.\nFunction sqlite3_backup_step() is called with a parameter of 5 to\ncopy 5 pages of database pDb to the backup database (file zFilename).\nIf there are still more pages to copy from database pDb, then the\nfunction sleeps for 250 milliseconds (using the sqlite3_sleep()\nutility) and then returns to step 2.\nFunction sqlite3_backup_finish() is called to clean up resources\nallocated by sqlite3_backup_init().\n3.1. File and Database Connection Locking\nDuring the 250 ms sleep in step 3 above, no read-lock is held on the database\nfile and the mutex associated with pDb is not held. This allows other threads\nto use database connection pDb and other connections to write to the\nunderlying database file.\nIf another thread or process writes to the source database while this\nfunction is sleeping, then SQLite detects this and usually restarts the\nbackup process when sqlite3_backup_step() is next called. There is one\nexception to this rule: If the source database is not an in-memory database,\nand the write is performed from within the same process as the backup\noperation and uses the same database handle (pDb), then the destination\ndatabase (the one opened using connection pFile) is automatically updated\nalong with the source. The backup process may then be continued after the\nsqlite3_sleep() call returns as if nothing had happened.\nWhether or not the backup process is restarted as a result of writes to\nthe source database mid-backup, the user can be sure that when the backup\noperation is completed the backup database contains a consistent and\nup-to-date snapshot of the original. However:\nWrites to an in-memory source database, or writes to a file-based\nsource database by an external process or thread using a\ndatabase connection other than pDb are significantly more expensive\nthan writes made to a file-based source database using pDb (as the\nentire backup operation must be restarted in the former two cases).\nIf the backup process is restarted frequently enough it may never\nrun to completion and the backupDb() function may never return.\n3.2. backup_remaining() and backup_pagecount()\nThe backupDb() function uses the sqlite3_backup_remaining() and\nsqlite3_backup_pagecount() functions to report its progress via the\nuser-supplied xProgress() callback. Function sqlite3_backup_remaining()\nreturns the number of pages left to copy and sqlite3_backup_pagecount()\nreturns the total number of pages in the source database (in this case\nthe database opened by pDb). So the percentage completion of the process\nmay be calculated as:\nCompletion = 100% * (pagecount() - remaining()) / pagecount()\nThe sqlite3_backup_remaining() and sqlite3_backup_pagecount() APIs report\nvalues stored by the previous call to sqlite3_backup_step(), they do not\nactually inspect the source database file. This means that if the source\ndatabase is written to by another thread or process after the call to\nsqlite3_backup_step() returns but before the values returned by\nsqlite3_backup_remaining() and sqlite3_backup_pagecount() are used, the\nvalues may be technically incorrect. This is not usually a problem.\nThis page was last updated on 2025-11-13 07:12:58Z\n",
            "rights": "SQLite documentation: the official copyright page states code and documentation are dedicated to the public domain, with qualifications below. No SQLite endorsement. Captured HTML was transformed to reading text: block/line whitespace normalized, historical strikethrough explicitly marked, superscripts expressed as ^(...); source prose not deliberately excerpted. Original HTML hashes and retrieval times are in retrieval.json. This is not original code formatting or a commit-pinned release.\n\nSQLite Is Public Domain\nSQLite is in the\nPublic Domain\nAll of the code and documentation in SQLite has been dedicated to the\npublic domain\nby the authors.\nAll code authors, and representatives of the companies they work for,\nhave signed affidavits dedicating their contributions to\nthe public domain and originals of\nthose signed affidavits are stored in a firesafe at the main offices\nof Hwaci. All contributors are citizens\nof countries that allow creative works to be dedicated into the public\ndomain.\nAnyone is free to copy, modify, publish, use, compile, sell, or distribute\nthe original SQLite code, either in source code form or as a compiled binary,\nfor any purpose, commercial or non-commercial, and by any means.\nThe previous paragraph applies to the deliverable code and documentation\nin SQLite - those parts of the SQLite library that you actually bundle and\nship with a larger application. Some scripts used as part of the\nbuild process (for example the \"configure\" scripts generated by autoconf)\nmight fall under other open-source licenses. Nothing from these build\nscripts ever reaches the final deliverable SQLite library, however, and\nso the licenses associated with those scripts should not be\na factor in assessing your rights to copy and use the SQLite library.\nAll of the deliverable code in SQLite has been written from scratch.\nNo code has been taken from other projects or from the open\ninternet. Every line of code can be traced back to its original\nauthor, and all of those authors have public domain dedications\non file. So the SQLite code base is clean and is\nuncontaminated with licensed code from other projects.\nOpen-Source, not Open-Contribution\nSQLite is open-source, meaning that you can make as many copies of it as\nyou want and do whatever you want with those copies, without limitation.\nBut SQLite is not open-contribution. In order to keep SQLite in the public\ndomain and ensure that the code does not become contaminated with proprietary\nor licensed content, the project does not accept patches from people who\nhave not submitted an affidavit dedicating their contribution into the\npublic domain.\nAll of the code in SQLite is original, having been written\nspecifically for use by SQLite. No code has been copied from unknown\nsources on the internet.\nWarranty of Title\nSQLite is in the public domain and does not require a license.\nEven so, some organizations want legal proof of their right to use\nSQLite. Circumstances where this might occur include the following:\nYour company desires indemnity against claims of copyright infringement.\nYou are using SQLite in a jurisdiction that does not recognize\nthe public domain.\nYou are using SQLite in a jurisdiction that does not recognize\nthe right of an author to dedicate their work to the public\ndomain.\nYou want to hold a tangible legal document\nas evidence that you have the legal right to use and distribute\nSQLite.\nYour legal department tells you that you must purchase a license.\nIf any of the above circumstances apply to you,\nHwaci, the company that employs\nall the developers of SQLite, will\nsell you\na Warranty of Title for SQLite.\nA Warranty of Title is a legal document that asserts that the claimed\nauthors of SQLite are the true authors, and that the authors\nhave the legal right to dedicate the SQLite library into the public domain, and\nthat Hwaci will vigorously defend against challenges to those claims.\nAll proceeds from the sale of SQLite Warranties of Title are used to fund\ncontinuing improvement and support of SQLite.\nContributed Code\nIn order to keep SQLite completely free and unencumbered by copyright,\nthe project does not accept patches from random people on the internet.\nThere is a process to get a patch accepted, but that process is involved\nand for smaller changes is not normally worth the effort.\nIf you would like to suggest a change and you include a patch as\na proof-of-concept, that would be great. However, please do not be\noffended if we rewrite your patch from scratch.\nThis page was last updated on 2026-01-12 12:33:24Z\n"
          }
        ],
        "response": "{\n  \"claims\": [\n    {\n      \"id\": \"C1\",\n      \"text\": \"Do not treat a shared network file as equivalent to an application server. SQLite’s guidance says to avoid direct, simultaneous access to the same database from many computers across a network, without an intervening application server.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"A good rule of thumb is to avoid using SQLite\\nin situations where the same database will be accessed directly\\n(without an intervening application server) and simultaneously\\nfrom many computers over a network.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C2\",\n      \"text\": \"A networked team application can still be a candidate for SQLite when its SQL-issuing application server and database content live on the same machine. The end user being another network hop away is not, by itself, the same as putting the database file across that hop.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"In this rule, \\\"application\\\" means the code that issues SQL statements.\\nIf the \\\"application\\\" is an application server and\\nif the content resides on the same physical machine as the application server,\\nthen SQLite might still be appropriate even though the end user is\\nanother network hop away.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C3\",\n      \"text\": \"Plan for one writer at a time per database file. If the workload needs simultaneous writes that cannot queue and take turns, SQLite’s checklist favors a client/server database engine. WAL also retains a single writer; improved read/write overlap is not concurrent writing.\",\n      \"citations\": [\n        {\n          \"source\": \"S1\",\n          \"quote\": \"If many threads and/or processes need to write the\\ndatabase at the same instant (and they cannot queue up and take turns)\\nthen it is best to select a database engine that supports that\\ncapability, which always means a client/server database engine.\\nSQLite only supports one writer at a time per database file.\"\n        },\n        {\n          \"source\": \"S3\",\n          \"quote\": \"since there is only one WAL file, there can only be one writer at\\na time.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C4\",\n      \"text\": \"Read locking advice in its journal-mode context. The file-locking document explicitly covers the older rollback mechanism, while the WAL guide describes readers and a writer proceeding concurrently. Do not apply the rollback document’s read/write-lock rules as a blanket description of WAL.\",\n      \"citations\": [\n        {\n          \"source\": \"S2\",\n          \"quote\": \"The document only describes locking for the older rollback-mode\\ntransaction mechanism.\"\n        },\n        {\n          \"source\": \"S3\",\n          \"quote\": \"WAL provides more concurrency as readers do not block writers and\\na writer does not block readers. Reading and writing can proceed\\nconcurrently.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C5\",\n      \"text\": \"Turning on WAL is not a workaround for sharing the live database across host machines. The WAL guide requires processes using the database to be on the same host and explicitly says WAL does not work over a network filesystem.\",\n      \"citations\": [\n        {\n          \"source\": \"S3\",\n          \"quote\": \"All processes using a database must be on the same host computer;\\nWAL does not work over a network filesystem. This is because WAL requires\\nall processes to share a small amount of memory and processes on\\nseparate host machines obviously cannot share memory with each other.\"\n        }\n      ]\n    },\n    {\n      \"id\": \"C6\",\n      \"text\": \"For online backups, evaluate a database-aware method rather than assuming a live file copy is enough. The Online Backup API can copy incrementally, taking brief source read locks instead of holding a lock for the whole copy. The guide warns that reading its examples is not a substitute for reading the API documentation.\",\n      \"citations\": [\n        {\n          \"source\": \"S4\",\n          \"quote\": \"The copy operation may be\\ndone incrementally, in which case the source database does not need\\nto be locked for the duration of the copy, only for the brief periods\\nof time when it is actually being read from. This allows other database\\nusers to continue without excessive delays while a backup of an online\\ndatabase is made.\"\n        },\n        {\n          \"source\": \"S4\",\n          \"quote\": \"Reading these examples\\nis no substitute for reading the API documentation!\"\n        }\n      ]\n    }\n  ],\n  \"questions\": [\n    \"Where will the code issuing SQL run: on one application host, or on multiple machines opening the same file directly? Draw that boundary before choosing an engine.\",\n    \"What are the write arrival rate, transaction durations and acceptable queue latency for the actual workload? These documents and this review do not measure them.\",\n    \"Which SQLite version, journal mode and filesystem will be used, and how will representative behavior be tested? Captured documentation is not a configuration audit.\",\n    \"How will a completed backup be verified, restored and kept off the failed host? An API capability is not an observed restore or disaster-recovery guarantee.\"\n  ]\n}",
        "notes": "Same-agent editorial review of four real SQLite documentation captures. No model call, workload measurement, engine change, filesystem test or production restore. The capture adapter was corrected before this work was built: prefatory rollback/WAL scope is retained, and obsolete struck-through WAL statements are explicitly marked rather than presented as current text. Original retrieval times and full HTML hashes remain; old plain-text conversion is preserved separately as transformation evidence, not the source used here. All six selected factual claims require source/version/context review; question prompts identify missing deployment evidence, not measured failures. Initial claim order follows source-reading order. The organized version changes presentation only; it does not prove improved comprehension. SQLite public-domain notice is retained in every source’s rights field."
      },
      "decisions": [
        {
          "id": "C1",
          "decision": "approve"
        },
        {
          "id": "C2",
          "decision": "approve"
        },
        {
          "id": "C3",
          "decision": "approve"
        },
        {
          "id": "C4",
          "decision": "approve"
        },
        {
          "id": "C5",
          "decision": "approve"
        },
        {
          "id": "C6",
          "decision": "approve"
        }
      ]
    }
  ]
}