Berkeley DB Reference Guide:
Berkeley DB Concurrent Data Store Applications

PrevRefNext

Architecting Data Store and Concurrent Data Store applications

When building Data Store and Concurrent Data Store applications, there are two special issues to consider whenever any thread of control exits for any reason with a Berkeley DB database or database environment open.

First, unexpected application or system failure may result in lost data, corruption or inconsistencies in Berkeley DB databases. When a thread of control exits for any reason while holding Berkeley DB resources, any database (modified since the database was last flushed to disk), should be either:

Applications where this is unacceptable should consider the Berkeley DB Transactional Data Store product, which offers standard transactional durability guarantees including recoverability after failure.

Second, unexpected application or system failure requires that any persistent database environment (that is, any database environment not created using the DB_PRIVATE flag), be removed to recover the Berkeley DB resources and release any locks or mutexes that may have been held to avoid starvation as the remaining threads of control block behind the failed thread's locks or mutexes.

The Berkeley DB library cannot determine when to remove and re-create a database environment; the application must make that decision. Furthermore, database environment removal must be single-threaded; that is, one thread of control or process must remove and re-create the database environment before any other thread of control or process attempts to join the Berkeley DB environment.

There are three ways to architect Berkeley DB Data Store and Concurrent Data Store applications. The one chosen is usually based on whether or not the application is comprised of a single process or group of processes descended from a single process (for example, a server started when the system first boots), or if the application is comprised of unrelated processes (for example, processes started by users logged into the system.

  1. The first, and simplest, way to architect applications is as a single, usually multithreaded, process.

    When this process starts, it removes any existing database environment, and removes, restores or verifies the databases as described previously. Then the process re-creates the database environment and opens the databases. From then on, the application can create new threads of control as it chooses. The threads of control can either share Berkeley DB DB_ENV and DB handles, or have their own. In this model, databases are rarely opened or closed when more than a single thread of control is running; that is, they are opened when only a single thread is running, and closed after all threads but one have exited. The last thread of control to exit closes the databases and the environment.

    This architecture is simplest because it requires no monitoring of other threads of control. No cleanup is required if the process or system fails and the application can simply be restarted.

  2. The second way to architect applications is as a group of related processes (the processes may or may not be multithreaded).

    This architecture requires the order in which threads of control are created and subsequently access the Berkeley DB environment be controlled because removal and re-creation of the database environment must be single-threaded (as must be the removal, restoration or verification of the databases). The first thread of control to access the environment must perform the cleanup processing, and no other thread should attempt to access the environment until that cleanup processing is complete.

    In addition, this architecture requires that threads of control be monitored. If any thread of control holding Berkeley DB resources exits without first cleanly discarding those resources, the cleanup of the database environment and databases must be re-done. Before performing the cleanup, all threads using the Berkeley DB environment must relinquish all of their Berkeley DB resources (it does not matter if they do so gracefully or because they are forced to exit). Then, cleanup can be done and the threads of control continued or restarted.

    The easiest way to structure groups of related processes is to first create a single process (often a shell script) that cleans up the database environment, and then creates the processes or threads that will actually perform work. The initial thread has no further responsibilities other than to monitor the threads of control it has created, to ensure that none of them unexpectedly exits. If one exits, the initial process then kill all of the threads of control using the Berkeley DB environment, cleans up, and restarts the working threads of control.

  3. The third way to architect transactional applications is as a group of unrelated processes (the processes may or may not be multithreaded).

    If it is not practical to have a single parent for the processes sharing a Berkeley DB environment, each process sharing the database environment should log their connection to and exit from the environment in a way allowing a monitoring process to detect if a thread of control might have acquired Berkeley DB resources and never released them.

    For example, initial "watcher" process would open/create the Berkeley DB environment, and then create a sentinel file. Any other process wanting to use the Berkeley DB environment checks for the sentinel file; if the sentinel file exists, the other process registers its process ID with the watcher and joins the database environment. When the process finishes with the environment, it unregisters its process ID with the watcher. The watcher periodically checks to ensure that no process has failed while using the environment. If a process does fail while using the environment, the watcher removes the sentinel file, kills all processes currently using the environment, cleans up the environment, and re-creates the sentinel file.

Obviously, when implementing the second and third architectures, it is important that any process monitoring other threads of control be as simple and well-tested as possible, because there is no recourse if it fails.


PrevRefNext

Copyright (c) 1996-2004 Sleepycat Software, Inc. - All rights reserved.