Crash recovery using the RESTART DATABASE command or the automatic restart enable configuration parameter (autorestart) protects a database from being left in an inconsistent, or unusable, state.
The following topics provide additional information:
Database commands and applications can fail for various reasons. A transaction failure is not the failure of a database action when it is caused by an incorrect parameter, a limit being exceeded, or a rollback caused by a deadlock. Rather, it is a severe error or condition that causes the database or database manager to end abnormally, and requires that the database be recovered. Examples include events such as a power failure on a machine (causing the database manager and database partitions on it to be down), or a COMMIT/ROLLBACK failure that causes the database to go down because the disk that contains the database log is full, and no additional log files can be allocated for writing the COMMIT/ROLLBACK record.
While applications or commands are running against a database, an interruption in power or the failure of an application may cause the immediate cessation or stopping of all activity with the database. One or more of the applications or commands may have started working with the data in the database but were not complete. Also, some committed units of work may not have been flushed to disk. The partially completed (or nonflushed) units of work leave the database in an inconsistent, or unusable, state.
See the following topics for more information:
The only consideration is whether you want the rollback of incomplete units of work at the time of a failure to be done automatically by the database manager. If you do, use the automatic restart enable (autorestart) configuration parameter. If not, you should be prepared to issue the RESTART DATABASE command when a database failure occurs.
Automatic restart is enabled through the autorestart database configuration parameter. The default for this parameter is that automatic restart is "on". See "Auto Restart Enable (autorestart)" for more information.
Typically, database recovery is required on both the failed database partition server and any other database partition server that was participating in the same transaction or application. Database recovery on the failed database partition server is often called crash recovery. Crash recovery occurs on the database partition server that failed after the condition that caused the failure is corrected (for example, the power supply is reactivated). Database recovery on the other (still active) database partition servers occurs immediately after the failure is detected. Sometimes called database partition failure recovery, in this recovery process, resources are transparently cleaned up for the failed transaction or application.
For more information, see "Failure Recovery on an Active Database Partition Server", and "Transaction Failure Recovery on the Failed Database Partition Server".
The discussion of two-phase commit protocol here is to introduce crash recovery in a partitioned database system. For more information about two-phase commit, refer to "Understanding the Two-Phase Commit Process".
In a partitioned database environment, the database partition server on which an application is submitted is the coordinator node, and the first agent that works for the application is the coordinator agent. The coordinator agent is responsible for distributing work to other database partition servers, and it keeps track of which ones are involved in the transaction. When the application issues a COMMIT for a transaction, the coordinator agent commits the transaction by using the two-phase commit protocol. In the first phase, the coordinator node distributes a PREPARE request to all the other database partition servers that are participating in the transaction. These servers then respond with one of the following:
If one of the servers responds "NO", the transaction is rolled back. Otherwise, the coordinator node begins the second phase.
In the second phase, the coordinator node writes a COMMIT log record, then distributes a COMMIT request to all the servers that responded "YES". After all the other database partition servers have committed, they send an acknowledgment of the COMMIT to the coordinator node. The transaction is complete when the coordinator agent has received all COMMIT acknowledgments from all the participating servers. At this point, the coordinator agent writes a FORGET log record.
If any database partition server detects that another server is down, all work that is associated with the failed database partition server is stopped:
Any process (such as an agent or deadlock detector) that attempts to send a request to the failed server is informed that it cannot send the request.
If the failure caused the database manager to end abnormally, when the processor is restarted, you can issue DB2START with the RESTART option to restart the database manager. If you cannot restart the processor, you can also use DB2START to restart the database manager on a different processor. For more information, see the START DATABASE MANAGER command and API in the Command Reference and API Reference respectively.
An abnormal end may result in database partitions on the server being left in an inconsistent state (meaning that they are unusable). To make them usable, crash recovery is required to make them consistent. Crash recovery can be triggered on a database partition server:
Crash recovery reapplies the log records in the active log files to ensure that the effect of all complete transactions are in the database. After all the changes are reapplied, all uncommitted transactions are rolled back locally, except for indoubt transactions. In a partitioned database environment, there are two types of indoubt transaction:
Crash recovery attempts to resolve all the indoubt transactions by doing one of the following. The action that is taken depends on whether the database partition server was the coordinator node for an application:
It is possible that crash recovery may not be able to resolve all the indoubt transactions (for example, some of the database partition servers are not available). In this situation, the SQL warning message SQL1061W is returned. You should note that indoubt transactions hold resources, such as locks and active log space. It is possible to get to a point where no changes can be made to the database because the active log space is held up by indoubt transactions. For this reason, you should investigate if indoubt transactions remain after crash recovery, and recover all database partition servers that are required to resolve the indoubt transactions as quickly as possible.
If one or more servers that are required to resolve an indoubt transaction cannot be recovered in time, and access is required to database partitions on other servers, you can manually resolve the indoubt transaction by making an heuristic decision. You can use the LIST INDOUBT TRANSACTIONS command to query, commit, and roll back the indoubt transaction on the server. For more information, see the LIST INDOUBT TRANSACTIONS command and API in the Command Reference and API Reference manuals respectively.
| Note: | The LIST INDOUBT TRANSACTIONS command is also used for transactions in a
distributed transaction environment. See Chapter 8. "Distributed Databases" and Chapter 9. "Using DB2 with an XA-Compliant Transaction Manager" for more information about distributed environments. To distinguish
between the two types of indoubt transactions, the "originator" field in
the output that is returned by LIST INDOUBT TRANSACTIONS displays one of the
following:
|
When a database partition server fails, the application will typically receive one of the following SQLCODEs. The method for detecting which database manager failed depends on the SQLCODE received:
Determining which database partition server failed is a two-step process. The SQLCA associated with SQLCODE SQL1229N contains the node number of the server that detected the error in the sixth array position of the sqlerrd field. (The node number that is written for the server corresponds to the node number in the db2nodes.cfg file.) On the database partition server that detects the error, a message that indicates the node number of the failed server is written in the db2diag.log file.
| Note: | If multiple logical nodes are being used on a processor, the failure of one logical node may cause other logical nodes on the same processor to fail. |
Typically, to recover from the failure of a database partition server: