Oracle® Real Application Clusters Administrator's Guide 10g Release 1 (10.1) Part Number B10765-02 |
|
|
View PDF |
This chapter describes storage topics such as Automated Storage Management (ASM) and the Oracle Cluster Registry (OCR). The topics in this chapter are:
Storage for RAC databases must be shared. In other words, datafiles must reside on either a cluster file system or on shared raw devices. Additionally, for each instance you must create at least two redo log files that reside on shared storage devices. You must also create one shared device for each instance to store its own tablespace for automatic undo management. If needed, you can use a local file system to store, for example, client-side parameter files (PFILEs).
Unless otherwise noted, Oracle storage features such as Automatic Storage Management (ASM), Oracle Managed Files, automatic segment-space management, and so on, function the same in RAC databases as they do in Oracle single-instance databases. Refer to the Oracle Database Administrator's Guide for additional information about these storage features.
If your platform does not support a cluster file system or if you do not want to use a cluster file system to store datafiles for RAC, then you must create additional raw devices as described inOracle Database 2 Day DBA and Oracle Real Application Clusters Installation and Configuration Guide. Oracle recommends that you use ASM for datafile storage as described in the next section.
ASM automatically optimizes storage to maximize performance by rebalancing the storage configuration across the disks that ASM manages. ASM spreads the storage load across all available storage within your cluster database environment for optimal performance. ASM partitions your total disk space into uniformly sized units across all disks in a disk group. ASM can also automatically mirror data to prevent data loss.
To use ASM in RAC, you only need to select ASM as your storage option when you create your database with the Database Configuration Assistant (DBCA). When your database is operating, storage management only requires you to administer disk groups as described in this section. As in single-instance Oracle databases, using ASM in RAC does not require I/O tuning.
When you use ASM, there is an ASM instance on each node of a RAC database. Each ASM instance has either an SPFILE or PFILE type parameter file. You may need to back up the parameter files and the TNS entries if you are using a nondefault Oracle Net Listener.
When you add disk groups in RAC, you only need to prepare the underlying operating system. ASM automatically rebalances the storage load after you add a disk group or when you delete a disk group.
When you add disk groups, you can also set a redundancy level and identify failure groups which are sets of disks that share a common resource. ASM uses failure groups to store redundant copies of your data to ensure that the loss of one failure group does not result in data loss. The default redundancy level is normal, which mirrors to two failure groups. You can also specify high redundancy for three-way mirroring. You can only use failure groups if you enable mirroring.
In RAC, each ASM instance manages all disk and failure groups for each node. In addition, each ASM instance coordinates disk and failure group information with other nodes in a RAC cluster database. As in single-instance Oracle databases, you can use the DBCA, SQL*Plus, and the Server Control Utility (SRVCTL) to administer disk groups for ASM in RAC as described in the following sections.
You can add or administer Automatic Storage Management (ASM) disk groups with Enterprise Manager. You can also monitor ASM disk group performance and control disk group availability at the instance level. For example, the additional ASM-related Enterprise Manager tasks that you can perform in RAC are:
When you add a disk group, the disk group definition includes a checkbox to indicate whether the disk group is automatically mounted to all of the cluster database instances.
When you examine the default Disk Group Performance page, you can see an instance-level performance details by clicking a performance characteristic such as Write Response Time or I/O Throughput.
When you mount and dismount ASM disk groups, you can use a checkbox to indicate which instances mount or dismount the ASM Disk Group.
See Also: Oracle Database Administrator's Guide and Oracle Database 2 Day DBA for detailed information about using Enterprise Manager |
You can use the Server Control Utility (SRVCTL) to add, remove, enable, and disable an ASM instance as described in the following procedures:
Use the following syntax to add configuration information about an existing ASM instance:
srvctl add asm -n node_name -i asm_instance_name -o oracle_home
Note: For all of the SRVCTL commands in this section for which the -i option is not required, if you do not specify an instance name, then the command applies to all the ASM instances on the node. |
Use the following syntax to remove an ASM instance:
srvctl remove asm -n node_name [-i asm_instance_name]
Use the following syntax to enable an ASM instance:
srvctl enable asm -n node_name [-i ] asm_instance_name
Use the following syntax to disable an ASM instance:
srvctl disable asm -n node_name [-i asm_instance_name]
You can also use SRVCTL to start, stop, and obtain the status of an ASM instance as in the following examples.
Use the following syntax to start an ASM instance:
srvctl start asm -n node_name [-i asm_instance_name] [-o start_options]
Use the following syntax to stop an ASM instance:
srvctl stop asm -n node_name [-i asm_instance_name] [-o stop_options]
Use the following syntax to show the configuration of an ASM instance:
srvctl config asm -n node_name
Use the following syntax to obtain the status of an ASM instance:
srvctl status asm -n node_name
All RAC instances must be able to access all datafiles. If a datafile needs to be recovered when the database is opened, then the first RAC instance to start performs the recovery and verifies access to the file. As other instances start up, they also verify their access to the datafiles. Similarly, when you add a tablespace or datafile or bring a tablespace or datafile online, all instances verify access to the file or files.
If you add a datafile onto a disk that other instances cannot access, then verification fails. Verification also fails if instances access different copies of the same datafile. If verification fails for any instance, then diagnose and fix the problem. Then execute the ALTER SYSTEM CHECK DATAFILES
statement on each instance in your RAC database to verify datafile access.
Each instance has its own online redo log groups which are called an instance's thread of online redo. Create these online redo log groups and establish group members as described in the Oracle Database Administrator's Guide.
Each instance must have at least two groups of online redo log files in its own thread. When the current group fills, an instance begins writing to the next log file group. If your database is in ARCHIVELOG mode, then each instance must save filled log files into its own archive log thread and update the control file with the status of its thread.
Oracle automatically manages undo segments within a specific undo tablespace that is assigned to an instance. Only the instance assigned to the undo tablespace can modify the contents of that tablespace. However, all instances can always read all undo blocks for consistent read purposes. Also, any instance can update any undo tablespace during transaction recovery, as long as that undo tablespace is not currently used by another instance for undo generation or transaction recovery. You assign undo tablespaces in your RAC database by specifying a different value for the UNDO_TABLESPACE
parameter for each instance in your SPFILE or individual PFILEs. You cannot simultaneously use automatic undo management and manual undo management in a RAC database. In other words, all instances of a RAC database must operate in the same undo mode.
See Also: Oracle Database Administrator's Guide for detailed information about creating and managing undo tablespaces |
You can dynamically switch undo tablespace assignments. Redirect the assignment for an instance by executing the ALTER
SYSTEM
SET
UNDO_TABLESPACE
statement. You might do this during planned maintenance to redirect an instance to use an idle undo tablespace. For example, assume that you have specified instances db1
and db2
to access undo tablespaces undotbs01
and undotbs02
respectively, and that you have an idle undo tablespace undotbs03
. To redirect instance db1
to use undo tablespace undotbs03
, execute the following SQL statement from instance db1
:
ALTER SYSTEM SET UNDO_TABLESPACE = undotbs3 SID=1;
In this example, the previously used undo tablespace undotbs01
remains assigned to instance db1
until the instance's last active transaction commits. User transactions proceed while Oracle performs the switching operation as follows:
The switching process does not wait for all user transactions to commit.
Oracle places the previous undo tablespace in a pending-offline state if there are active transactions in that tablespace. This means that the pending-offline tablespace may be unavailable for other instances until all transactions against that tablespace are committed.
Refer to the Oracle Database Administrator's Guide for more information about the ALTER
SYSTEM
SET
UNDO_TABLESPACE
statement and information about setting the undo retention period and dropping undo tablespaces.
This section describes how to administer the Oracle Cluster Registry (OCR). The OCR contains cluster and database configuration information for RAC and Cluster Ready Services (CRS). Some of this information includes the cluster node list, cluster database instance-to-node mapping information, and the CRS application resource profiles.
There are two methods of copying OCR content and to use the content for recovery. The first method uses automatically generated physical OCR file copies and the second method uses manually created logical OCR export files as described in the following sections:
Administering Oracle Cluster Registry Backup Files in Real Application Clusters
Restoring the Oracle Cluster Registry from Automatically Generated OCR Backups
Restoring the Oracle Cluster Registry on Windows-Based Systems
One CRS instance in a cluster automatically creates OCR backups every four hours. At any one time, the last three backup copies are always retained. The CRS instance also creates and retains an OCR backup for each full day and at the end of each week. You cannot customize the backup frequencies or the number of files that Oracle retains. However, because of the importance of the OCR information, Oracle recommends that you use the ocrconfig
tool to make copies of the automatically generated backup files at least once daily. The ocrconfig
tool commands are described later in this section.
Note: You must beroot user to run ocrconfig commands. |
Use the ocrconfig
tool as root
user with the showbackup
option to identify the backup files. Then copy each file to a location that is redundant to the location of the original OCR backup file. This ensures that there are at least two copies of each OCR backup file. Oracle also recommends that the OCR location reside on RAID arrays. If possible, use a backup location that is shared by all nodes in the cluster. The default target location of each OCR backup file is as follows where cluster name
is the name that you assigned to the cluster when you installed CRS:
CRS Home/cdata/cluster name
You can change this location using the following syntax: ocrconfig -backuploc directory name
If an application fails, then before attempting to resolve configuration-related problems retry the application. If the problem continues, then use one of the following platform-specific procedures to restore the OCR configuration.
Restoring the Oracle Cluster Registry on Windows-Based Systems
Note: You cannot restore your configuration from an automatically created OCR backup file using the -import option. You must instead use the -restore option. |
Stop the CRS software on all of the nodes in your cluster database by executing the init.crs stop
command on all of the nodes.
Identify the recent backups using the ocrconfig
-showbackup
command.
Execute the restore by applying an OCR backup file identified in Step 2 with the ocrconfig
-restore
file name
command.
Restart the CRS software on all of the nodes in your cluster by restarting each node.
Shut down all but one node in your cluster.
On the remaining node, disable the following OCR clients and stop them using the Service Control Panel: OracleClusterVolumeService
, OracleCSService
, OracleCRService
, and the OracleEVMService
.
Identify the recent backups using the ocrconfig
-showbackup
command.
Execute the restore by applying an OCR backup file identified in Step 3 with the ocrconfig
-restore
file name
command.
Start all of the services that were stopped in step 2. Restart all of the nodes and resume operations in cluster mode.
Stop the CRS software on all of the nodes in your cluster database by executing the init.crs stop
command on all of the nodes.
Edit the /var/opt/oracle/ocr.loc
file on all of the nodes and set the ocrconfig_loc
parameter to ocr_config_loc=
new_location
where new_location
is the new location of the OCR.
Restore the OCR from one of the automatic physical backups using the command ocrconfig -restore
.
Run the ocrcheck
command to verify the new OCR location.
Restart the CRS software on all of the nodes in your cluster by restarting each node.
Disable and stop the CRS software on all of the nodes by disabling and stopping the CRS services using the Services Control Panel.
Identify the recent OCR backup using the ocrconfig -showbackup
command.
Edit the HKEY_LOCAL_MACHINE\Software\Oracle\OCR\ocrconfig_loc
Registry key on every node to show the new OCR location.
Restore the OCR by applying the OCR backup file that you identified in Step 2 using the ocrconfig -restore
file name
command.
Start all of the services on all of the nodes that were stopped in step 1.
In addition to using the automatically created OCR backup files, you should also export the OCR contents before and after making significant configuration changes. Do this by using the ocrconfig
-export
command. This enables you to restore the OCR if your configuration changes cause errors. For example, if you have unresolvable configuration problems, or if you are unable to restart your clusterware, restore your configuration using one of the following platform-specific procedures:
Importing Oracle Cluster Registry Content on UNIX-Based Systems
Importing Oracle Cluster Registry Content on Windows-Based Systems
Note: You cannot import an exported OCR backup file using the -restore option. You must instead use the -import option. |
Shut down all the nodes in the cluster and restart one of them in single-user mode.
Import an OCR export file using the ocrconfig
-import
command from any node.
Start all the nodes in the cluster in multiuser mode.
Stop the following OCR clients using the Service Control Panel: OracleClusterVolumeService, OracleCMService, OracleEVMService, OracleCSService, and the OracleCRService.
Import an OCR export file using the ocrconfig
-import
command from one node.
Re-start all of the affected services on all nodes.
Execute OCR commands with administrative privileges on UNIX-based platforms or as a user with Administrator privileges on Windows-based systems. The OCR command syntax is as follows where options
is one of the verbs shown in the Option column of Table 3-1:
ocrconfig -option
Table 3-1 The ocrconfig Command Options
Option | Purpose |
---|---|
-export |
To export the contents of the OCR into a target file. |
-import |
To import OCR contents from a previously exported OCR file. |
-restore |
To restore the OCR from an automatically created OCR backup file. |
-backuploc |
To change OCR backup file location. For this entry, use a full path that is accessible by all nodes. |
-showbackup |
To display the location, timestamp, and the node name of origin for the last three automatically created backup files. |
-upgrade | To upgrade the OCR to a later version. |
-downgrade | To downgrade the OCR to an earlier version. |
-help |
To display help for the ocrconfig commands. |
For example, to export the OCR contents to a binary file, use the ocrconfig
command with the following syntax where file_name
is the file to which you want to export the OCR contents as follows:
ocrconfig -export file_name
Do not attempt to edit the export file. If OCR clients such as the clusterware are running, then Oracle reports an error.
The Oracle Hardware Assisted Resilient Data (HARD) initiative prevents data corruptions before they occur. With HARD enabled, the OCR writes HARD-compatible blocks. To determine whether the device used by the OCR supports HARD and enable it, review the Oracle HARD white paper at:
http://otn.oracle.com/deploy/availability/htdocs/HARD.html
When you install CRS, Oracle automatically executes the ocrconfig
-upgrade
command. To downgrade, follow the downgrade instructions for each component and also downgrade the OCR using the ocrconfig
-downgrade
command.
In Oracle9i, the OCR did not write HARD-compatible blocks. If the device used by OCR is enabled for HARD, then use the method described in the HARD white paper to disable HARD for the OCR device before downgrading your OCR. If you do not disable HARD, then the downgrade operation fails.
Use SRVCONFIG
to import and export SRVM cluster database configuration information to and from the OCR. For example, the following syntax exports the configuration information to a text file:
srvconfig -exp file_name.txt
The following syntax imports the configuration information from a text file into the OCR:
srvconfig -imp file_name.txt