The biggest challenge for an Oracle DBA is migration of Oracle RAC database across storages.

In that consider migrating a RAC database on Linux (RHEL/OEL) platform with many of the critical components present in RAW/block devices, is even a bigger challenge. For example in this blog we will discuss how to migrate Oracle RAC database running on an EMC CLARiiON to Dell Compellent Storage Array. This article can be leveraged for any Oracle Database storage migration.

Let’s have a brief introduction to Oracle RAC components and their significance.

Oracle RAC Database Cluster includes the below critical components.

• Votedisk : The voting disk is a file that manages information about node membership.
• OCR : It is a file that manages cluster and Oracle RAC database configuration information.
• Datafiles : Datafiles are physical files of the operating system that store the data of all logical structures in the database.

In this blog we will discuss about migrating the above mentioned critical components of 10g R2, 11gR1 and 11gR2 Oracle RAC from an EMC CLARiiON to Dell Compellent Storage Array

Storage Migration of Oracle RAC database involves the following steps.

• Migrating Oracle Cluster Registry (OCR) to the new storage device
• Migrating Cluster heartbeat (Voting disk) to the new storage device
• Migrating the Database to the new storage.

General Pre-requisites:

Before migrating to the Compellent storage, the below pre-requisites are mandatory.

• Make sure the corresponding hardware infrastructure (Eg: Host HBA, Storage Switches) is in place before migration.
• Make sure that the Compellent storage has enough space to fit the legacy storage data.
• Create the number of volumes and their sizes to be equal or greater on the Compellent storage.
• Make sure that the new volumes are discovered, multipath is configured in the target nodes.
• New Raw devices are configured in case of 10gR2 and 11gR1, ASM stamping is done for the disks wherever appropriate.
• Appropriate permissions set for the newly discovered Compellent Volumes.

Note:

• This document doesn’t refer to any OS migration or Oracle Database migration, rather talks only about Data Migration across Storages.
• This document doesn’t apply for Oracle Database Cluster configured on OCFS File System.

Backup procedure for OCR, Voting Disk and Data :

Before migrating the data to a new Storage, it is always recommended to backup the critical components of Oracle RAC using the below mentioned steps.

 OCR Backup :

 Make sure there is a recent copy of the OCR file before migration, below command will help you to check whether backups are in place

 #ocrconfig –showbackup

Host1     2012/01/31 08:54:02     /u01/app/oracle/11.1.0/crs/cdata/11g_cluster/backup00.ocr
Host1     2012/01/31 04:54:02     /u01/app/oracle/11.1.0/crs/cdata/11g_cluster /backup01.ocr
Host1     2012/01/31 00:54:02     /u01/app/oracle/11.1.0/crs/cdata/11g_cluster /backup02.ocr
Host1     2012/01/30 00:54:01     /u01/app/oracle/11.1.0/crs/cdata/11g_cluster /day.ocr
 

If there is no recent backup copy of the OCR file, Use the following command to generate a backup of the online OCR file:

For Oracle 10R2 and 11gR1

#ocrconfig export <OCR export_filename> -s online

 For Oracle 11g R2

#ocrconfig -manualbackup

 Voting Disk Backup :

Run the below command to find the configured voting devices for backup.

#crsctl query css votedisk

1. 0    /dev/raw/raw6
2. 0    /dev/raw/raw7
3. 0    /dev/raw/raw8

Take a backup of any one of the above configured voting disk using the below command

For Oracle 10R2 and 11gR1

$ dd if=voting_disk_name of=backup_file_name

Eg:  $ dd if=/dev/raw/raw6 of=/root/votebackup.bkp

In Oracle Clusterware 11g release 2 (11.2), you no longer have to back up the voting disk separately. The voting disk data is automatically backed up in OCR as part of any configuration change.

 Database Backup :

Perform a full database backup including control files, redologs, database files, init files and password files using any of the Oracle recommended backup method.

Migration procedure for OCR, Voting Disk and Data :

After performing all the mandatory checks and necessary backups, it’s time to migrate the same to the new Storage Array.

1) Migration procedure for OCR to the Compellent storage Device.

Note:

• An ocrmirror must be in place before trying to replace the OCR device in all the Database versions.
• You must be logged in as the root user, because root owns the OCR files.
• If an OCR device is replaced with a device of a different size, the size of the new device will not be reflected until the Clusterware is restarted.

For Oracle 10gR2 and 11gR1

• In the below example, We migrate the OCR from EMC disks /dev/raw/raw1, /dev/raw/raw2 to newly discovered Compellent disks /dev/raw/raw10 and /dev/raw/raw11

  #<CRS_HOME>/bin/ocrcheck

  Status of Oracle Cluster Registry is as follows:
  Version                  :          2
  Total space (kbytes)     :    5248800
  Used space (kbytes)      :       1168
  Available space (kbytes) :    5247632
  ID                       :  220474074
  Device/File Name         : /dev/raw/raw1
  Device/File integrity check succeeded
  Device/File Name         : /dev/raw/raw2
  Device/File integrity check succeeded
  Cluster registry integrity check succeeded

• Execute the below commands for migrating OCR to the Compellent Storage,  /dev/raw/raw11and /dev/raw/raw10 are the new Compellent OCR disks.

 # <CRS_HOME>/bin/ocrconfig -replace ocrmirror /dev/raw/raw11
 # <CRS_HOME>/bin/ocrconfig -replace ocr /dev/raw/raw10

• Status of Oracle Cluster Registry is as follows:

# <CRS_HOME>/bin/ocrcheck  

Version                  :          2
Total space (kbytes)     :    6286092
Used space (kbytes)      :       1168
Available space (kbytes) :    6284924
ID                       :  220474074
Device/File Name         : /dev/raw/raw10
Device/File integrity check succeeded
Device/File Name         : /dev/raw/raw11
Device/File integrity check succeeded
Cluster registry integrity check succeeded

 For Oracle 11g R2:

• Execute the below commands to replace the OCR

# ocrconfig -replace <current filename> -replacement <new filename>

Eg: # ocrconfig -replace +QUORUM -replacement +OCRVOTE

Note: Make sure the OCR location is pointed to the newly migrated disks in /etc/oracle/ocr.loc

2) Migrating the Voting Disk to the Compellent storage Device.

Note: You must be logged in as the root user, because root owns the OCR files.

For Oracle 10gR2 and 11gR1

• To verify the configured voting devices run the below command

 # crsctl query css votedisk

1. 0    /dev/raw/raw6
2. 0    /dev/raw/raw7
3. 0    /dev/raw/raw8

located 3 votedisk(s).

•  For Oracle 10g R2, Shutdown Oracle Clusterware on all nodes  as root using the below command .

 #<CRS_HOME>/bin$ crsctl stop crs

 Note: This step is only required for  Oracle 10g R2 CRS. For Oracle 11gR1 this is an online operation and no cluster outage is required.

•  For Voting disk migration execute the below commands on any one of the node:

 Note: It is not allowed to delete the last defined voting disk without adding a new disk.

#<CRS_HOME>/bin$  crsctl delete css votedisk /dev/raw/raw6 -force
#<CRS_HOME>/bin$  crsctl add css votedisk  /dev/raw/raw12  -force
#<CRS_HOME>/bin$  crsctl delete css votedisk /dev/raw/raw7 –force
#<CRS_HOME>/bin$ crsctl add css votedisk /dev/raw/raw13 -force
#<CRS_HOME>/bin$  crsctl delete css votedisk /dev/raw/raw8 -force
#<CRS_HOME>/bin$ crsctl add css votedisk /dev/raw/raw14 –force 

• Verify with crsctl query css votedisk:

 #<CRS_HOME>/bin$  crsctl query css votedisk

1. 0    /dev/raw/raw12
2. 0    /dev/raw/raw13
3. 0    /dev/raw/raw14

located 3 votedisk(s).

For Oracle 11g R2

• To verify the configured voting devices run the below command

# crsctl query css votedisk

#  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. ONLINE   466a8bc2ad994f19bfb4086597a9bd7f (ORCL:OCR3) [OCR]
2. ONLINE   7623300042264f63bf3071c1abf0755a (ORCL:OCR2) [OCR]
3. ONLINE   36409c1444614fecbfb5083e2dd663b3 (ORCL:OCR1) [OCR]

Located 3 voting disk(s).

•  Perform the below steps for Voting disk migration

# crsctl replace votedisk +OCRVOTE

CRS-4256: Updating the profile
Successful addition of voting disk 3d34623f09b64f9dbfa44fabf455513e.
Successful addition of voting disk 7043c38000a24f1abf36473ca7e9cd9e.
Successful addition of voting disk 18de241007df4f9cbf3fbb4193f0ecb4.
Successful deletion of voting disk a32c9b158e644fabbfdcc239c76f22a0.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced

3) Migrating the Data to the Compellent storage Devices.

Before migrating the data to the newly discovered compellent device, stamp the disks using oracleasm library. Add the stamped disks to the existing diskgroup using the below commands.

• Add new disks to ASM diskgroups.

 ALTER DISKGROUP DATA ADD DISK ‘ORCL:NEWDATA1’ REBALANCE POWER 11;

•  Check rebalance status from Enterprise Manager or v$ASM_OPERATION.
•  When rebalance completes, drop the old disks.

 ALTER DISKGROUP DATA DROP DISK OLDDISK1 REBALANCE POWER 11;

4) Status Check after Migration

• Verify the Oracle Grid Infrastructure installation by typing the following command:

<CRS_HOME>/bin/olsnodes -n -v

This command lists the host names of all nodes in the cluster.

•  Type the following command to check the status of all the grid services:

 <CRS_HOME>bin/crsctl stat res -t

Verify that all Oracle grid services that are running are displayed.

Ensure that all the services are online.

•  Verify the status of the database

<CRS_HOME>/bin/srvctl config database –d <databasename>

The above mentioned steps will help you in order to achieve a safe and orderly transition of Oracle RAC Components to the new Dell Compellent Storage with minimum downtime and efforts.