Thursday, July 5, 2018

Multiplexing OCR and voting disk (11gR2)


Creating Diskgroup in ASM and Multiplexing OCR and voting disk (11gR2)
Hai, this article will illustrate you to how to create Disk groups in ASM. However it can be done in GUI mode using ASMCA, here it shows how to do in Command line.
List of Topics (Linux, Database, RAC, EBS)
For Oracle ASM to mirror files, specify the redundancy level as NORMAL REDUNDANCY (2-way mirroring by default for most file types) or HIGH REDUNDANCY (3-way mirroring for all files). Specify EXTERNAL REDUNDANCY if you do not want mirroring by Oracle ASM. For example, you might choose EXTERNAL REDUNDANCY to use storage array protection features.

The RAC database has 3 ASM disk group:
Redundancy Type
Disk Group Name
Mirrored Copies
Disks required
EXTERNAL
DATA
0
1
NORMAL
FRAN
2
3
HIGH
FRAH
3
5

Location of OCR & Voting Disk:
[root@rac1 ~]#
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3180
         Available space (kbytes) :     258940
         ID                       :  376377258
         Device/File Name         :      +DATA

[root@rac1 ~]#
[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   b1cede96f21a4fcbbf0d87634da5b01d (/dev/oracleasm/disks/DELLASM) [DATA]
Located 1 voting disk(s).
[root@rac1 ~]#

List of Disk groups, Redundancy Type and Compatability:
[root@rac1 ~]# 
[root@rac1 ~]# su - grid
[grid@rac1 ~]$
[grid@rac1 ~]$ . dell.env
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl status asm -a
ASM is running on rac2,rac1
ASM is enabled.
[grid@rac1 ~]$
[grid@rac1 ~]$ sqlplus  / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Wed Jul 4 13:07:38 2018

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL>
SQL>
set linesize 100;
col NAME for a10;
col FAILGROUP for a10;
col path for a35;
col COMPATIBILITY for a15;
col DATABASE_COMPATIBILITY for a25;
col HOST_NAME for a15;

SQL> 
SQL> select INSTANCE_NAME,STATUS,VERSION,HOST_NAME from v$instance;

INSTANCE_NAME    STATUS       VERSION           HOST_NAME
---------------- ------------ ----------------- ---------------
+ASM1            STARTED      11.2.0.4.0        rac1.dell.com

SQL>
SQL> SELECT GROUP_NUMBER,NAME,STATE,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME       STATE       TYPE   COMPATIBILITY   DATABASE_COMPATIBILITY
------------ ---------- ----------- ------ --------------- -------------------------
           1 DATA       MOUNTED     EXTERN 11.2.0.0.0      10.1.0.0.0

SQL>
SQL> SELECT NAME,FAILGROUP,HEADER_STATUS,PATH FROM V$ASM_DISK;

NAME       FAILGROUP  HEADER_STATU PATH
---------- ---------- ------------ -----------------------------------
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK05
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK04
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK03
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK02
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK01
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK00
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK04
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK03
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK02
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK01
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK00

NAME       FAILGROUP  HEADER_STATU PATH
---------- ---------- ------------ -----------------------------------
DATA_0000  DATA_0000  MEMBER       /dev/oracleasm/disks/DELLASM

12 rows selected.

SQL>
SQL>
PROVISIONED –it is the disk available for adding it to an Existing Diskgroup or, used to create a New Diskgroup.


Create DiskGroups (FRAN & FRAH) with Normal, High Redundancy Level:
SQL>
SQL> CREATE DISKGROUP FRAN NORMAL REDUNDANCY DISK '/dev/oracleasm/disks/FRAN_DISK00','/dev/oracleasm/disks/FRAN_DISK01','/dev/oracleasm/disks/FRAN_DISK02' ATTRIBUTE 'compatible.asm'='11.2.0.0.0';

Diskgroup created.

SQL> CREATE DISKGROUP FRAH HIGH REDUNDANCY DISK '/dev/oracleasm/disks/FRAH_DISK00','/dev/oracleasm/disks/FRAH_DISK01','/dev/oracleasm/disks/FRAH_DISK02','/dev/oracleasm/disks/FRAH_DISK03','/dev/oracleasm/disks/FRAH_DISK04' ATTRIBUTE 'compatible.asm'='11.2.0.0.0';

Diskgroup created.

SQL>
SQL> SELECT GROUP_NUMBER,NAME,STATE,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME       STATE       TYPE   COMPATIBILITY   DATABASE_COMPATIBILITY
------------ ---------- ----------- ------ --------------- -------------------------
           1 DATA       MOUNTED     EXTERN 11.2.0.0.0      10.1.0.0.0
           2 FRAN       MOUNTED     NORMAL 11.2.0.0.0      10.1.0.0.0
           3 FRAH       MOUNTED     HIGH   11.2.0.0.0      10.1.0.0.0

SQL>
SQL>
SQL> SELECT NAME,FAILGROUP,HEADER_STATUS,PATH FROM V$ASM_DISK;

NAME       FAILGROUP  HEADER_STATU PATH
---------- ---------- ------------ -----------------------------------
                      PROVISIONED  /dev/oracleasm/disks/FRAH_DISK05
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK04
                      PROVISIONED  /dev/oracleasm/disks/FRAN_DISK03
DATA_0000  DATA_0000  MEMBER       /dev/oracleasm/disks/DELLASM
FRAH_0004  FRAH_0004  MEMBER       /dev/oracleasm/disks/FRAH_DISK04
FRAH_0003  FRAH_0003  MEMBER       /dev/oracleasm/disks/FRAH_DISK03
FRAH_0002  FRAH_0002  MEMBER       /dev/oracleasm/disks/FRAH_DISK02
FRAH_0001  FRAH_0001  MEMBER       /dev/oracleasm/disks/FRAH_DISK01
FRAH_0000  FRAH_0000  MEMBER       /dev/oracleasm/disks/FRAH_DISK00
FRAN_0002  FRAN_0002  MEMBER       /dev/oracleasm/disks/FRAN_DISK02
FRAN_0001  FRAN_0001  MEMBER       /dev/oracleasm/disks/FRAN_DISK01

NAME       FAILGROUP  HEADER_STATU PATH
---------- ---------- ------------ -----------------------------------
FRAN_0000  FRAN_0000  MEMBER       /dev/oracleasm/disks/FRAN_DISK00

12 rows selected.


SQL>

Mount the newly added Diskgroups on Node-2:
[root@rac2 ~]# su - grid
[grid@rac2 ~]$ . dell.env
[grid@rac2 ~]$ sqlplus  / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Thu Jul 5 09:03:16 2018

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> 
SQL>
set linesize 100;
col NAME for a10;
col FAILGROUP for a10;
col path for a35;
col COMPATIBILITY for a15;
col DATABASE_COMPATIBILITY for a25;
col HOST_NAME for a15;
SQL>
SQL> SELECT INSTANCE_NAME,STATUS,VERSION,HOST_NAME FROM V$INSTANCE;

INSTANCE_NAME    STATUS       VERSION           HOST_NAME
---------------- ------------ ----------------- ---------------
+ASM2            STARTED      11.2.0.4.0        rac2.dell.com

SQL>
SQL> SELECT GROUP_NUMBER,NAME,STATE,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME       STATE       TYPE   COMPATIBILITY   DATABASE_COMPATIBILITY
------------ ---------- ----------- ------ --------------- -------------------------
           1 DATA       MOUNTED     EXTERN 11.2.0.0.0      10.1.0.0.0
           0 FRAH       DISMOUNTED         0.0.0.0.0       0.0.0.0.0
           0 FRAN       DISMOUNTED         0.0.0.0.0       0.0.0.0.0

SQL>
SQL>
SQL> alter diskgroup FRAN,FRAH mount;

Diskgroup altered.

SQL>
SQL>
SQL> SELECT GROUP_NUMBER,NAME,STATE,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME       STATE       TYPE   COMPATIBILITY   DATABASE_COMPATIBILITY
------------ ---------- ----------- ------ --------------- -------------------------
           1 DATA       MOUNTED     EXTERN 11.2.0.0.0      10.1.0.0.0
           3 FRAH       MOUNTED     HIGH   11.2.0.0.0      10.1.0.0.0
           2 FRAN       MOUNTED     NORMAL 11.2.0.0.0      10.1.0.0.0


SQL>

Add Disk groups (FRAN,FRAH) for OCR:
[root@rac1 ~]# 
[root@rac1 ~]# crs_stat -t

Name           Type           Target    State     Host
------------------------------------------------------------
ora.DATA.dg    ora....up.type ONLINE    ONLINE    rac1
ora.FRAH.dg    ora....up.type ONLINE    ONLINE    rac1
ora.FRAN.dg    ora....up.type ONLINE    ONLINE    rac1

[root@rac1 ~]#
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3172
         Available space (kbytes) :     258948
         ID                       :  376377258
         Device/File Name         :      +DATA
                                    Device/File integrity check succeeded

[root@rac1 ~]# ocrconfig -add +FRAN
[root@rac1 ~]#
[root@rac1 ~]# ocrconfig -add +FRAH
[root@rac1 ~]#
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3172
         Available space (kbytes) :     258948
         ID                       :  376377258
         Device/File Name         :      +DATA
                                    Device/File integrity check succeeded
         Device/File Name         :      +FRAN
                                    Device/File integrity check succeeded
         Device/File Name         :      +FRAH
                                    Device/File integrity check succeeded


[root@rac1 ~]# 
[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   b1cede96f21a4fcbbf0d87634da5b01d (/dev/oracleasm/disks/DELLASM) [DATA]
Located 1 voting disk(s).
[root@rac1 ~]#

Multiplexing of Voting Disk for NORMAL REDUNDANCY (FRAN):
[root@rac1 ~]#
[root@rac1 ~]# crsctl replace votedisk +FRAN
Successful addition of voting disk 77b5ddfb00c04f6abfb03c94ee910534.
Successful addition of voting disk 9f1d449a843c4fd5bfeacdf4ce33cae1.
Successful addition of voting disk 60445a961e204fabbf303e9ff416de38.
Successful deletion of voting disk b1cede96f21a4fcbbf0d87634da5b01d.
Successfully replaced voting disk group with +FRAN.
CRS-4266: Voting file(s) successfully replaced
[root@rac1 ~]#
[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   77b5ddfb00c04f6abfb03c94ee910534 (/dev/oracleasm/disks/FRAN_DISK00) [FRAN]
 2. ONLINE   9f1d449a843c4fd5bfeacdf4ce33cae1 (/dev/oracleasm/disks/FRAN_DISK01) [FRAN]
 3. ONLINE   60445a961e204fabbf303e9ff416de38 (/dev/oracleasm/disks/FRAN_DISK02) [FRAN]
Located 3 voting disk(s).
[root@rac1 ~]#

Multiplexing of Voting Disk for HIGH REDUNDANCY (FRAH):
[root@rac1 ~]#
[root@rac1 ~]# crsctl replace votedisk +FRAH
Successful addition of voting disk 723736518b314f9abf5a5c084a122717.
Successful addition of voting disk 4f1eceb889f34faabf3df2ede2952945.
Successful addition of voting disk 08eb418213304f60bf716b0bb1a5f79c.
Successful addition of voting disk 7cc6491a7e294f90bf3a2b039c9b0463.
Successful addition of voting disk 7bffba1ee6434fdabf12092dbb551312.
Successful deletion of voting disk 77b5ddfb00c04f6abfb03c94ee910534.
Successful deletion of voting disk 9f1d449a843c4fd5bfeacdf4ce33cae1.
Successful deletion of voting disk 60445a961e204fabbf303e9ff416de38.
Successfully replaced voting disk group with +FRAH.
CRS-4266: Voting file(s) successfully replaced
[root@rac1 ~]#
[root@rac1 ~]#
[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   723736518b314f9abf5a5c084a122717 (/dev/oracleasm/disks/FRAH_DISK00) [FRAH]
 2. ONLINE   4f1eceb889f34faabf3df2ede2952945 (/dev/oracleasm/disks/FRAH_DISK01) [FRAH]
 3. ONLINE   08eb418213304f60bf716b0bb1a5f79c (/dev/oracleasm/disks/FRAH_DISK02) [FRAH]
 4. ONLINE   7cc6491a7e294f90bf3a2b039c9b0463 (/dev/oracleasm/disks/FRAH_DISK03) [FRAH]
 5. ONLINE   7bffba1ee6434fdabf12092dbb551312 (/dev/oracleasm/disks/FRAH_DISK04) [FRAH]
Located 5 voting disk(s).
[root@rac1 ~]#


--------------------
Normal Redundancy:
Note that we cannot use the ‘crsctl add css votedisk’ to add a vote disk on ASM disk group/ACFS file system. We can use crsctl replace votedisk’ to move voting disk to a ASM disk group with normal redundancy. The new ASM Diskgroup has minimum of 3 fail groups (total of 3 disks). This configuration will provide 3 Voting Disks (1 on each fail group) and a single OCR which takes on the redundancy of the disk group. Hence, use separate disk group with normal redundancy with one disk quorum is the method to multiplexing voting disks.

--------------------
High Redundancy:
Note that we cannot use the ‘crsctl add css votedisk’ to add a vote disk on ASM disk group/ACFS file system. We can use crsctl replace votedisk’ to move voting disk to a ASM disk group with High redundancy. The new ASM Diskgroup has minimum of 5 fail groups (total of 5 disks). This configuration will provide 5 Voting Disks (1 on each fail group) and a single OCR which takes on the redundancy of the disk group. Hence, use separate disk group with High redundancy with one disk quorum is the method to multiplexing voting disks.

================================================================================================================
Supportive Information
Attach new 100 GB Hard disk to Openfiler and configure it in GUI
Format to Logical Disk and provide each logical disk of same size:

----------------------------- 
Your Disk is Not Available. So, Discover your IQN
[root@rac1 ~]# 
[root@rac1 ~]#  iscsiadm -m discovery -t st -p 192.168.1.40
192.168.1.40:3260,1 iqn.2006-01.com.openfiler:tsn.02d3a5650e16
192.168.1.40:3260,1 iqn.2006-01.com.openfiler:tsn.5cbf23534a21
[root@rac1 ~]#
[root@rac1 ~]# service iscsi restart
[root@rac1 ~]#
[root@rac1 ~]# init 6
[root@rac1 ~]#
----------------------------- 
[root@rac2 ~]#  iscsiadm -m discovery -t st -p 192.168.1.40
192.168.1.40:3260,1 iqn.2006-01.com.openfiler:tsn.02d3a5650e16
192.168.1.40:3260,1 iqn.2006-01.com.openfiler:tsn.5cbf23534a21
[root@rac2 ~]#
[root@rac2 ~]# service iscsi restart
[root@rac2 ~]#
[root@rac2 ~]# init 6
[root@rac2 ~]# 
----------------------------- 

 root@rac1 ~]# 
root@rac1 ~]# fdisk –l
root@rac1 ~]# fdisk /dev/sdd

p
n
e
1


p
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
n
l

+2G
p
p
w
pwd


[root@rac1 ~]# partx -a /dev/sdd
[root@rac1 ~]# partx -a /dev/sdd

[root@rac2 ~]# partx -a /dev/sdd
[root@rac2 ~]# partx -a /dev/sdd


----------------------------- 
Creation of ASM DISKS
oracleasm createdisk FRAN_DISK00 /dev/sdd5
oracleasm createdisk FRAN_DISK01 /dev/sdd6
oracleasm createdisk FRAN_DISK02 /dev/sdd7
oracleasm createdisk FRAN_DISK03 /dev/sdd8
oracleasm createdisk FRAN_DISK04 /dev/sdd9
oracleasm createdisk FRAH_DISK00 /dev/sdd10
oracleasm createdisk FRAH_DISK01 /dev/sdd11
oracleasm createdisk FRAH_DISK02 /dev/sdd12
oracleasm createdisk FRAH_DISK03 /dev/sdd13
oracleasm createdisk FRAH_DISK04 /dev/sdd14
oracleasm createdisk FRAH_DISK05 /dev/sdd15
pwd
ll /dev/oracleasm/disks
[root@rac1 ~]#


Scan disk on RAC-2:
[root@rac2 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASM_DISK01"
Instantiating disk "ASM_DISK02"
Instantiating disk "ASM_DISK03"
Instantiating disk "ASM_DISK04"
Instantiating disk "ASM_DISK05"
Instantiating disk "ASM_DISK06"
Instantiating disk "ASM_DISK07"
Instantiating disk "ASM_DISK08"
Instantiating disk "ASM_DISK09"
Instantiating disk "ASM_DISK010"
Instantiating disk "ASM_DISK011"
[root@rac2 ~]#
[root@rac2 ~]#

 ----------------------------- 
For changing compatibility of Diskgroup:
SQL>
SQL> alter diskgroup FRA set attribute 'compatible.asm'='11.2.0.0.0';

Diskgroup altered.

SQL>
================================================================================================================


Posted by at
Labels: ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)