Email: service@parnassusdata.com 7 x 24 online support!
ASMdisk Status - Candidate disk after reboot [ recover ASM header files ]
If you cannot recover the data by yourself, ask Parnassusdata, the professional ORACLE database recovery team for help.
Parnassusdata Software Database Recovery Team
Service Hotline: +86 13764045638 E-mail: service@parnassusdata.com
Customer has migrated oracle databases running on old SAN to new SAN using ASM rebalance operation. Customer is using External redundancy. After completing rebalance operation, customer rebooted all server and removed old SAN device entry on weekend. Customer is unable to bring databases online on One of the 4 servers. Customer is getting following error :
ORA-15032: not all alterations performed
ORA-15017: diskgroup “DATA” cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “DATA”
I have attached asm-kfed result for your reference. Is it possible to recover ASM header files without so customer doesn’t need to backup/restore 5 TB database ?
Total System Global Area 284008448 bytes Fixed Size 2158616 bytes Variable Size 256684008 bytes ASM Cache 25165824 bytes ORA-15032: not all alterations performed ORA-15017: diskgroup "DATA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA" SQL> select group_number,disk_number,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,PATH from V$asm_disk; GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME PATH ------------ ----------- ------- ------------ ------- -------- ------------------------------ ---------------------------------------- 0 0 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d26s6 0 23 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d22s6 0 2 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM01 0 3 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM12 0 4 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d27s6 0 5 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d25s6 0 6 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM05 0 7 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM02 0 8 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM08 0 9 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d24s6 0 10 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM11 0 11 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d30s6 0 12 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM07 0 13 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d31s6 0 14 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d21s6 0 15 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM10 0 16 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d23s6 0 17 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM09 0 18 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM03 0 19 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d29s6 0 20 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/c0d28s6 0 21 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM04 0 22 IGNORED MEMBER ONLINE NORMAL /dev/rdsk/c0d32s6 0 1 CLOSED CANDIDATE ONLINE NORMAL /dev/rdsk/san03dp_dbs05dp_ASM06 dev/rdsk/san03dp_dbs05dp_ASM03 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] /dev/rdsk/san03dp_dbs05dp_ASM04 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] /dev/rdsk/san03dp_dbs05dp_ASM05 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] /dev/rdsk/san03dp_dbs05dp_ASM06 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] /dev/rdsk/san03dp_dbs05dp_ASM07 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] /dev/rdsk/san03dp_dbs05dp_ASM08 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 10037FE00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
ASM disks – ASM03/04/05/06/07/08 – showing the status as “CANDIDATE” is a bit worry. But if the devices were not part of DATA diskgroup before, these devices are not the major cause of ORA-15063.
Please check ASM alert.log whether these 5 devices were belong to DATA.
I’m more concerned about the following 4 devices as they show the status as “IGNORED” which indicates there are other devices showing the same disk information given asm_diskstring parameter.
– ASM01/02/0910/11
Chances are that the following devices below show the same disk information as ASM01/02/0910/11 and there is good chance that these different path point to the same physical devices.
~~
/dev/rdsk/c0d22s6
/dev/rdsk/c0d30s6
/dev/rdsk/c0d31s6
/dev/rdsk/c0d21s6
/dev/rdsk/c0d29s6
/dev/rdsk/c0d32s6
~~
Please check all disk header which disks show the duplicate ASM disk information using the following perl script in the note below
And if duplicate paths point to the same physical device, the additional device path should be disabled using “chmod 000 <device_path>”
– KFED.PL for diagnosing – ORA-15063 ORA-15042 ORA-15020 (Doc ID 1346190.1)