Email: service@parnassusdata.com 7 x 24 online support!
TROUBLESHOOTING - Oracle ASM disk not found/visible/discovered issues
APPLIES TO:
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Cloud Machine - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Oracle Database Cloud Exadata Service - Version N/A and later
Information in this document applies to any platform.
PURPOSE
This note will assist in troubleshooting disk not found / visible / discovered issues with ASM
Typically this means that the disk in question is not found in the v$asm_disk view
Common errors indicating that a disk is missing/not found are :
ORA-15063: ASM discovered an insufficient number of disks for diskgroup s%
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "%" is missing
TROUBLESHOOTING STEPS
If an existing diskgroup is suddenly missing a disk ...
Determine what disk is missing
If on non-RAC ... or ALL RAC nodes exhibit these errors
a) Locate and open the ASM alert log (this may require multiple logs for RAC)
b) Locate the last successfull mount of the diskgroup (this will show a list of disks)
c) Locate each successful ALTER DISKGROUP ... ADD DISK since that time
d) Combine the lists of disks found in b) and c) above
e) Compare this list against those shown in V$ASM_DISK and determine the missing disk(s)
If on RAC and at least one node can mount the diskgroup
Compare the V$ASM_DISK entries on the node that can mount to those that cannot this will show which disk(s) is missing
The following is a set of steps that will assist in resolving disk not found issues
1) ASM_DISKSTRING is not set correctly
Examine the ASM_DISKSTRING setting in the parameter file or via SHOW PARAMETER
If ASM_DISKSTRING is NOT SET ... then the following default is used
Default ASM_DISKSTRING per OS
Operating System Default Search String
=======================================
Solaris (32/64 bit) /dev/rdsk/*
Windows NT/XP \\.\orcldisk*
Linux (32/64 bit) /dev/raw/*
LINUX (ASMLIB) ORCL:*
LINUX (ASMLIB) /dev/oracleasm/disks/* ( as a workaround )
HPUX /dev/rdsk/*
HP-UX(Tru 64) /dev/rdisk/*
AIX /dev/rhdisk*
IF ASM_DISKSTRING is SET ... then verify that the setting includes the disks that are needed to be seen by ASM
2) Operating system drive ownership
Make sure that the disk is owned by the OS user who installed the ASM Oracle Home ... and that the disk is mounted correctly (with the correct owner)
3) Operating system drive permissions
Make sure that the permissions are set correctly at the disk level ... 660 is normal ... but if there are problems use 777 as a test
4) RAC is being used
If RAC is in use ... then ALL disks need to be visible on all nodes where ASM is / will be running ... before an attempt is made to add the disks to a diskgroup
5) Use OS utilities to determine which disk cannot be found
TRUSSing or STRACEing the RBAL process while selecting * from v$asm_disk can often show errors in the path of the command
EXAMPLE:
=========
SESSION #1
strace -f -o /tmp/rbal.trc -p <OS pid of RBAL process>
<OR>
truss -ef -o /tmp/rbal.out -p <OS pid for RBAL process>
SESSION #2
select * from v$asm_disk
SESSION #3
<CTRL-C>
Examine the rbal.out for errors: For example,
1147090: 1871929: chdir("dev/") = 0
1147090: 1871929: statx("rhdisk8, ", 0x0FFFFFFFFFFFAA80, 176, 010) Err#2 ENOENT
<< This says that rhdisk8 cannot be found >>
NOTE ... If a crash occured during an Add or Drop of a disk ... then the disk(s) in question may still be part of the diskgroup ... so all of the steps above need to take into consideration this (these) disks
PORT SPECIFIC ISSUES
1) HEWLETT PACKARD (HP)
If this is problem is occuring on HP (RISC or Itanium) ... and all of the above are not helping ... and the customer is using HP Logical Volume Manager (LVM) then
Note.433770.1 Cannot Discover Disks in ASM After Upgrade on 10.2.0.3 on HP-UX Itanium
Note.434500.1 When Starting A Database With The 10.2.0.3 Executables, Error "ORA-15059 invalid device type for ASM disk"
2) IBM AIX
Note.353761.1 Assigning a Physical Volume ID (PVID) To An Existing ASM Disk Corrupts the ASM Disk Header
NOTE:1174604.1 - ASM Is Not Detecting EMC PowerPath Raw Devices Or Regular Raw Devices On AIX
3) SOLARIS
Note.368840.1 ASM does not discover disk on Solaris platform
4) LINUX
Note.457369.1 RASM is Unable to Detect ASMLIB Disks/Devices.