SuperCluster – 如何从由于HAIP导致的失败的11.2.0.3 到11.2.0.4 的升级进行恢复

如果自己搞不定可以找诗檀软件专业ORACLE数据库修复团队成员帮您恢复!

诗檀软件专业数据库修复团队

服务热线 ： 13764045638 QQ号:47079569 邮箱：service@parnassusdata.com

症状

Host的RAC 警告日志显示类似如下：
[client(14264)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/ocrcheck_14264.log.
2015-04-05 00:33:17.971:
[client(18393)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:33:29.135:
[client(19012)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:33:39.677:
[client(19799)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:33:50.227:
[client(20622)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:34:07.815:
[client(21927)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:34:21.661:
[client(23310)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0.4/grid/log/hostname/client/crsctl_grid.log.
2015-04-05 00:35:28.999:
[ctssd(1312)]CRS-2409:The clock on host hostname is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
2015-04-05 00:35:42.338:
[/u01/app/11.2.0.4/grid/bin/orarootagent.bin(28629)]CRS-5818:Aborted command ‘start’ for resource ‘ora.cluster_interconnect.haip’. Details at (:CRSAGF00113:) {0:0:242} in /u01/app/11.2.0.4/grid/log/hostname/agent/ohasd/orarootagent_root//orarootagent_root.log.
2015-04-05 00:35:46.343:
[ohasd(27426)]CRS-2757:Command ‘Start’ timed out waiting for response from the resource ‘ora.cluster_interconnect.haip’. Details at (:CRSPE00111:) {0:0:242} in /u01/app/11.2.0.4/grid/log/hostname/ohasd/ohasd.log.
2015-04-05 00:35:46.674:
[/u01/app/11.2.0.4/grid/bin/oraagent.bin(28409)]CRS-5011:Check of resource “+ASM” failed: details at “(:CLSN00006:)” in “/u01/app/11.2.0.4/grid/log/hostname/agent/ohasd/oraagent_grid//oraagent_grid.log”

crsctl status res -t -init shows
ora.cluster_interconnect.haip interface not starting and ASM not started.
crsctl status res ora.asm -p
shows a startup an shutdown dependency on ora.cluster_interconnect.haip.

更改

在一个LDom 或zone host执行一个11.2.0.3到 11.2.0.4的升级，在rootupgrade.sh阶段发生故障。

原因
该问题的原因是HAIP 不在Solaris IPMP 界面上被支持。我们在SuperCluster上的集群互联使用ipmp。
有一个存档的bug /强化在haip启动时获取在IPMP <Defect 17837626> SuperCluster / Solaris exadata rootupgrade 到11.2.0.4 失败的HAIP支持。为避免这个情况，你需要在升级进程之前，编辑以下文件，一旦你运行它而???未创建，你需要遵守以下指定的恢复进程。once you have run it without the edit nd this haip resource is created you have to follow the recovery process specified below.

cd <GRID_HOME>/crs/install# vi s_crsconfig_lib.pm

Look for the function s_is_sun_ipmp and make the following change
at the end of that function

/is_sun_ipmp

# made it all the way out without finding any IPMP private
#return FALSE;
return TRUE;
“s_crsconfig_lib.pm” 2811 lines, 83736 characters written

解决方案
1) 在11.2.0.4 home env下更改ora.asm参数

crsctl modify resource ora.asm -attr “START_DEPENDENCIES=’hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.drivers.acfs)'” -f -init

crsctl modify resource ora.asm -attr “STOP_DEPENDENCIES=hard(intermediate:ora.cssd)” -f -init
2) 在11.2.0.4环境中删除错误的 ora.cluster_interconnect.haip 资源格式crs。
crsctl delete resource ora.cluster_interconnect.haip -f -init
3) 将 GI 从11.2.0.4 降级到11.2.0.3，形成11.2.0.4 环境，使用force flag

$NEW_HOME/crs/install/rootcrs.pl -downgrade -force -oldcrshome $OLD_HOME -version $OLD_VERSION
4) 重复升级进程形成一开始的，但仅在你验证了以上更改的s_crsconfig_lib.pmis之后。Repeat the upgrade process form the begining but only after you verify the s_crsconfig_lib.pmis modified as above.