测试环境,安装EM12C AGENT后,登陆ASM时发现问题:
1.故障现象
[grid@AAAA grid]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Wed May 11 14:54:39 2016
Copyright (c) 1982, 2013, Oracle. All rights reserved.
ERROR:
ORA-12547: TNS:lost contact
[root@AAAA ~]# crsctl stat res -t
——————————————————————————–
NAME TARGET STATE SERVER STATE_DETAILS
——————————————————————————–
Local Resources
——————————————————————————–
ora.ARCH.dg
ONLINE OFFLINE AAAA
ora.DATA.dg
ONLINE OFFLINE AAAA
ora.FRA.dg
ONLINE OFFLINE AAAA
ora.LISTENER.lsnr
ONLINE OFFLINE AAAA STARTING
ora.asm
ONLINE OFFLINE AAAA
ora.ons
OFFLINE OFFLINE AAAA
——————————————————————————–
Cluster Resources
——————————————————————————–
ora.cssd
1 ONLINE OFFLINE
ora.AAAA.db
1 ONLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE AAAA
———————————-
2.排查过程
登陆ASM报错,首先排查ASM的日志:
Fri Apr 15 22:01:04 2016
NOTE: Advanced to new COD format for group ARCH
Wed May 11 14:11:51 2016
Errors in file /u01/app/grid/diag/asm/+asm/+ASM/trace/+ASM_ora_8967.trc (incident=5761):
ORA-00600: internal error code, arguments: [spstp: ORACLE_HOME uid does not match euid], [500], [507], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/grid/diag/asm/+asm/+ASM/incident/incdir_5761/+ASM_ora_8967_i5761.trc
—-ORACLE_HOME的UID不相符,从下面检查用户权限可以看到是将UID 507、500对象的是GRID/ORACLE用户。
[grid@AAAA grid]$ id grid
uid=507(grid) gid=501(oinstall) groups=501(oinstall),502(dba),504(asmadmin),505(asmdba),506(asmoper)
[grid@AAAA grid]$ id oracle
uid=500(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper),504(asmadmin),505(asmdba)
如下验证可以发现GRID用户的安装权限都被误操作修改为OWNER为oracle。
[grid@AAAA +ASM]$ cd /u01/app/
[grid@AAAA app]$ ls -al
total 28
drwxrwxr-x. 7 oracle oinstall 4096 May 11 13:58 .
drwxrwxr-x. 3 oracle oinstall 4096 Apr 12 11:41 ..
drwxrwxr-x. 67 oracle oinstall 4096 Apr 15 11:49 11.2.0
drwxr-xr-x. 6 oracle oinstall 4096 May 11 14:03 agent12c
drwxrwxr-x. 9 oracle oinstall 4096 Apr 15 12:39 grid
drwxrwxr-x. 7 oracle oinstall 4096 Apr 15 13:38 oracle
drwxrwx—. 6 oracle oinstall 4096 May 11 14:24 oraInventory
临时解决方法:是重新使用命令:chown -R /u01/app/11.2.0 等方式修改目录权限后恢复正常,不过GRID软件中有一些文件的权限是root,此时暂时不管了。
–从运行稳定角度来看,建议是重新安装了。
扩展一下:
ORA-12547: TNS:lost contact在RAC中常见的还有oracle程序的权限不对,
ORA-12547 Errors
The error ORA-12547 indicates that the communication channel has been broken. It’s most often thrown because the other end of the process went away unexpectedly.
Note 1307075.1 Oracle Database Fails to Start with Error ORA-12547
Note 381566.1 connect / as sysdba Fails with Ora-12547 And Tns-12514
Note 744512.1 Ora-12547: Tns:Lost Contact Creating Database After Clean Installation