Issue in starting opmn process due to database error

ISSUE 1:

Unable to start opmn (Oid and oidmon) process
C:\Oracle\Middleware\asinst_1\bin
opmnctl.bat startall

ERROR OBSERVED:

Logs: C:\Oracle\Middleware\asinst_1\diagnostics\logs\OID\oid1

[OID] [NOTIFICATION:16] [] [OIDMON] [host: HOSTNAME] [pid: 10056] [tid: 0] Guardian: Connecting to database, connect string is OIDDB

[OID] [NOTIFICATION:16] [] [OIDMON] [host: HOSTNAME] [pid: 10056] [tid: 0] Guardian: [gsdsiConnect] ORA-12541, ORA-12541: TNS:no listener

[[OID] [NOTIFICATION:16] [] [OIDMON] [host: HOSTNAME] [pid: 10056] [tid: 0] Guardian: [oidmon]: Unable to connect to database,

REASON:

• As per error it clearly states that application is unable to connect to database.
• First step is to check whether database and listener are up and running.
• In our case we had upgraded database from 11.2.0.2 to 11.2.0.3 and had configured SCAN IP.
• Hence application was unable to connect to database with old configured Database connect string used in tnsnames.ora

SOLUTION:

Step 1) Goto below mentioned location
C:\Oracle\Middleware\asinst_1\config 

Step 2) Take backup of tnsnames.ora.  

Step 3) Change tnsnames.ora with SCAN IP after consulting your database team who upgraded DB.

Example:
OIDDB=(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = SCANIP)(PORT = PORTNO)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = DBName))) 

Step 4) Start opmnctl process
C:\Oracle\Middleware\asinst_1\bin
opmnctl.bat startall

ISSUE 2:

Unable to start opmn (Oid and oidmon) process
C:\Oracle\Middleware\asinst_1\bin
opmnctl.bat startall

ERROR OBSERVED:

Logs: C:\Oracle\Middleware\asinst_1\diagnostics\logs\OID\oid1

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 1809] [tid: 0] Guardian: Could not connect to database, retry again after 10 sec

[OID] [TRACE:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 1809] [tid: 0] Guardian: sgslupsCheckProcess: sigsend to pid 1834 failed, os error = 3

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 1809] [tid: 0] Guardian: [gsdsiConnect] ORA-12541, ORA-12541: TNS:no listener

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: Connecting to database, connect string is oiddb

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: Sent SIGTERM to process id = 1809

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: No such Process!!


[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: gslsgrDeleteInstace: Deleted registry entry cn=asinst_oid2_1_oid.us.oracle.com,cn=osdldapd,cn=subregistrysubentry

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: Updating Process Table...

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: Killed process pid = 1809

[OID] [NOTIFICATION:16] [] [OIDMON] [host: oid.us.oracle.com] [pid: 596] [tid: 0] OIDMON_STOP: oidmon: Stopped all OID process successfuly

REASON:

 

• After stopping all opmn process(opmnctl stopall), status of oidldapd and oidmon showed as "Stopped".
• All other ias-components were  in a "Down" state.
• This indicates that oidldapd and oidmon were not stopped cleanly.
• If an ias-component is not stopped cleanly then it is possible the contents of the "States" directory contains improper data which prevents the ias-component from starting.

SOLUTION:

Step 1) Goto below mentioned location
C:\Oracle\Middleware\asinst_1\bin
opmnctl.bat stopall 

Step 2) Goto below mentioned location
C:\Oracle\Middleware\asinst_1\config\OPMN\opmn\states 

Backup the contents of C:\Oracle\Middleware\asinst_1\config\OPMN\opmn\states to a temp directory and then delete the contents of this directory. 

Step 3) Goto below mentioned location
C:\Oracle\Middleware\asinst_1\bin
opmnctl.bat startall