While starting system components using opmnctl, it fails with below error.
$ ./opmnctl startall
opmnctl startall: starting opmn and all managed processes...
/orasw/MW_HOME/Oracle_BI1/opmn/bin/opmn: unexpected exit: code 512
opmnctl startall: opmn failed to start.
Reason
Check OPMN logs to find the issue.
cd $INSTANCE_HOME/diagnostics/logs/OPMN/opmn
$ ls -lrt
total 10072
-rw-r----- 1 oracle oinstall 0 Sep 27 2011 debug.log
-rw-r----- 1 oracle oinstall 432 Jan 13 07:06 opmn.out
-rw-r----- 1 oracle oinstall 257891 Jan 22 00:32 logquery~5.log
-rw-r----- 1 oracle oinstall 269575 Jan 22 00:32 logquery~4.log
-rw-r----- 1 oracle oinstall 273531 Jan 22 00:32 logquery~3.log
-rw-r----- 1 oracle oinstall 279896 Jan 22 00:32 logquery~2.log
-rw-r----- 1 oracle oinstall 290644 Jan 22 00:32 logquery~1.log
-rw-r----- 1 oracle oinstall 8894259 Feb 5 00:41 opmn.log
In the log we found that OPMN Local Port and Remote ports are already in use by some other process.
[2013-02-04T04:48:36][opmn][ERROR:1][17][OPMN][code:ons-listener]any: BIND (Address already in use)
[2013-02-04T04:49:50][opmn][NOTIFICATION:1][90][OPMN][code:ons-internal]ONS server initiated
[2013-02-04T04:49:50][opmn][NOTIFICATION:1][522][OPMN][code:pm-internal]PM state directory exists: /orasw/MW_HOME/instances/instance1/config/OPMN/opmn/states
[2013-02-04T04:49:50][opmn][NOTIFICATION:1][675][OPMN][code:pm-internal]OPMN server ready. Request handling enabled.
[2013-02-04T04:49:50][opmn][ERROR:1][17][OPMN][code:ons-listener]any: BIND (Address already in use)
[2013-02-05T00:41:07][opmn][NOTIFICATION:1][90][OPMN][code:ons-internal]ONS server initiated
[2013-02-05T00:41:07][opmn][NOTIFICATION:1][522][OPMN][code:pm-internal]PM state directory exists: /orasw/MW_HOME/instances/instance1/config/OPMN/opmn/states
[2013-02-05T00:41:07][opmn][NOTIFICATION:1][675][OPMN][code:pm-internal]OPMN server ready. Request handling enabled.
[2013-02-05T00:41:07][opmn][ERROR:1][17][OPMN][code:ons-listener]any: BIND (Address already in use)
Solution:
1. Go to $INSTANCE_HOME/config/OPMN//opmn.xml and check the ports.
In our case it was 9500 and 9501 ports
2. Check who is using these ports using below command.
$ netstat -an |grep 9500
tcp 0 0 ::ffff:10.241.37.9:9500 :::* LISTEN
tcp 0 0 ::ffff:10.241.37.9:9500 ::ffff:10.241.37.9:60741 ESTABLISHED
tcp 0 0 ::ffff:10.241.37.9:60741 ::ffff:10.241.37.9:9500 ESTABLISHED
$ netstat -an |grep 9501
tcp 0 0 :::9501 :::* LISTEN
So both ports are being used.
3. Either terminate processes using these ports or change ports in opmn.xml with ports not in use.
In our case, 9502 and 9503 weren't in use so replaced with these ports in opmn.xml.
4.Start system components now.
cd $INSTANCE_HOME/bin
$ ./opmnctl startall
opmnctl startall: starting opmn and all managed processes...
$ ./opmnctl status
Processes in Instance: instance1
---------------------------------+--------------------+---------+---------
ias-component | process-type | pid | status
---------------------------------+--------------------+---------+---------
coreapplication_obiccs1 | OracleBIClusterCo~ | 31392 | Alive
coreapplication_obisch1 | OracleBIScheduler~ | 31394 | Alive
coreapplication_obijh1 | OracleBIJavaHostC~ | 31393 | Alive
coreapplication_obips1 | OracleBIPresentat~ | 31391 | Alive
coreapplication_obis1 | OracleBIServerCom~ | 31395 | Alive
interesting piece of information, I had come to know about your web-page from my friend pramod, jaipur,i have read atleast eight posts of yours by now, and let me tell you, your blog gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanks a million once again, Regards,obiee online training
ReplyDeleteThanks a lot Dinesh!
ReplyDeleteAppreciation for nice Updates, I found something new and folks can get useful info about BEST obiee ONLINE TRAINING
ReplyDeleteThis comment has been removed by the author.
DeleteThank you
DeleteHai From the above post I understood that you had a good knowledge on Oracle 11g DBA.. I too had good knowledge on Oracle 11g DBA because i have Oracle 11g DBA online training in 123trainings at hyderabad.. They will provide excellent training.
ReplyDeletevery useful really good information thanks for posting such a good information it will hepls the people a lot keep it up , Regards, obiee training in hyderabad
ReplyDeleteI appreciate you sharing this article. Really thank you! Much obliged.
ReplyDeleteThis is one awesome blog article. Much thanks again.
oracle online training
sap sd online training
sap-crm-online-training
hadoop online training
Pretty good post. I just came across your site and wanted to say that I’ve really enjoyed reading your posts. In any case I’ll be subscribing to your feed and I hope you will keep a good work!Cheer!
ReplyDeletesap online training
sap fico online training
dotnet online training
qa-qtp-software-testing-training-tutorial
I was reading your blog this morning and noticed that you have a awesome
ReplyDeleteresource page. I actually have a similar blog that might be helpful or useful
to your audience.
Regards
sap sd and crm online training
sap online tutorials
sap sd tutorial
sap sd training in ameerpet
The best kits online trainings,thanks for sharing
ReplyDeleteOracle DBA Online Training institute
Oracle SOA Online Training institute
SalesForce Online Training institute
SAP ABAP Online Training institute
SAP Basis Online Training institute
Hi, thanks for sharing such an informative blog. I have read your blog and I gathered some needful information from your blog. Keep update your blog. Awaiting for your next update. sap abap crm training
ReplyDelete