Friday 11 February 2022

OEM 13c - emctl start oms failing with error - java.io.IOException: Cannot run program "null/common/bin/wlst.sh": error=2, No such file or directory

After the abnormal server reboot, we are unable to startup the OEM services and encounter the below error.

Error:

[oracle@krishna bin]$ ./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved.
[oracle@krishna bin]$ ./emctl status oms
Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved.
WebTier is Down
[oracle@krishna bin]$ ./emctl stop oms -all -force
Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved.
Stopping Oracle Management Server...
Error Occurred: Cannot run program "null/common/bin/wlst.sh": error=2, No such file or directory
Please check /u01/oracle/product/13.5.0/em/gc_inst/em/EMGC_OMS1/sysman/log/emctl.log for error details
[oracle@krishna bin]$

Solution:

OPTION1:
Restore the file emgc.properties file from valid backup.

OPTION2:
1. A typical emgc.properties in EM13c would have sample entries such as below:
  
COMMON_ORACLE_HOME=<OMS_HOME>/oracle_common
oracle.sysman.emSDK.svlt.ConsoleServerName=<OMS_HOST>\:<PORT>_Management_Service
JBO_POOL_TTL=-1
AS_PORT=0
EM_UPLOAD_HTTP_PORT=<EM_UPLOAD_HTTP_PORT>
EM_NODEMGR_HOME=<GC_INST>/user_projects/domains/GCDomain/nodemanager
MSPORT=<MSPORT>
ADMIN_SERVER_NAME=EMGC_ADMINSERVER
AS_HTTPS_PORT=7102
EM_REPOS_CONNECTDESCRIPTOR=(DESCRIPTION\=(ADDRESS_LIST\=(ADDRESS\=(PROTOCOL\=TCP)(HOST\=<REPO_DB_HOSTNAME>)(PORT\=<REPO_DB_LSNR_PORT>)))(CONNECT_DATA\=(SID\=<REPO_DB_SID>)))
OHS_ADMIN_PORT=<OHS_ADMIN_PORT>
OMS_HEAP_MAX=1740M
EM_CONSOLE_HTTP_PORT=<EM_CONSOLE_HTTP_PORT>
WLS_HOME=<OMS_HOME>/wlserver
JBO_LAZY_LOAD=true
EM_UPLOAD_HTTPS_PORT=<EM_UPLOAD_HTTPS_PORT>
oracle.sysman.emRep.repositoryMode=repository
AS_HOST=<OMS_HOSTNAME>
OMS_PERMGEN_MAX=768M
EM_DOMAIN_HOME=<GC_INST>/user_projects/domains/GCDomain
EM_CONSOLE_HTTPS_PORT=<EM_CONSOLE_HTTPS_PORT>
CURRENT_OMS_MODE=default
EM_NODEMGR_PORT=<EM_NODEMGR_PORT>
EM_DOMAIN_NAME=GCDomain
MS_HTTPS_PORT=<MS_HTTPS_PORT>
EM_INSTANCE_HOST=<OMS_HOSTNAME>
AS_COHERENCE_PORT=<AS_COHERENCE_PORT>
WEBTIER_ORACLE_HOME=<OMS_HOME>/ohs
MW_HOME=<OMS_HOME>
JBO_MIN_POOL_SIZE=1
EM_INSTANCE_HOME=<GC_INST>/em/EMGC_OMSn
JBO_MAX_POOL_SIZE=50
IS_ADMIN_HOST=true
AS_USERNAME=weblogic
OMS_HEAP_MIN=256M
ORACLE_HOME=<OMS_HOME>
NM_USER=nodemanager
OPSS_USER=SYSMAN_OPSS
OMSNAME=EMGC_OMSn
JBO_MAX_CURSORS=5
EM_REPOS_USER=SYSMAN
JBO_RECYC_THRESHOLD=50
OMS_PERMGEN_MIN=128M
OHS_COMP_NAME=ohsn
 
Note: The port numbers and other settings are OMS specific and the above-depicted entries are just a sample.
Recreate the file using the entries from one of the EMGC_OMSn.out* file under <gc_inst>/user_projects/domains/GCDomain/servers/EMGC_OMSn/logs on the respective OMS node.

2. Start the OMS

Reference - EM 12c: Emctl start/stop/status oms Failed With Error Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.log4j.Logger (Doc ID 2065293.1)