How to Synchronize the AD tables FND_NODES, ADOP_VALID_NODES, and FND_OAM_CONTEXT_FILES in 12.2 when adop fails.

Adop failing with the Error adop exiting with status = 255 (Fail), due to fnd_nodes table has NULL for domain entry of this valid server.

Error:

adop failing with below error.

FUNCTION: ADOP::GlobalVars::_GetMandatoryArgs [ Level 1 ]
ERRORMSG: adop is not able to detect any valid application tier nodes in ADOP_VALID_NODES table.

Ensure autoconfig is run on all nodes.

[STATEMENT] Please run adopscanlog utility, using the command
"adopscanlog -latest=yes"
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 255 (Fail)

In this case, The fnd_nodes table has NULL for domain entry of this valid server.

Fix:

Test the following steps in a cloned instance first, and then migrate accordingly once the desired result is confirmed:

NOTE: 1. Due to the method required for "cleaning out" / "re-synchronizing" the following tables, it is EXPECTED / REQUIRED that the Applications have been shutdown.

                  The only thing running should be the Database Tier.

           2. A full backup should be taken before any testing begins.


1. Backup the fnd_oam_context_files, fnd_nodes, and adop_valid_nodes tables, and if on AD/TXK 8 or higher; ad_nodes_config_status in the EBS env nodes:

      Linux#> sqlplus applsys/<pwd>

      SQLPlus#> create table fnd_oam_context_files_bkp as select * from fnd_oam_context_files;

      SQLPlus#> create table fnd_nodes_bk as select * from fnd_nodes;

        SQLPlus#> create table adop_valid_nodes_bk as select * from adop_valid_nodes;

      If on AD/TXK 8 or higher:

      SQLPlus#> create table ad_nodes_config_status_bk as select * from ad_nodes_config_status;


 2. Truncate the following tables (Continue from step 1):

      SQLPlus#> truncate table fnd_oam_context_files;

      SQLPlus#> truncate table fnd_nodes;

      SQLPlus#> truncate table adop_valid_nodes;

      If on AD/TXK 8 or higher:

      SQLPlus#> truncate table ad_nodes_config_status;


3.  Run AutoConfig on the DB tier:

     Source the <RDBMS_ORACLE_HOME> home.

     Linux#> cd <RDBMS_ORACLE_HOME>/appsutil/scripts/<SID>_<HOSTNAME>/

     Linux#> ./adautocfg.sh

  ... Confirm Autoconfig completes successfully.
  ... If RAC, Repeat step 3 on all RAC nodes.

4.  Run Autoconfig on the run file system.

    Linux#> source <EBS_BASE>/EBSapps.env run

    Linux#> cd $ADMIN_SCRIPTS_HOME

    Linux#> ./adautocfg.sh

      ... Confirm Autoconfig completes successfully.

      ... If more than one EBS node, repeat step 4 on all EBS nodes.


5.  Run Autoconfig on the patch file system

Before running Autoconfig on the patch file system the ebs_login trigger MUST be disabled
After the successful completion of Autoconfig the ebs_login trigger MUST be re-enabled.

    a. Disable the ebs_login trigger using the following SQL.

    Linux#> sqlplus system/<pwd>

    SQLPlus#> alter trigger ebs_logon disable;


    b. At this time Run autoconfig with the patch env sourced .       

    Linux#> source <EBS_BASE>/EBSapps.env patch

    Linux#> cd $ADMIN_SCRIPTS_HOME

    Linux#> ./adautocfg.sh

     ... Confirm Autoconfig completes successfully

     ... If more than one EBS node, repeat step 5.b on all EBS nodes.


    c. Enable the ebs_login trigger using the following SQL.

    Linux#> sqlplus system/<pwd>

    SQLPlus#> alter trigger ebs_logon enable;

 6.  After Autoconfig has been run successfully on all nodes, run the following two (2) queries in order to verify the tables have been correctly populated:

SQL>    set pagesize 5
        set linesize 132
        col node_name format a15
        col server_id format a8
        col server_address format a15
        col platform_code format a4
        col webhost format a12
        col domain format a20
        col virtual_ip format a12

        select node_id, platform_code, support_db D, support_cp C, support_admin A, support_forms F, support_web W, node_name, server_id,
               server_address, domain, webhost, virtual_ip, status
          from fnd_nodes order by node_id;

SQL>    set pagesize 5
        set linesize 132
        col NAME format A20
        col VERSION format A12
        col PATH format A110
        col STATUS format A10

        select NAME,VERSION,PATH, STATUS from FND_OAM_CONTEXT_FILES;

How to create the patch file system in oracle apps R12.2

Below is an action plan to recreate the patch file system. You will need to make sure to have a valid backup of your system before you execute it as this plan will imply to remove information from your system and we might need to go back to the point the action plan is executed in case anything may not go as expected.

1) Adpreclone:

On the RUN File-System, source RUN File-System:

$ source EBSapps.env run

# RUN Admin server MUST be up for this step

# check status:

$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh status

# start if needed:

$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start

# execute adpreclone:

$ cd $INST_TOP/admin/scripts

$ perl adpreclone.pl appsTier

2) Detach PATCH homes:

# This step is performed from the RUN file-system detaching oracle_homes on the PATCH file-system:

$ cd $FMW_HOME/oracle_common/oui/bin

$ ./runInstaller -detachhome ORACLE_HOME={oracle home} -silent

 # for each PATCH oracle_home

./runInstaller -detachhome ORACLE_HOME=/u01/app/fs1/FMW_Home/webtier -silent

./runInstaller -detachhome ORACLE_HOME=/u01/app/fs1/FMW_Home/Oracle_EBS-app1 -silent

./runInstaller -detachhome ORACLE_HOME=/u01/app/fs1/FMW_Home/oracle_common -silent

# The 10.1.2 Oracle Home needs a different syntax

$ ./runInstaller -removeHome ORACLE_HOME=<s_tools_oh> -silent

$ ./runInstaller -removeHome ORACLE_HOME=/u01/app/fs1/EBSapps/10.1.2 -silent

3) Copy RUN to PATCH:

# it may be helpful for RUN environment to be DOWN for this step

 # copy each of these locations from RUN to PATCH

 <APPL_TOP>                              [ /u03/oracle/VIS/fs1/EBSapps/appl ]

 <COMMON_TOP>                            [ /u03/oracle/VIS/fs1/EBSapps/comn ]

 <OracleAS Tools 10.1.2 ORACLE_HOME>     [ /u03/oracle/VIS/fs1/EBSapps/10.1.2 ]

4) Remove PATCH FMW_HOME directory:

[ /u03/oracle/VIS/fs1/FMW_Home ]

5) Disable ebs_login:

Linux#> sqlplus system/<pwd>

SQLPlus#> alter trigger ebs_logon disable;

6) adcfgclone:

$ cd <PATCH COMMON_TOP>/clone/bin$ perl adcfgclone.pl appsTier

For the prompts in this step:

Do you want to add a node (yes/no) [no] : no

Enter the full path of Run File System Context file : <absolute path to the Run File System Context file>

RC-00217: Warning: Configuration home directory (s_config_home) evaluates to <PATCH s_config_home>. A directory with this name already exists and is not empty.

Do you want to continue (y/n)   : y

This is expected as part of the rebuild.

Target System Port Pool [0-99] : <desired PATCH File System Port Pool>

This will be the PATCH File System port pool. Would suggest that the RUN and PATCH port pools be different by approx twice number of managed servers on this node.

7) Verify work:

In the new terminal - source PATCH file-system

$ EBSapps.env patch

$ echo $FILE_EDITION

In new terminal - source RUN file-system

# Run Admin server MUST be up for this step

$ EBSapps.env run

$ adop phase=fs_clone force=yes

8) Enable ebs_login:

Linux#> sqlplus system/<pwd>

SQLPlus#> alter trigger ebs_logon enable;

OEM 12c: ORA-28001: the password has expired The Connect Descriptor was (DESCRIPTION=(ADDRESS_LIST = (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=)))(CONNECT_DATA=(SID=)(SERVER=DEDICATED)))

This error occurred on DB home page in OEM 12c.
ORA-28001: the password has expired The Connect Descriptor was (DESCRIPTION=(ADDRESS_LIST = (ADDRESS=(PROTOCOL=tcp)(HOST=<host>)(PORT=<port>)))(CONNECT_DATA=(SID=<sid>)(SERVER=DEDICATED)))

Cause:

The cause of this issue is monitoring user "DBSNMP" password expired.

select username, account_status from dba_users where username='DBSNMP';

Solution:

To resolve this issue we need to reset the password and unlock the user account using the below steps.

select 'alter user "'||d.username||'" identified by values '''||u.password||''';' from dba_users d, sys.user$ u where d.username = upper('DBSNMP') andu.user# = d.user_id;

'ALTERUSER"'||D.USERNAME||'"IDENTIFIEDBYVALUES'''||U.PASSWORD||''';'
--------------------------------------------------------------------------------
alter user "DBSNMP" identified by values '31B5G2AQ92890956';

SQL> alter user "DBSNMP" identified by values '31B5G2AQ92890956';

Note: If you want to keep the old password then you no need to do any changes in OEM but if you want to update the new password then follow the below steps in OEM.
1.      Click Targets->Databases.
2.      Select radio button for Database Instance Name and then Click Configure
3.      Change the Monitor Password, Click Next and Submit