Thursday, December 2, 2021

Applying AD TXK 13 Release Update Packs to Oracle E-Business Suite Release 12.2 (Doc ID 1617461.1) (Doc ID 2758994.1)


Hai, in this blog we learn how to upgrade AD/TXK to latest version. Here we are upgrading to AD/TXK to 13 C Delta

#############################################################################
Important MOS DOC to refer upgrading AD/TXK:
1.) Applying the Latest AD and TXK Release Update Packs to Oracle E-Business Suite Release 12.2 (Doc ID 1617461.1) 
1.1) Oracle E-Business Suite Release 12.2 System Schema Migration (Doc ID 2755875.1)
1.2) Oracle E-Business Suite Applications DBA and Technology Stack Release Notes for R12.AD.C.Delta.13 and R12.TXK.C.Delta.13 (Doc ID 2758994.1)

2.) Database Patches Required by Oracle E-Business Suite on Oracle Engineered Systems: Exadata Database Machines and SuperClusters (Doc ID 1392527.1)

3.) Oracle E-Business Suite Release 12.2: Consolidated List of Patches and Technology Bug Fixes (Doc ID 1594274.1)

4.) Critical Patch Update (CPU) Program Oct 2021 Patch Availability Document (PAD) (Doc ID 2796575.1)

5.) Oracle E-Business Suite Release 12 Critical Patch Update Availability Document (July 2021) (Doc ID 2770321.1)

6.) Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12 (Doc ID 2484000.1)



OS Check Before Prepare:
root@ebsprodapp3:~# cat /etc/release
                            Oracle Solaris 11.4 SPARC
  Copyright (c) 1983, 2020, Oracle and/or its affiliates.  All rights reserved.
                             Assembled 26 March 2020
root@ebsprodapp3:~#
root@ebsprodapp3:~# uname -a
SunOS ebsprodapp3 5.11 11.4.20.4.0 sun4v sparc sun4v
root@ebsprodapp3:~#
root@ebsprodapp3:~# prtconf
System Configuration:  Oracle Corporation  sun4v
Memory size: 391680 Megabytes
System Peripherals (Software Nodes):

prtconf: devinfo facility not available
root@ebsprodapp3:~#
root@ebsprodapp3:~# pkg info entire
             Name: entire
          Summary: entire incorporation including Support Repository Update
                   (Oracle Solaris 11.4.20.4.0).
      Description: This package constrains system package versions to the same
                   build.  WARNING: Proper system update and correct package
                   selection depend on the presence of this incorporation.
                   Removing this package will result in an unsupported system.
                   For more information see:
                   https://support.oracle.com/rs?type=doc&id=2433412.1
         Category: Meta Packages/Incorporations
            State: Installed
        Publisher: solaris
          Version: 11.4 (Oracle Solaris 11.4.20.4.0)
           Branch: 11.4.20.0.1.4.0
   Packaging Date: Thu Mar 26 17:29:32 2020
Last Install Time: Wed Oct 14 20:51:30 2020
             Size: 2.52 kB
             FMRI: pkg://solaris/entire@11.4-11.4.20.0.1.4.0:20200326T172932Z
root@ebsprodapp3:~#

Current AD & TXK version:
SQL> col CODELEVEL for a30;
select ABBREVIATION,CODELEVEL from ad_trackable_entities where abbreviation in('ad','txk');
ABBREVIATION                   CODELEVEL
------------------------------ ------------------------------
ad                             C.12
txk                            C.12
SQL>

Upgrade to latest OPATCH on DB Node:
oraoci@ebsproddb1:~$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.24

OPatch succeeded.
oraoci@ebsproddb1:~$
oraoci@ebsproddb1:/u01$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch_old_24

oraoci@ebsproddb1:~$ unzip /u01/p6880880_122010_SOLARIS64.zip -d $ORACLE_HOME
oraoci@ebsproddb1:~$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.28

OPatch succeeded.
oraoci@ebsproddb1:~$

Run latest ETCC checkDBpatch.sh on DB Node:
oraoci@ebsproddb1:~/ETCC$ checkDBpatch.sh

 +===============================================================+
 |    Copyright (c) 2005, 2020 Oracle and/or its affiliates.     |
 |                     All rights reserved.                      |
 |             Oracle E-Business Suite Release 12.2              |
 |          Database EBS Technology Codelevel Checker            |
 +===============================================================+

Validating context file: /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/PRODPDB_ebsproddb1.xml

Using context file from currently set database environment:
/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/PRODPDB_ebsproddb1.xml


Starting Database EBS Technology Codelevel Checker, Version 120.87
Wed Nov 17 09:06:40 +03 2021
Log file for this session : /export/home/oraoci/ETCC/log/checkDBpatch_178.log

Identifying database release.
Database release set to 12.1.0.2.

Connecting to database.
Database connection successful.

Database PRODPDB is in READ WRITE mode.

Identifying APPS and APPLSYS schema names.
 - APPS schema : APPS
 - APPLSYS schema : APPLSYS

Checking for DB-ETCC results table.
Table to store DB-ETCC results already exists in the database.

Checking if InMemory option is enabled.
InMemory option is not enabled in the database.

Running on an Engineered System.

Bugfix file ./db/onprem/txk_R1220_DB_base_bugs.xml : 120.0.12020000.61
This file will be used for identifying missing bugfixes.

Mapping file ./db/onprem/txk_R1220_DB_mappings.xml : 120.0.12020000.36
This file will be used for mapping bugfixes to patches.

Identified RDBMS DST version 18.

Checking Bugfix XML file for 12.1.0.2_BP

Obtained list of bugfixes to be applied and the list to be rolled back.
Now checking Database ORACLE_HOME.

The opatch utility is version 12.2.0.1.24.
DB-ETCC is compatible with this opatch version.

Found patch records in the inventory.

Checking Mapping XML file for 12.1.0.2.210119ProactiveBP

Patch mapping data is not available for 12.1.0.2.210119ProactiveBP
Refer to Doc ID 1594274.1 for the recommended Bundle Patch versions.

Some required one-off bugfixes are missing from the Database ORACLE_HOME.
The missing bugfixes are:
17777718
26436717
32355765
33114885

** Refer to MOS Doc ID "1392527.1:Database Patches Required by Oracle E-Business Suite on Oracle Engineered Systems: Exadata Database Machines and SuperClusters" to find the recommended combination of patches.
 **

Apply the missing bugfixes and then rerun the script.

+-----------------------------------------------------------------------------+
A consolidated zip file with the required patches for Database release
12.1.0.2.210119 ProactiveBP is available on My Oracle Support via:
 
  Patch 33186784
    - EBS RELEASE 12.2 CONSOLIDATED DATABASE FIXES FOR OCT 2021
 

Note: This zip does not include Release Updates, Bundle Patches or Patch Set Updates.

+-----------------------------------------------------------------------------+

Stored Technology Codelevel Checker results in the database PRODPDB successfully.

Finished checking fixes for Oracle Database: Wed Nov 17 09:07:06 +03 2021

Log file for this session: /export/home/oraoci/ETCC/log/checkDBpatch_178.log

===============================================================================
oraoci@ebsproddb1:~/ETCC$



Run latest ETCC checkMTpatch.sh on APPS Node:
apploci@ebsprodapp3:/u01/AD_TXK_13$ checkMTpatch.sh

 +===============================================================+
 |    Copyright (c) 2005, 2020 Oracle and/or its affiliates.     |
 |                     All rights reserved.                      |
 |             Oracle E-Business Suite Release 12.2              |
 |        Application Tier Technology Codelevel Checker         |
 +===============================================================+

Using context file from currently set applications environment:
/u01/OCI/fs1/inst/apps/PRODPDB_ebsprodapp3/appl/admin/PRODPDB_ebsprodapp3.xml

Starting Application Tier Technology Codelevel Checker
Version: 120.0.12020000.51.
Wed Nov 17 09:41:16 +03 2021
Log file for this session: /u01/AD_TXK_13/log/checkMTpatch_1004.log

Bugfix XML file version: 120.0.12020000.53
This file will be used for identifying missing bugfixes.

Mapping XML file version: 120.0.12020000.39
This file will be used for mapping bugfixes to patches.

Checking for prerequisite bugfixes in File Edition: run

Enter the password for the APPS user:
Connecting to database.
Database connection successful.
The installed AD.C.12 codelevel supports storing the results in the database.

Checking for ETCC results table.
Table to store results already exists in database.

===============================================================================
Oracle Forms and Reports
===============================================================================
Now examining product Oracle Forms and Reports.

Oracle Home = /u01/OCI/fs1/EBSapps/10.1.2.
Product version = 10.1.2.3.0.
Checking required bugfixes for Oracle Forms and Reports 10.1.2.3.0.
All required bugfixes are present for Oracle Forms and Reports.
These results have been stored in the database.

Checking required bugfixes for RSF within Forms 10.1.0.5.0.
All required bugfixes are present for RSF within Forms.
These results have been stored in the database.

===============================================================================
Oracle Fusion Middleware (FMW) - Web Tier
===============================================================================
Now examining product Oracle Fusion Middleware (FMW) - Web Tier.

Oracle Home = /u01/OCI/fs1/FMW_Home/webtier.
Product Version = 11.1.1.9.0
Checking required bugfixes for FMW - Web Tier 11.1.1.9.0.
  Missing Bugfix: 31909295  -> Patch 32287205
  Missing Bugfix: 31748000  -> Patch 32287205
  Missing Bugfix: 31696577  -> Patch 32287205
  Missing Bugfix: 27072923  -> Patch 32287205
  Missing Bugfix: 31169366  -> Patch 32287205
  Missing Bugfix: 17428816  -> Patch 32287205
  Missing Bugfix: 31658943  -> Patch 32287205
  Missing Bugfix: 32294066  -> Patch 33311587
  Missing Bugfix: 32593782  -> Patch 33311587
  Missing Bugfix: 31586302  -> Patch 33311587
  Missing Bugfix: 29857440  -> Patch 33311587
  Missing Bugfix: 32928416  -> Patch 32928416
  Missing Bugfix: 32949290  -> Patch 33144848
The above list shows missing bugfixes for FMW - Web Tier.
These results have been stored in the database.

Checking required bugfixes for RSF within FMW Web tier 11.1.0.7.0.
All required bugfixes are present for RSF within FMW Web tier.
These results have been stored in the database.

===============================================================================
Oracle Fusion Middleware (FMW) - oracle_common
===============================================================================
Now examining product Oracle Fusion Middleware (FMW) - oracle_common.

Oracle Home = /u01/OCI/fs1/FMW_Home/oracle_common.
Product Version = 11.1.1.9.0
Checking required bugfixes for FMW - oracle common 11.1.1.9.0.
  Missing Bugfix: 26933408  -> Patch 26933408
The above list shows missing bugfixes for FMW - oracle common.
These results have been stored in the database.

===============================================================================
Oracle WebLogic Server (WLS)
===============================================================================
Now examining product Oracle WebLogic Server (WLS).

Oracle Home = /u01/OCI/fs1/FMW_Home/wlserver_10.3.
Product Version = 10.3.6.0.210119

Note that for Oracle WebLogic Server, patches rather than bugfixes are verified.

Checking required patches for Oracle WebLogic Server (WLS) 10.3.6.0.210119.
  Missing Patch ID: 31241365
The above list shows missing patches for Oracle WebLogic Server.
If you have applied other Oracle WebLogic Server patches, they may have included the bugfixes needed.
Contact Oracle Support if you require assistance in determining whether this is the case.
These results have been stored in the database.

===============================================================================

Generating Patch Recommendation Summary.

===============================================================================
PATCH RECOMMENDATION SUMMARY
===============================================================================
One or more products have bugfixes missing.
The default patch recommendations to install these missing bugfixes are:

-------------------------------------------------------------------------------
Oracle Fusion Middleware (FMW) - Web Tier 11.1.1.9.0
-------------------------------------------------------------------------------
  Patch 32287205 [IMPORTANT: Follow Note 2555355.1 before applying.]
    - Filename: p32287205_111190_SOLARIS64.zip

  Patch 33311587
    - Filename: p33311587_111190_SOLARIS64.zip

  Patch 32928416
    - Filename: p32928416_111190_SOLARIS64.zip

  Patch 33144848
    - Filename: p33144848_111190_SOLARIS64.zip


-------------------------------------------------------------------------------
Oracle Fusion Middleware (FMW) - oracle_common 11.1.1.9.0
-------------------------------------------------------------------------------
  Patch 26933408
    - Filename: p26933408_111190_Generic.zip


-------------------------------------------------------------------------------
Oracle WebLogic Server (WLS) 10.3.6.0.210119
-------------------------------------------------------------------------------
  Patch 31241365 [SU Patch [KKLY]]
    - Filename: p31241365_1036_SOLARIS64.zip


Apply the required patches and rerun this script.

+-----------------------------------------------------------------------------+
A consolidated zip file with the required application tier patches is
available on My Oracle Support via:
 
  Patch 33186821
    - EBS RELEASE 12.2 CONSOLIDATED FMW FIXES FOR OCT 2021

+-----------------------------------------------------------------------------+

See Doc ID 1594274.1 for any special instructions regarding these patches.
Footnotes in Doc ID 1594274.1 also apply to corresponding overlay patches.

Finished checking prerequisite patches for File Edition: run.
Wed Nov 17 09:42:15 +03 2021

Log file for this session: /u01/AD_TXK_13/log/checkMTpatch_1004.log

===============================================================================
apploci@ebsprodapp3:/u01/AD_TXK_13$

================================================================================================================
Flashback Backup and Restore Plan for Failure: (1. Bkp DB Home, 2. Database, 3. Apps cold backp)
root@ebsprodapp3:/u01# tar -cvzf OCI_bkp_29112011.tar.gz OCI
root@ebsprodapp3:/u01#

If Applying Big Patches like DB-PSU, ad/txk, EBS Upgrade, patch then modify sga pga accordingly and run gather stats to speedup process verify on TEST servers:
- Follow flashback enable blog for all steps
- Tar Backup EBS Application at least Current RFS
- Tar backup RDBMS Home
- Setting up SGA,PGA
- gather stats for SYS Schema
- Flashback point

###############################################################
###############################################################
###############################################################

SQL> create restore point before_adtxk guarantee flashback database;

Restore point created.

SQL> select * from v$restore_point;

       SCN DATABASE_INCARNATION# GUA STORAGE_SIZE TIME
---------- --------------------- --- ------------ ---------------------------------------------------------------------------
RESTORE_POINT_TIME                                                          PRE
--------------------------------------------------------------------------- ---
NAME                                                                                                                                 CON_ID
-------------------------------------------------------------------------------------------------------------------------------- ----------
4.3186E+10                     2 YES    524288000 29-NOV-21 01.26.19.000000000 PM
                                                                            YES
BEFORE_ADTXK                                                                                                                              0


SQL> 
SQL> flashback database to restore point BEFORE_ADTXK; (approx 2 mins)

Flashback complete.

SQL> alter database open resetlogs;

Database altered.

SQL> drop restore point BEFORE_ADTXK;

Restore point dropped.

SQL> alter database flashback off;

Database altered.

SQL> 


================================================================================================================

Download patches mentioned in checkDB and checkMT :
3.2 Path B: Existing Release 12.2 customers (below AD-TXK Delta 13 codelevel)
Unzip ad and txk 13 patches to patch_top directory:
unzip /u01/AD_TXK_13/p32394134_R12.AD.C_R12_SOLARIS64.zip -d $PATCH_TOP
unzip /u01/AD_TXK_13/p32394134_R12.AD.C_R12_ar.zip  -d $PATCH_TOP

unzip /u01/AD_TXK_13/p32392507_R12.TXK.C_R12_GENERIC.zip -d $PATCH_TOP
unzip /u01/AD_TXK_13/p32392507_R12.TXK.C_R12_ar.zip -d $PATCH_TOP

unzip /u01/AD_TXK_13/p33441060_R12.AD.C_R12_GENERIC.zip -d $PATCH_TOP
unzip /u01/AD_TXK_13/p33550674_R12.TXK.C_R12_GENERIC.zip -d $PATCH_TOP
pwd


Copy adgrants form ad patch folder to APPL_TOP/admin folder:
apploci@ebsprodapp3:~$ head $APPL_TOP/admin/adgrants.sql
apploci@ebsprodapp3:~$ mv $APPL_TOP/admin/adgrants.sql $APPL_TOP/admin/adgrants_bkp.sql
apploci@ebsprodapp3:~$ 
apploci@ebsprodapp3:~$ find $PATCH_TOP -iname adgrants.sql :(need to copy and run correct adgrants file check version of adgrants in adpatch & support_patch folders given patches in section 4.1 Critical AD Patches)
apploci@ebsprodapp3:~$ strings -a $PATCH_TOP/*/admin/adgrants.sql | grep Header
apploci@ebsprodapp3:~$ head $PATCH_TOP/32394134/admin/adgrants.sql
apploci@ebsprodapp3:~$ head $PATCH_TOP/33441060/admin/adgrants.sql
apploci@ebsprodapp3:~$ cp $PATCH_TOP/33441060/admin/adgrants.sql $APPL_TOP/admin


Copy adgrants form ad patch folder to RDBMS Home:
apploci@ebsprodapp3:...e/EBSapps/patch/33441060/admin$ scp $PATCH_TOP/33441060/admin/adgrants.sql oraoci@192.168.4.12:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/admin
Password:
adgrants.sql                                                                                                                                                          100%   85KB  54.3MB/s   00:00
root@ebsprodapp3:/u01/AD_TXK_13/32394134/admin

change permission to orauser on DB node:
root@ebsproddb1:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/admin# chown oraoci:oinstall adgrants.sql
oraoci@ebsproddb1:~$
oraoci@ebsproddb1:~$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 17 19:06:31 2021

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
ORACLE instance started.

Total System Global Area 4294967296 bytes
Fixed Size                  2922264 bytes
Variable Size            3388999912 bytes
Database Buffers          889192448 bytes
Redo Buffers               13852672 bytes
Database mounted.
Database opened.
SQL> create restore point before_adtxk guarantee flashback database;
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
oraoci@ebsproddb1:~$
oraoci@ebsproddb1:~$
oraoci@ebsproddb1:~$
oraoci@ebsproddb1:~$ sqlplus /nolog
SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 17 19:07:38 2021

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

SQL> @$ORACLE_HOME/appsutil/admin/adgrants.sql apps; (approx. 1+Mins)
Connected.

---------------------------------------------------
--- adgrants.sql started at 2021-11-30 08:21:14 ---

Creating PL/SQL profiler objects.

---------------------------------------------------
--- profload.sql started at 2021-11-30 08:21:14 ---
Testing for correct installation
SYS.DBMS_PROFILER successfully loaded.

-----------------------------------------------------
--- profload.sql completed at 2021-11-30 08:21:44 ---

--------------------------------------------------
--- proftab.sql started at 2021-11-30 08:21:44 ---

-----------------------------------------------------
--- profltab.sql completed at 2021-11-30 08:21:45 ---

Installing Hierarchical Profiler.


HPTAB_SCRIPT
---------------------------
?/rdbms/admin/dbmshptab.sql

Creating PL/SQL Package AD_ZD_SYS.

End of Creating PL/SQL Package AD_ZD_SYS.

Removing logs from prior runs of adgrants.sql
-
Loading Stylesheets if missing
*****************************************************
***************** ATTENTION *************************
*                                                   *
*    New user EBS_SYSTEM has been created           *
*                                                   *
* Unlock EBS_SYSTEM with following command:         *
*                                                   *
*   ALTER USER EBS_SYSTEM ACCOUNT UNLOCK;           *
*                                                   *
* The initial EBS_SYSTEM Password is required to    *
* be changed to the same as the SYSTEM password     *
*                                                   *
* Set password with the following command:          *
*                                                   *
*   ALTER USER EBS_SYSTEM IDENTIFIED BY <password>; *
*                                                   *
*****************************************************
-
Start granting to EBS_SYSTEM

Done granting to  EBS_SYSTEM

Creating ADGRANTS_VER_PKG

Start of PURGE DBA_RECYCLEBIN.

Generating list of ERRORS and WARNINGS to print out

The following ERRORS and WARNINGS have been encountered during this adgrants session:


Grants given by this script have been written to the ad_zd_logs table.
You can run $AD_TOP/sql/ADZDSHOWLOG.sql to produce a report showing these grants.

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced Analytics
and Real Application Testing options
oraoci@ebsproddb1:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/admin$
oraoci@ebsproddb1:~$
SQL> select USERNAME,ACCOUNT_STATUS from dba_users where USERNAME='EBS_SYSTEM';

USERNAME
--------------------------------------------------------------------------------
ACCOUNT_STATUS
--------------------------------
EBS_SYSTEM
LOCKED
SQL>

SQL> ALTER USER EBS_SYSTEM ACCOUNT UNLOCK;

User altered.

SQL> ALTER USER EBS_SYSTEM IDENTIFIED BY manager1;

User altered.

SQL>

Validate Schema from APPS_TIER:
apploci@ebsprodapp3:~$ perl $PATCH_TOP/32394134/ad/bin/adValidateEbssystemSchema.pl

Enter the APPS password:
Enter the SYSTEM password:
Validating APPS credentials...validated successfully
Validating SYSTEM credentials...validated successfully
Validating EBS_SYSTEM user
ebssys entity is not registered
All required validations completed successfully
apploci@ebsprodapp3:~$



Stop application services and apply in downtime mode direct on RFS:
apploci@ebsprodapp3:~$ adstpall.sh apps/appsocim7 
apploci@ebsprodapp3:~$ pkill -u apploci
apploci@ebsprodapp3:~$ adop -status

Enter the APPS password:
Connected.
==============================================================
ADOP (C.Delta.12)
Session Id: 27
Command: status
Output: /u01/OCI/fs_ne/EBSapps/log/adop/27/20211130_083445/adzdshowstatus.out
===============================================================
Node Name       Node Type  Phase           Status          Started              Finished             Elapsed
--------------- ---------- --------------- --------------- -------------------- -------------------- ------------
ahcsc1zioclient master     FS_CLONE        COMPLETED       2021/11/22 13:13:28  2021/11/22 14:14:02  1:60:34
020301

File System Synchronization Type: Full


adop exiting with status = 0 (Success)
apploci@ebsprodapp3:~$

AD-13 upgrade patch along with section 4.1 Critical AD Patches as per (Doc ID 1617461.1):
It is better to run adop phases in screen session under root user:

10. Run the following command to apply Patch 32394134 (R12.AD.C.Delta.13) plus any additional critical patches as mentioned in the note box:
Note: As per 4.1 Critical AD Patches, add any relevant AD patches as a comma-separated list of patches to this command. Add merge=yes to the end of your command, as shown here:
$ adop phase=apply patches=32394134,<Your 1st critical AD patch>,<Your 2nd critical AD patch> merge=yes

Prechecks before AD TXK Patches
free -h   ---> free memory on both DB and apps node
echo 3 > /proc/sys/vm/drop_caches && swapoff -a && swapon -a && printf '\n%s\n' ' Ram-cache and the swap get cleared'
grep -A 5 MISSING_TRANSLATED_VERSION  $APPL_TOP/admin/${TWO_TASK}/adalldefaults.txt   ---> change to Yes
select count(*) from all_objects where status like 'INVALID' and owner='APPS';

root@ebsprodapp3:~# screen -S ADTXK
apploci@ebsprodapp3:~$ time adop phase=apply patches=32394134,33441060,32394134_AR:u32394134.drv merge=yes apply_mode=downtime (approx. 2hrs 30+Mins)

Tip for AD-patch: after applying ad patch finally patch will run autoconfig, then ad patching finishes

apploci@ebsprodapp3:~$ find $ADOP_LOG_HOME/27 -iname u_ad_*.log
apploci@ebsprodapp3:~$ find $ADOP_LOG_HOME/28 -iname u_ad_*.lgi
apploci@ebsprodapp3:~$ find $ADOP_LOG_HOME/27 -iname adwork*.log
apploci@ebsprodapp3:~$ adopmon

apploci@ebsprodapp3:~$ adctrl
apploci@ebsprodapp3:~$ ps -ef | grep adworker
apploci@ebsprodapp3:~$

SQL> select BUG_NUMBER,CREATION_DATE,TRACKABLE_ENTITY_ABBR,LANGUAGE from apps.ad_bugs where BUG_NUMBER in ('32394134'.'33441060','32392507','33550674') order by LANGUAGE;
SQL> select ad_patch.is_patch_applied('R12',-1,32394134) from dual;
AD_PATCH.IS_PATCH_APPLIED('R12',-1,32394134)
--------------------------------------------------------------------------------
EXPLICIT
SQL>
SQL> select BUG_NUMBER,CREATION_DATE,TRACKABLE_ENTITY_ABBR,LANGUAGE from ad_bugs where BUG_NUMBER='32394134';
BUG_NUMBER                     CREATION_ TRACKABLE_ENTITY_ABBR          LANG
------------------------------ --------- ------------------------------ ----
32394134                       17-NOV-21 ad                             US
32394134                       17-NOV-21 ad                             AR
SQL>


TXK-13 upgrade patch along with section 4.2 Critical TXK Patches as per (Doc ID 1617461.1):
Tip for TXK-patch: txk-upgrade patch includes compiling ojsp which it self requires around 30 mins and finally autoconfig before completing txk-14 patching
apploci@ebsprodapp3:~$ time adop phase=apply patches=32392507,33550674,32392507_AR:u32392507.drv merge=yes apply_mode=downtime (approx. 60+Mins)

apploci@ebsprodapp3:~$ adop -status

Enter the APPS password:
Connected.
==============================================================
ADOP (C.Delta.13)
Session Id: 28
Command: status
Output: /u01/OCI/fs_ne/EBSapps/log/adop/28/20211130_111803/adzdshowstatus.out
===============================================================
Node Name       Node Type  Phase           Status          Started              Finished             Elapsed
--------------- ---------- --------------- --------------- -------------------- -------------------- ------------
ahcsc1zioclient master     APPLY           ACTIVE          2021/11/30 08:40:57  2021/11/30 11:16:34  2:35:37
020301

                           CLEANUP         NOT STARTED

File System Synchronization Type: None

adop exiting with status = 0 (Success)
apploci@ebsprodapp3:~$
apploci@ebsprodapp3:~$ adautocfg.sh
apploci@ebsprodapp3:~$ adop phase=cleanup cleanup_mode=full workers=24 (if req do cleanup approx. 1+Mins)
apploci@ebsprodapp3:~$ adop phase=fs_clone (approx. 120+Mins)
apploci@ebsprodapp3:~$


Section 5: Update Database Tier to Latest AutoConfig Code:
Re-create appsutil directory for RDBMS Home:
apploci@ebsprodapp3:~$ perl $AD_TOP/bin/admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at /u01/OCI/fs2/inst/apps/PRODPDB_ebsprodapp3/admin/log/MakeAppsUtil_12011830.log
output located at /u01/OCI/fs2/inst/apps/PRODPDB_ebsprodapp3/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
apploci@ebsprodapp3:~$ scp $INST_TOP/admin/out/appsutil.zip oraoci@192.168.4.12:/u01/app/oraoci/product/12.1.0.2/dbhome_1
Password:
appsutil.zip                                                                                                                                                          100% 3986KB  54.9MB/s   00:00
apploci@ebsprodapp3:~$

oraoci@ebsproddb1:~$ cd $ORACLE_HOME
oraoci@ebsproddb1 dbhome_1$ mv $ORACLE_HOME/appsutil $ORACLE_HOME/appsutil_old
oraoci@ebsproddb1 dbhome_1$ unzip -o $ORACLE_HOME/appsutil.zip -d $ORACLE_HOME
oraoci@ebsproddb1 dbhome_1ls -ltrh $ORACLE_HOME/appsutil_old/jre*.tar.gz
oraoci@ebsproddb1 dbhome_1cp $ORACLE_HOME/appsutil_old/jre*.tar.gz $ORACLE_HOME/appsutil
oraoci@ebsproddb1 dbhome_1cd $ORACLE_HOME/appsutil
oraoci@ebsproddb1 appsutil$ tar -xvzf jre-7u251-solaris-sparc.tar.gz
oraoci@ebsproddb1 appsutil$ tar -xvzf jre-7u251-solaris-sparcv9.tar.gz
oraoci@ebsproddb1 appsutil$ mv jre1.7.0_251 jre

Create Contextfile for Database Tier:
oraoci@ebsproddb1:~$ cd $ORACLE_HOME/appsutil/bin
oraoci@ebsproddb1 bin$ perl adbldxml.pl appsuser=apps template=$ORACLE_HOME/appsutil/template/adxdbctx.tmp out=$ORACLE_HOME/appsutil/$CONTEXT_NAME.xml

Starting context file generation for db tier..
Using JVM from /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/jre/bin/java to execute java programs..

The log file for this adbldxml session is located at:
/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/log/adbldxml_12011857.log

Enter Database Service Name: PRODPDB

Enter the value for Display Variable: :0.0

The context file has been created at:
/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/PRODPDB_ebsproddb1.xml
oraoci@ebsproddb1 bin$

Create Scripts directory: 
oraoci@ebsproddb1 bin$ sh adconfig.sh contextfile=$ORACLE_HOME/appsutil/$CONTEXT_NAME.xml
Enter the APPS user password:
The log file for this session is located at: /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/log/PRODPDB_ebsproddb1/12011901/adconfig.log

AutoConfig is configuring the Database environment...

AutoConfig will consider the custom templates if present.
        Using ORACLE_HOME location : /u01/app/oraoci/product/12.1.0.2/dbhome_1

Value of s_dbcset is AR8MSWIN1256

Character set is not present in the allowed list. Need to add orai18n.jar to the CLASSPATH.

Library orai18n.jar exists.

Value of s_dbcset is AR8MSWIN1256

Character set is not present in the allowed list. Need to add orai18n.jar to the CLASSPATH.

Library orai18n.jar exists.
        Classpath                   : :/u01/app/oraoci/product/12.1.0.2/dbhome_1/jdbc/lib/ojdbc6.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/java/xmlparserv2.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/java:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/netcfg.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/ldapjclnt12.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/orai18n.jar

        Using Context file          : /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/PRODPDB_ebsproddb1.xml

Context Value Management will now update the Context file

        Updating Context file...COMPLETED

        Attempting upload of Context file and templates to database...COMPLETED

Updating rdbms version in Context file to db121
Updating rdbms type in Context file to 64 bits
Configuring templates from ORACLE_HOME ...

AutoConfig completed successfully.
oraoci@ebsproddb1:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/bin$



Run AutoConfig on the <RDBMS ORACLE_HOME>:
oraoci@ebsproddb1:$ $ORACLE_HOME/appsutil/scripts/$CONTEXT_NAME/adautocfg.sh
Enter the APPS user password:
The log file for this session is located at: /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/log/PRODPDB_ebsproddb1/12011902/adconfig.log

AutoConfig is configuring the Database environment...

AutoConfig will consider the custom templates if present.
        Using ORACLE_HOME location : /u01/app/oraoci/product/12.1.0.2/dbhome_1

Value of s_dbcset is AR8MSWIN1256

Character set is not present in the allowed list. Need to add orai18n.jar to the CLASSPATH.

Library orai18n.jar exists.

Value of s_dbcset is AR8MSWIN1256

Character set is not present in the allowed list. Need to add orai18n.jar to the CLASSPATH.

Library orai18n.jar exists.
        Classpath                   : :/u01/app/oraoci/product/12.1.0.2/dbhome_1/jdbc/lib/ojdbc6.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/java/xmlparserv2.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/java:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/netcfg.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/ldapjclnt12.jar:/u01/app/oraoci/product/12.1.0.2/dbhome_1/jlib/orai18n.jar

        Using Context file          : /u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/PRODPDB_ebsproddb1.xml

Context Value Management will now update the Context file

        Updating Context file...COMPLETED

        Attempting upload of Context file and templates to database...COMPLETED

Updating rdbms version in Context file to db121
Updating rdbms type in Context file to 64 bits
Configuring templates from ORACLE_HOME ...

AutoConfig completed successfully.
oraoci@ebsproddb1:/u01/app/oraoci/product/12.1.0.2/dbhome_1/appsutil/scripts/PRODPDB_ebsproddb1$

Run AutoConfig on the Apps Tier
[apploci@ebsprodapp3 ~]$ adautocfg.sh

AutoConfig completed successfully.
[apploci@ebsprodapp3 ~]$
----------------------------------------- 
IF autoconfig faile with belwo error then follw this steps in ID
AutoConfig failed with Version Conflicts after patch 27102203 applied as post-installation task (Doc ID 2639310.1)
solution
[applmgr@uatapp01 ~]$ adautocfg.sh
Enter the APPS user password:

The log file for this session is located at: /u01/TEST/fs2/inst/apps/TEST_uatapp04/admin/log/02161113/adconfig.log


wlsDomainName: EBS_domain_TEST
WLS Domain Name is VALID.
AutoConfig is configuring the Applications environment...

AutoConfig will consider the custom templates if present.
        Using CONFIG_HOME location     : /u01/TEST/fs2/inst/apps/TEST_uatapp04
        Classpath                   : /u01/TEST/fs2/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/u01/TEST/fs2/EBSapps/comn/java/classes

Version Conflicts among development maintained and customized templates encountered; aborting AutoConfig run.

The logfile for this session is located at: /u01/TEST/fs2/inst/apps/TEST_uatapp04/admin/log/02161113/adconfig.log
[applmgr@uatapp01 ~]$

[applmgr@uatapp01 ~]$ cd $FND_TOP/admin/template/custom/
[applmgr@uatapp01 ~]$ vi $FND_TOP/admin/template/custom/oacore_web_xml_FMW.tmp
[applmgr@uatapp01 ~]$ vi $FND_TOP/admin/template/oacore_web_xml_FMW.tmp
[applmgr@uatapp01 ~]$
[applmgr@uatapp01 ~]$ ls -ltrh $FND_TOP/admin/template/custom/oacore_web_xml_FMW.tmp*
-rwxr-xr-x 1 applmgr oinstall 34K Apr  5  2020 /u01/UAT/fs1/EBSapps/appl/fnd/12.0.0/admin/template/custom/oacore_web_xml_FMW.tmp
[applmgr@uatapp01 ~]$ mv $FND_TOP/admin/template/custom/oacore_web_xml_FMW.tmp $FND_TOP/admin/template/custom/oacore_web_xml_FMW.tmp_bkp_old_adtxk
[applmgr@uatapp01 ~]$ ls -ltrh $FND_TOP/admin/template/oacore_web_xml_FMW.tmp
-rwxr-xr-x 1 applmgr oinstall 33K Feb  7 16:05 /u01/UAT/fs1/EBSapps/appl/fnd/12.0.0/admin/template/oacore_web_xml_FMW.tmp
[applmgr@uatapp01 ~]$ cp $FND_TOP/admin/template/oacore_web_xml_FMW.tmp $FND_TOP/admin/template/custom
[applmgr@uatapp01 ~]$ grep OAF -B 52 
$FND_TOP/admin/template/custom/oacore_web_xml_FMW.tmp_bkp_new_adtxk
[applmgr@uatapp01 ~]$ vi $FND_TOP/admin/template/oacore_web_xml_FMW.tmp

Now re-run autoconfig again on Apps Tier:
[applmgr@uatapp01 ~]$ adautocfg.sh
AutoConfig completed successfully.
[applmgr@uatapp01 custom]$

----------------------------------------- 
17 Run the Online Patching Database Compliance Checker report to check for online patching database objects standards violations: (as advised in ad/txk-14 patching doc)
Run after adtxk cleanup phase completes
[applmgr@uatapp01 ~]$ time sqlplus apps @$AD_TOP/sql/ADZDDBCC.sql
real    38m1.694s
user    0m0.937s
sys     0m0.866s
[applmgr@uatapp01 ~]$

Upgraded AD & TXK version:
SQL> col CODELEVEL for a30;
select ABBREVIATION,CODELEVEL from apps.ad_trackable_entities where abbreviation in('ad','txk');
ABBREVIATION                   CODELEVEL
------------------------------ ------------------------------
ad                             C.13
txk                            C.13
SQL>

After performing AD/TXK 13 Patches now follow steps from below doc for System Schema Migration
Oracle E-Business Suite Release 12.2 System Schema Migration (Doc ID 2755875.1)
High level steps are below:
- EBS System Schema Migration Consolidated Patch
3.5 Apply EBS System Schema Migration Completion Patch
-step2-- adop phase=cleanup cleanup_mode=full
-step3-- Run the Oracle E-Business Suite System Schema Migration Compliance Checker report, ADSYSCC.sql
-step5-- EBS System Schema Migration Completion Patch
adop phase=apply patches=32573930 apply_mode=hotpatch options=forceapply,nocopyportion,nogenerateportion
-step6-- Run the adsysmigclnup.sql cleanup script on the database tier by performing substeps a to d below. This will drop AD objects from the SYS schema and revoke the associated privileges.
- $ sqlplus /nolog @adsysmigclnup.sql APPS

#############################################################################
#############################################################################
#############################################################################
#############################################################################
Known Issues and troubleshooting during AD/TXK-14 Upgrade.


Issue 1.1 AD-14 patch failing at executing script ad_objects.sql on worker 1:
ERROR at line 1:
ORA-20000: ORA-01720: grant option does not exist for 'SYS.DBA_OBJECTS_AE':
Error in creating ad_objects view
ORA-06512: at line 50
Solution given by oracle is 
Please give explicit grants as below and test.

grant all on SYS.DBA_OBJECTS_AE to apps with grant option;
grant all on SYS.DBA_OBJECTS_AE to EBS_SYSTEM with grant option;

Issue 1.2: while patching AD-14 along with 41. critical patch and language patch on current ad-9 version
$ adop phase=apply patches=33600809,34668508,34669333,33600809_AR:u33600809.drv merge=yes apply_mode=downtime    ---->>> for ad/tx 14 (approx. 1hr 30 mins after fixing issue)

as per log ---> find $ADOP_LOG_HOME/28 -iname u_ad_*.log
FAILED: file ademusr.sql  on worker  1 for product ad  username APPLSYS.

as per log ---> find $ADOP_LOG_HOME/28 -iname adwork001.log
Connected.
declare
*
ERROR at line 1:
ORA-01031: insufficient privileges
ORA-06512: at line 168
ORA-06512: at line 257  <<<<<<<<<<<<<<<<<<<<<<<


Solution given by oracle is 
It will tell you the exact grant which is failing in script ademusr.sql.

SQL> set echo on;
SQL> spool ademusr_fail.log
SQL> @$AD_TOP/patch/115/sql/ademusr.sql
SQL> connect SYSTEM/&&1
Enter value for 1: manager
Enter value for 2: applsys
Enter value for 4: apps
SQL> connect &&4/&&5
Enter value for 5: apps_pswd

SQL> spool off;
[applmgr@testapp ~]$ grep 257 ademusr_fail.log
257    give_grants_on_apps_obj('SELECT', 'WF_ERROR', 'EM_OAM_MONITOR_ROLE');



Get ORA-01031: Insufficient Privileges Running Ademusr.Sql On WF_ERROR Table (Doc ID 437760.1)
To implement the solution, please execute the following steps:
1. Please run :
    Sqlplus applsys/...
     GRANT select on WF_ERROR to APPS with grant option;
2. Restart the job failed.


Issue 2: while patching TXK-14 on current txk-9 version
adop phase=apply patches=33602997,34708635,34654260,33602997_AR:u33602997.drv merge=yes apply_mode=downtime    ---->>> for ad/tx 14 (approx 1hr 30 mins after fixing issue)

as per log ---> find $ADOP_LOG_HOME/28 -iname u_*.log
sqlplus -s XXXSHR/***** @/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adsqlwrapper.sql '/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adaddls.pls ***** XXXSHR *****'
sqlplus -s XXXHR/***** @/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adsqlwrapper.sql '/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adaddls.pls ***** XXXHR *****'
sqlplus -s XXFIN/***** @/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adsqlwrapper.sql '/u01/UAT/fs2/EBSapps/appl/ad/12.0.0/patch/115/sql/adaddls.pls ***** XXFIN *****'
ERROR:
ORA-01017: invalid username/password; logon denied


R12.TXK.C.Delta.8 Patch Failing For Instance With Custom Schema With Error 'ORA-01017: invalid username/password; logon denied' (Doc ID 2196359.1)
Adsplice Fails Adding Custom Schemas with error "ORA-01017 invalid username/password" (Doc ID 1935085.1)
select * from apps.fnd_application where application_short_name like '%XX%'

FNDCPASS apps/scapps1 0 Y system/manager ORACLE XXXSHR XXXSHR
FNDCPASS apps/scapps1 0 Y system/manager ORACLE XXXHR XXXHR
FNDCPASS apps/scapps1 0 Y system/manager ORACLE XXFIN XXFIN




Thanks for Reading


Regards,
Mohammed Areefuddin.

No comments:

Post a Comment