Autoconfig failing after reporting error with afcpctx.sh

We were running autoconfig in our R12.1.1/11gR2 environment as part of SSO integration step and we ran into this issue:
[AutoConfig Error Report]The following report lists errors AutoConfig encountered during each phase of its execution. Errors are grouped by directory and phase.The report format is:
[PROFILE PHASE] AutoConfig could not successfully execute the following scripts: Directory: /apps/local/TESIF/inst/apps/TESIF_oa1-ipls-35/admin/install afcpctx.sh INSTE8_PRF 1

Autoconfig failed with the above error.

As I started troubleshooting this issue, I noticed the following error in the autoconfig log:
Uploading Context file /apps/local/TESIF/inst/apps/TESIF_oa1-ipls-35/appl/admin/TESIF_oa1-ipls-35.xml
Context File upload for /apps/local/TESIF/inst/apps/TESIF_oa1-ipls-35/appl/admin/TESIF_oa1-ipls-35.xml failed.

Check log file created by FNDCPUCF program.

SQL*Plus: Release 10.1.0.5.0 - Production on Fri Jul 9 09:37:00 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Enter value for 1: Enter value for 2: Enter value for 3: Connected.
COUNT(*)---------- 14
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Real Application Clusters, OLAP, Data Miningand Real Application Testing options
Uploading Metadata file /apps/TESIF/apps/apps_st/appl/ad/12.0.0/admin/template/adctxinf.tmpERRORCODE = 1 ERRORCODE_ENDLog filename : L470980.log

When I checked the log L470980.log, it had the following error:
Oracle error -6502: ORA-06502: PL/SQL: numeric or value error has been detected in FND_GSM_UTIL.APPEND_CTX_FRAGMENT.

Now, if you look up metalink for the above error, it talks about two possible situations:
- Hostname + Domain Name > 30 characters
- Some bug in 11.5.10 and to apply a patch.

However, in our case, I did not think any of these would apply, as in the past autoconfig was running successfully in the past. So the focus was on recent patches which were applied to the system but one key factor helped me eliminate that possibility as well i.e., the last_ddl_time of FND_GSM_UTIL did not reflect a equal or later timestamp as that of the patches. Also, the $FND_TOP/patch/115/sql/AFCPGUTB.pls did not have the timestamp of the patch application time, which confirmed that no patch changed anything as far as this package body is concerned.

After some intense troubleshooting moments, I zeroed in on the real culprit:
< xxim_top oa_var="s_xximtop" oa_type="PROD_TOP" oa_enabled="FALSE">/apps/TESIF/apps/apps_st/appl/xxim/12.0.0< / xxim_top>
< xxcau_top oa_var="s_xxcautop" oa_type="PROD_TOP" oa_enabled="FALSE">/apps/TESIF/apps/apps_st/appl/xxcau/12.0.0< / xxcau_top>
The above entries were found in our xml which turned out to be the culprits. Actually these custom top entries should be present under $APPL_TOP/XCUSTOM.env (custom environment file) and not in the context file. Got rid of the two lines and the autoconfig works the way we knew it in the past!
- Aravind Kamath Posral