Not able to create new database connection: FND^@SECURITY_APPL_LOGIN_FAILED^@ when starting Concurrent Managers

Hi,

Recently we faced a issue after cloning R12 environment

Issue Description:

-----------------

The concurrent managers were not coming up after cloning

Workflow Agent Listener Service

Workflow Mailer Service

Workflow Document Web Service

Email Center Download Processor

Output Post Processor

When checking the manager logs for each of the managers I found the below error

Not able to create new database connection: FND^@SECURITY_APPL_LOGIN_FAILED^@

Troubleshooting:

---------------

Quick list of managers which were not started showed us that the mangers using the
framework were the ones affected.

Error pointed issue with the GUEST user.

Hence we checked the GUEST password in the dbc file $FND_SECURE/CDEV.dbc on CM node
and found the below entry

GUEST_USER_PWD=GUEST/CPRD

Also when we checked the context file $CONTEXT_FILE on CM node we found the parameter
s_guest_pass set to CPRD

Now we wanted to validate GUEST password at database base level

1. Conn to database as apps

2. Find the password at database level

SQL> select fnd_profile.value('GUEST_USER_PWD') from dual;

FND_PROFILE.VALUE('GUEST_USER_PWD')

——————————————————————————

GUEST/CDEV

Here we found that in the database the password for GUEST was different

3. Then we checked if the guest user was working fine with the password at database level

SQL> select FND_WEB_SEC.VALIDATE_LOGIN('GUEST','CDEV') from dual;

FND_WEB_SEC.VALIDATE_LOGIN('GUEST','CDEV')

——————————————————————————–-----

Y

4. We also found that mismatch in database and the CM node was because, as this is a multi
node architecture the autoconfig was not run last on the CM node but some other node which
had the value for s_guest_pass as CDEV. Hence we inferred that the database was updated
with the password for GUEST user with the value of s_guest_pass in the context file of the
node on which the autoconfig was run last

Solution:

--------

We can resolve this issue in two ways

Method 1

-----------

The traditional way of changing the GUEST password is to edit the parameter s_guest_pass in
the context file and running autoconfig

In our case as only the context file on the CM node had the wrong entry, we just need to make
the change in the context file on CM node and run autoconfig to fix the issue.

If you have more nodes where the context file is having the wrong password, then you need to correct the entry and run autoconfig on all those nodes

Method 2

-----------

In this method we can avoid running the autoconfig. We followed this method to fix the issue

i. Open the adgendbc.sh at $INST_TOP/admin/install and change the GUEST password to the
right password

ii. Take backup of the dbc file at location $FND_SECURE

iii. Run adgendbc.sh from location $INST_TOP/admin/install. It will prompt for Apps
username and password

iv. Check the new dbc file generated at location $FND_SECURE and see if the correct GUEST
password is generated

v. Do steps i to iv on all nodes where the dbc file has the wrong GUEST password

vi. Restart the concurrent managers

vii. Make the change in the conext file on the all nodes were the above steps i to iv were done,
so that the next autoconfig will generate the right dbc file

Regards,

Tanveer

ORA-01620 When mounting a standby database

Hi,

Recently we had task to setup a DG for a RAC database.

The primary and the DG were 11g.

After copying the primary database to the DG server, we tried to mount the database on the
DG server before starting the recovery. But we got the below error

SQL> alter database mount standby database;

alter database mount standby database

*

ERROR at line 1:

ORA-01620: no public threads are available for mounting

The alert log had only the below entry and also no trace file was generated for the error.

ORA-1620 signalled during: alter database mount

From the error it was clear that it was issue with the redo log threads. But for building a DG we need not add redo threads at the standby database and also as we were not going to do a DG
Broker setup hence standby redo log file addition was also not necessary. Hence it became sure to us that the issue was not on the standby database side but on the primary database

TROUBLESHOOTING:

On the Primary Database when we ran the below query we found that the threads were
enabled for the RAC instances in PRIVATE mode and not in PUBLIC mode which is needed for the DG to work.

SQL> select thread#,status,enabled from v$thread;

THREAD# STATUS ENABLED

---------- ------ --------

1 OPEN PRIVATE

2 OPEN PRIVATE

3 OPEN PRIVATE

4 OPEN PRIVATE

In the RAC system, each instance has to have its own redo log groups. The redo log file groups of an instance are collectively called a thread, or more appropriately, a redo log thread. The redo threads can be private or public. If the redo thread is public then any instance
can acquire it and if it is private then only a specific instance can acquire it as mentioned in the
initialization file.

SOLUTION:

Inorder to get the DG running, we must convert all the redo threads to PUBLIC in primary
database, create the standby control file, and mount the standby database with this standby
control file.

Below are the steps followed by us, but there are can be deviations.

Note: To enable or disable threads the database has to be in open mode

1. Shut down all instances other than the instance 1

2. Disable the threads as you cannot change the mode from PRIVATE to PUBLIC directly.

SQL> alter database enable public thread 1;

alter database enable public thread 1

*

ERROR at line 1:

ORA-01612: instance TSEP (thread 1) is already enabled

Hence we have disable it and then enable in PUBLIC mode

alter database disable thread 2;

alter database disable thread 3;

alter database disable thread 4;

Also you cannot disable thread 1 at this point, as the instance 1 will be using it

SQL> alter database disable thread 1;

alter database disable thread 1

*

ERROR at line 1:

ORA-01615: instance TSEP (thread 1) is mounted - cannot disable

3. Enable the threads in PUBLIC mode

alter database enable public thread 2;

alter database enable public thread 3;

alter database enable public thread 4;

4. Shut down instance 1 and open the database in instance 2.

Repeat the disable thread and enable thread in PUBLIC mode for instance 1

alter database disable thread 2;

alter database enable public thread 2;

5. Start all the other instances of the RAC including instance 1

6. Create the standby control file again on the primary database

7. Copy the standby control file to the standby database and mount the database

SQL> alter database mount standby database;

Database mounted

8. Start the recovery process

-- Tanveer Madan