ORA-16572 DMON: cannot open configuration file

Hi,

This can be a interesting read.

Recently we created a Data Guard Configuration on a Non-RAC Database using cooked file system. We pro-actively started monitoring the DMON logs and the alert logs so that we can trace all the steps during this setup.

Issue:

We were getting the error in the DMON log when we set the below parameters.

alter system set dg_broker_config_file1='/oracle/oradata/CDEV/drCDEV_PRIMARY_01.dat' scope=both;

alter system set dg_broker_config_file2='/oracle/oradata/CDEV/drCDEV_PRIMARY_02.dat' scope=both;

alter system dg_broker_start=true;

Error in the DMON log was as below

DMON: >> Starting Data Guard Broker bootstrap <<

DMON: Attach state object

DMON: cannot open configuration file "/oracle/oradata/CDEV/drCDEV_PRIMARY_01.dat", retrying

DMON: cannot open configuration file "/oracle/oradata/CDEV/drCDEV_PRIMARY_01.dat"

ORA-27037: unable to obtain file status

Linux-x86_64 Error: 2: No such file or directory

Additional information: 3

DMON: Error opening /oracle/oradata/CDEV/drCDEV_PRIMARY_01.dat, error = ORA-16572

DMON: Establishing /oracle/oradata/CDEV/drCDEV_PRIMARY_02.dat as the most current file

DMON: cannot open configuration file "/oracle/oradata/CDEV/drCDEV_PRIMARY_02.dat", retrying

DMON: cannot open configuration file "/oracle/oradata/CDEV/drCDEV_PRIMARY_02.dat.dat"

ORA-27037: unable to obtain file status

Linux-x86_64 Error: 2: No such file or directory

Additional information: 3

We got this error even on the standby when we set the above parameters

Troubleshooting

From the error it was obvious that DMON was looking for the dg_broker_config file and was unable to find at the physical level.

We were sure that this file was to be created Oracle as this was not a database using the raw volumes or block devices.

Hence we checked for permission on the location and also created a file, etc., but found that everything was fine.

We are tried changing the location of the files to various mount points just to check if it was issue at the mount point level but we got the same error.

Observation

We then decided to go ahead and create the configuration using DGMGRL

On Primary database

DGMGRL> CREATE CONFIGURATION 'DR_CDEV' AS

PRIMARY DATABASE IS 'CDEV_PRIMARY'

CONNECT IDENTIFIER IS CDEV_PRIMARY_DGMGRL;

Once this command was executed we found that the dg_broker_files got created on the Primary database.

But still the files were missing on the Standby database. Encouraged by the previous finding we went ahead and added the Standby database to the configuration and when we enabled the configuration we saw that the db_broker_files got created on the Standby as well.

On Primary Database

DGMGRL> enable configuration

Regards,

Tanveer

Lesson Learnt: PRO-ACTIVENESS sometimes can be DISASTROUS :)