Tag: ORA-01274

  • Oracle 19c Data Guard: Recovering UNNAMED Datafiles on a Multitenant Physical Standby

    Alhamdulillah.

    Oracle Data Guard physical standbys keep redo synchronized automatically — until the primary adds a datafile that the standby can’t place on disk. When STANDBY_FILE_MANAGEMENT=MANUAL, the standby refuses to guess where the new file should live. Managed Standby Recovery stops instead, leaving the control file ahead of the actual datafiles and the standby parked at a consistent SCN.

    This post walks through a real 19c multitenant incident: MRP0 hit ORA-01274 after a new datafile couldn’t be added, recovery stalled, and the fix required creating the missing file inside the correct pluggable database container, then resuming and verifying redo apply. In this incident, the missing datafile belonged to PROD. Creating the file from within the owning PDB resolved the placeholder, after which Redo Apply could be restarted from CDB$ROOT.

    Failure Diagnosis

    The alert log during redo apply:
    Recovery of Online Redo Log: Thread 1 Group 4 Seq 105902 Reading mem 0
    Mem# 0: /uora/PROD/db/product/19.0.0/dbs/broken3
    2025-05-12T08:49:56.027319-04:00
    PROD(3):File #433 added to control file as ‘UNNAMED00433’ because
    PROD(3):the parameter STANDBY_FILE_MANAGEMENT is set to MANUAL
    PROD(3):The file should be manually created to continue.
    PR00 (PID:2796228): MRP0: Background Media Recovery terminated with error 1274
    2025-05-12T08:49:56.092976-04:00
    Errors in file /uora/PROD/db/diag/rdbms/proddr/cdbPROD/trace/cdbPROD_pr00_2796228.trc:
    ORA-01274: cannot add data file that was originally created as
    ‘/uora/PROD/data/proddata/users02.dbf’
    PR00 (PID:2796228): Managed Standby Recovery not using Real Time Apply
    Recovery interrupted!
    2025-05-12T08:49:57.933554-04:00
    Recovery stopped due to failure in applying recovery marker (opcode 17.30).
    Datafiles are recovered to a consistent state at change 36278361292
    but controlfile could be ahead of datafiles.
    Stopping change tracking


    Three things happening in sequence:

    1. **Control file registered a file that doesn’t exist.** The redo stream carried a datafile creation from the primary. The standby’s control file logged it — but because STANDBY_FILE_MANAGEMENT=MANUAL, the standby recorded it as UNNAMED00433 instead of creating it.

    2. **Recovery stopped on the recovery marker.** Opcode 17.30 is the file operation that couldn’t be satisfied. The standby parked at SCN 36278361292 — a point where datafiles are internally consistent, but the control file believes file 433 exists when it doesn’t.

    3. **MRP0 exited.** The managed standby process shut down cleanly. The alert log shows “not using Real Time Apply,” indicating recovery was interrupted and the standby could not continue applying the current redo stream.

    **Root cause:** Configuration, not corruption. STANDBY_FILE_MANAGEMENT=MANUAL means every new primary datafile must be created manually on the standby. With AUTO and proper DB_FILE_NAME_CONVERT / PDB_FILE_NAME_CONVERT setup, new files appear automatically. This environment chose MANUAL, so a routine tablespace addition on the primary became a recovery outage on the standby.

    ## Recovery Workflow

    MRP0 stops → Query UNNAMED files → Switch to PDB → Create datafile → Resume MRP → Verify

    ## Step 1 — Locate the UNNAMED File

    Confirm the standby’s role and find the placeholder. Run from CDB$ROOT:


    SET ECHO OFF PAGESIZE 60 LINESIZE 120 TRIMSPOOL ON
    COL NAME FORMAT A50
    SQL> SELECT name, open_mode, database_role FROM v$database;
    NAME OPEN_MODE DATABASE_ROLE
    ——— ——————– —————-
    CDBSTBY MOUNTED PHYSICAL STANDBY
    SQL> SELECT FILE#, NAME FROM V$DATAFILE WHERE NAME LIKE ‘%UNNAMED%’;
    FILE# NAME
    ———- ————————————————–
    433 /u01/app/oracle/product/19.0.0/dbhome_1/dbs/UNNAMED00433


    File number is 433. Which PDB owns it? Use CON_ID:


    COL CON_ID FORMAT 9999
    COL FILE# FORMAT 9999
    COL NAME FORMAT A50
    SQL> SELECT con_id, file#, name FROM V$DATAFILE WHERE file# = 433;
    CON_ID FILE# NAME
    ———- —— ————————————————–
    3 433 /u01/app/oracle/product/19.0.0/dbhome_1/dbs/UNNAMED00433


    CON_ID = 3 is PROD. That’s where the fix must run.

    ## Step 2 — PDB Recovery: Create the File in the Right Container

    From CDB$ROOT, the command appeared to succeed:


    SQL> ALTER DATABASE CREATE DATAFILE ‘UNNAMED00433’
    AS ‘/uora/PROD/data/proddrdata/users02.dbf’;
    Database altered.
    “`

    But checking from the PDB showed the placeholder still registered:


    SQL> ALTER SESSION SET CONTAINER = PROD;
    Session altered.
    SQL> SELECT FILE#, NAME FROM V$DATAFILE WHERE NAME LIKE ‘%UNNAMED%’;
    FILE# NAME
    ———- ————————————————–
    433 /uora/PROD/db/product/19.0.0/dbs/UNNAMED00433


    After switching to the PROD container and reissuing the command:


    SQL> ALTER DATABASE CREATE DATAFILE ‘UNNAMED00433’
    AS ‘/uora/PROD/data/proddrdata/users02.dbf’;
    Database altered.
    SQL> SELECT FILE#, NAME FROM V$DATAFILE WHERE NAME LIKE ‘%UNNAMED%’;
    no rows selected


    **Important:** Validate this behavior against Oracle 19c documentation and test in non-production before applying to your standby.

    ## Step 3 — Redo Apply Resumption


    SQL> ALTER SESSION SET CONTAINER = CDB$ROOT;
    Session altered.
    SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
    USING CURRENT LOGFILE DISCONNECT FROM SESSION;
    Database altered.
    SQL> SELECT PROCESS, STATUS, THREAD#, SEQUENCE# FROM V$MANAGED_STANDBY;
    PROCESS STATUS THREAD# SEQUENCE#
    ———- ————— ——- ———
    MRP0 APPLYING_LOG 1 105902
    ARCH CLOSING 1 105905
    RFS IDLE 1 105906


    MRP0 is back in APPLYING_LOG.

    ## Step 4 — Expect More: The Second UNNAMED File

    Recovery resumed and encountered another placeholder:

    “`sql
    SQL> ALTER SESSION SET CONTAINER = PROD;
    SQL> SELECT FILE#, NAME FROM V$DATAFILE WHERE NAME LIKE ‘%UNNAMED%’;
    FILE# NAME
    ———- ————————————————–
    435 /uora/PROD/db/product/19.0.0/dbs/UNNAMED00435
    SQL> ALTER DATABASE CREATE DATAFILE ‘UNNAMED00435’
    AS ‘/uora/PROD/data/proddrdata/transaction_table_16.dbf’;
    Database altered.


    Expect a loop. Every datafile added on the primary after the gap will appear as another UNNAMED entry. Grep the alert log for the full backlog first:


    grep -i ‘unnamed’ alert_cdbstby.log | head -20
    grep ‘ORA-01274’ alert_cdbstby.log | wc -l


    ## Recovery Verification

    Once placeholders are gone, resume recovery:


    ALTER SESSION SET CONTAINER = CDB$ROOT;
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
    USING CURRENT LOGFILE DISCONNECT FROM SESSION;
    “`

    Verify:


    SELECT con_id, file#, name FROM V$DATAFILE WHERE name LIKE ‘%UNNAMED%’;
    SELECT process, status, thread#, sequence# FROM V$MANAGED_STANDBY WHERE process LIKE ‘MRP%’;
    SELECT * FROM V$DATAGUARD_STATS;


    Clean results: zero UNNAMED rows, MRP0 in APPLYING_LOG, apply lag returning to normal.

    ## Alternative: Restore from Backup

    If redo is unavailable or purged:

    1. Identify primary backup with missing file
    2. Restore to standby location
    3. Register with control file via RMAN
    4. Resume recovery

    Follow Oracle RMAN and Data Guard documentation for your configuration.

    ## Lessons Learned

    **Decide STANDBY_FILE_MANAGEMENT deliberately.** MANUAL works for locked-down environments. AUTO prevents this class of manual intervention when file naming and storage support it. ما شاء الله — the infrastructure, when well-designed, removes recovery work.

    **Session container context can affect recovery steps.** In this incident, CREATE DATAFILE from CDB$ROOT appeared to succeed but didn’t resolve the placeholder until run from PROD PDB. Validate against Oracle 19c documentation and test in non-production.

    **Expect a loop.** Every datafile added after the gap arrives as another UNNAMED entry. Grep the alert log for full backlog first.

    **Verify before closing.** Confirm zero UNNAMED files, MRP0 running, settled lag before closing incident.

    ## Conclusion

    ORA-01274 on a physical standby is a configuration boundary, not data loss. This incident resolved via CREATE DATAFILE in the PROD PDB, followed by managed recovery restart from CDB$ROOT. Before applying: Verify steps against Oracle 19c documentation, test in non-production, verify standby status and lag.