Background
This post documents a real production incident where an Oracle E-Business Suite (EBS) environment became completely unresponsive due to a full Fast Recovery Area (FRA). Users were reporting that everything was either slow or hanging — a classic sign that something fundamental had broken at the database layer.
Symptoms
- EBS application users reporting sessions hanging or extremely slow response
sqlplus apps/<password>failing with:
ORA-00257: Archiver error. Connect AS SYSDBA only until resolved.- Non-SYSDBA connections completely blocked
- 30 FNDLIBR (Concurrent Manager) processes running on the application server — higher than expected for a QA environment
Environment
- Oracle Database 19c (CDB/PDB architecture)
- PDB: Application database PDB
- EBS 12.2 on AIX
- FRA configured on ASM diskgroup (+RECO), size 4095 GB
- No explicit
log_archive_dest— archivelogs defaulting to FRA
Step 1 — Identify the Root Cause
Connected to the CDB as SYSDBA and confirmed the error:
SHOW PARAMETER db_recovery_file_dest;NAME TYPE VALUE
--------------------------- ----------- ------
db_recovery_file_dest string +RECO
db_recovery_file_dest_size big integer 4095GChecked FRA usage:
COLUMN name FORMAT A10
COLUMN limit_gb FORMAT 999,999.99 HEADING 'LIMIT GB'
COLUMN used_gb FORMAT 999,999.99 HEADING 'USED GB'
COLUMN reclaimable_gb FORMAT 999,999.99 HEADING 'RECLAIMABLE GB'
COLUMN number_of_files FORMAT 99999 HEADING 'FILES'
SELECT name,
ROUND(space_limit/1024/1024/1024,2) limit_gb,
ROUND(space_used/1024/1024/1024,2) used_gb,
ROUND(space_reclaimable/1024/1024/1024,2) reclaimable_gb,
number_of_files
FROM v$recovery_file_dest;Output:
NAME LIMIT GB USED GB RECLAIMABLE GB FILES
---------- ----------- ----------- -------------- ------
+RECO 4,095.00 4,075.65 .00 2426FRA was at 99.5% — 4,075 GB used out of 4,095 GB, zero reclaimable, 2,426 archivelog files.
This was the root cause. With the FRA full and nothing reclaimable, the archiver (ARCn) process could not write new archive logs, blocking all database activity.
Step 2 — Assess Backup Status Before Taking Action
Before deleting any archivelogs, it is critical to understand the backup posture. Blindly deleting archivelogs without knowing the backup state can leave the database unrecoverable.
Started an RMAN session and ran a crosscheck first:
rman target /RMAN> crosscheck archivelog all;All 2,416 objects returned “validation succeeded” — no expired logs. This meant the FRA was genuinely full of valid, undeleted archivelogs.
Next, checked backup history:
RMAN> list backup summary;Backups were listed daily going back several weeks — however, on closer inspection:
SELECT session_key,
TO_CHAR(start_time,'DD-MON-YY HH24:MI') start_time,
TO_CHAR(end_time,'DD-MON-YY HH24:MI') end_time,
status,
input_bytes,
output_bytes
FROM v$rman_backup_job_details
ORDER BY start_time DESC
FETCH FIRST 10 ROWS ONLY;Output revealed a critical finding:
SESSION_KEY START_TIME END_TIME STATUS INPUT_BYTES OUTPUT_BYTES
----------- ----------------- ----------------- -------- ----------- ------------
19986 30-APR-26 01:00 30-APR-26 01:00 COMPLETED 0 98304
19904 29-APR-26 01:00 29-APR-26 01:00 COMPLETED 0 98304INPUT_BYTES = 0— no datafiles were being backed upOUTPUT_BYTES = 98304(96 KB) — only a controlfile/SPFILE autobackup- Jobs completing in under a minute — impossible for a real full DB backup
Additionally:
RMAN> list backup of archivelog all;
-- specification does not match any backup in the repositoryNo archivelog backups existed at all. The scheduled backup job was not performing proper datafile or archivelog backups — a separate issue to be addressed post-incident.
Step 3 — Escalation and Approval
Given that there were no archivelog backups and the datafile backup was questionable, the decision to delete archivelogs and Approval was obtained with the instruction to start conservatively — delete older than 15 days first, then 7 days if needed.
Step 4 — Resolution
First attempt — delete older than 15 days:
RMAN> delete noprompt archivelog until time 'SYSDATE-15';RMAN returned warnings:
RMAN-08138: warning: archived log not deleted - must create more backupsThis is because the RMAN retention policy was blocking deletion of logs that had never been backed up. With Tier 3 approval, used the force option to override:
RMAN> delete noprompt force archivelog until time 'SYSDATE-15';This ran successfully, deleting archivelogs from January through late April.
FRA usage after deletion:
USED_GB RECLAIMABLE_GB NUMBER_OF_FILES
------- -------------- ---------------
246.71 0 149Over 3,800 GB freed — FRA dropped from 99.5% to ~6%.
Step 5 — Verification
Forced a log switch and verified archiver status:
ALTER SYSTEM ARCHIVE LOG CURRENT;
SELECT dest_id, status, error
FROM v$archive_dest
WHERE status != 'INACTIVE';Output:
DEST_ID STATUS ERROR
------- -------- -----
1 VALIDArchiver resumed successfully. Application connectivity was restored and users confirmed sessions were working normally.
Redo Log Status Check
As part of the verification, also confirmed redo log health:
SELECT thread#,
sequence#,
TO_CHAR(first_time,'DD-MON-YY HH24:MI:SS') first_time,
first_change#,
archived,
status
FROM v$log
ORDER BY thread#, sequence#;THREAD# SEQUENCE# FIRST_TIME FIRST_CHANGE# ARC STATUS
------- --------- ------------------------ ------------- --- -------
1 2417 05-MAY-26 01:56:44 1.7867E+10 NO INACTIVE
1 2418 05-MAY-26 03:27:20 1.7867E+10 NO INACTIVE
1 2419 05-MAY-26 07:57:10 1.7868E+10 NO CURRENTNo stuck or unarchived redo logs — database in healthy state.
Incident Summary
| Item | Detail |
|---|---|
| Error | ORA-00257: Archiver error |
| Root Cause | FRA (+RECO ASM diskgroup) at 99.5% capacity |
| FRA Before | 4,075 GB used / 2,426 files |
| FRA After | 246 GB used / 149 files |
| Action Taken | delete noprompt force archivelog until time 'SYSDATE-15' |
| Space Freed | ~3,829 GB |
| Resolution Time | ~45 minutes from identification to restoration |
Key Lessons Learned
1. Monitor FRA Proactively
Set up OEM 13c threshold alerts on FRA usage at 70% and 85%. Do not wait for ORA-00257 to discover the problem.
SELECT ROUND(space_used/space_limit*100,2) pct_used
FROM v$recovery_file_dest;Alert when this exceeds 80%.
2. Always Check Backup Status Before Deleting Archivelogs
In this incident, the backup job appeared to be running daily but was only backing up the controlfile (INPUT_BYTES=0). This is a serious gap — verify actual backup content, not just job status.
3. Investigate the Backup Job
A proper RMAN backup script for a 19c CDB should include:
BACKUP DATABASE PLUS ARCHIVELOG DELETE INPUT;4. Consider an Archivelog Deletion Policy
If archivelog backups are not being taken, configure an RMAN retention policy to prevent FRA accumulation:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;5. FRA Sizing Review
Even at 4095 GB, the FRA filled up due to months of uncleaned archivelogs. Review archivelog generation rate and size FRA accordingly, or implement regular cleanup.
Conclusion
ORA-00257 is one of those errors that brings an entire EBS environment to its knees instantly. The fix itself is straightforward — free up FRA space — but the investigation matters. Rushing to delete archivelogs without understanding the backup posture can result in an unrecoverable database.
In this case, careful investigation revealed a deeper issue with the backup job that would have gone unnoticed had we not looked. Always verify, always escalate, and always get approval before deleting recovery-critical files.
Syed Anwar Ahmed is an Oracle Apps DBA with over 11 years of production experience across Oracle EBS, Database, RAC, GoldenGate, and OEM environments. He writes about real-world Oracle incidents at syedanwarahmedoracle.blog.
