ORA-19909: datafile 1 belongs to an orphan incarnation

ORA-19909: datafile 1 belongs to an orphan incarnation

When I tried to Flashback on in standby database then suddenly below message found in alert log and standby database not able to apply archive-logs also. So, here are messages from standby database alert log:

-- Message from alert log ( in standby database)
ORA-19909: datafile 1 belongs to an orphan incarnation
ORA-01110: data file 1: '/PROD/PROD/xxxx.dbf'
Recovery Slave PR00 previously exited with exception 19909

RCA:
Primary and standby have different incarnations:

-- in standby db

RMAN> list incarnation of database;
using target database control file instead of recovery catalog
List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       PROD  3680942556       PARENT  5982315579513 30-MAY-2019 00:31:04
2       2       PROD  3680942556       PARENT  5982448719535 30-MAY-2019 22:31:15
4       4       PROD  3680942556       ORPHAN  5983354187673 09-JUN-2019 04:01:27
3       3       PROD  3680942556       CURRENT 5983354274164 10-JUN-2019 04:02:53

-- in primary db

RMAN> list incarnation of database;
using target database control file instead of recovery catalog
List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       PROD  3680942556       PARENT  5982315579513 30-MAY-2019 00:31:04
2       2       PROD  3680942556       CURRENT 5982448719535 30-MAY-2019 22:31:15

-- Solution:

You need to reset the standby database's incarnation to match the primary's:

-- in standby

RMAN> list incarnation of database;
using target database control file instead of recovery catalog
List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       PROD  3680942556       PARENT  5982315579513 30-MAY-2019 00:31:04
2       2       PROD  3680942556       PARENT  5982448719535 30-MAY-2019 22:31:15
4       4       PROD  3680942556       ORPHAN  5983354187673 09-JUN-2019 04:01:27
3       3       PROD  3680942556       CURRENT 5983354274164 10-JUN-2019 04:02:53

RMAN> reset database  to incarnation 2;
database reset to incarnation 2

RMAN> list incarnation of database;
List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       PROD  3680942556       PARENT  5982315579513 30-MAY-2019 00:31:04
2       2       PROD  3680942556       CURRENT 5982448719535 30-MAY-2019 22:31:15
4       4       PROD  3680942556       ORPHAN  5983354187673 09-JUN-2019 04:01:27
3       3       PROD  3680942556       ORPHAN  5983354274164 10-JUN-2019 04:02:53

-- Verify MRP is active or not

$ ps -ef|grep mrp
oracle   23358 21699  0 12:56 pts/1    00:00:00 grep mrp
-- Verify standby database status and role
SQL> select name,open_mode ,database_role from v$database;
NAME      OPEN_MODE            DATABASE_ROLE
--------- -------------------- ----------------
PROD      MOUNTED              PHYSICAL STANDBY

-- now start MRP in standby database

$ !sql
SQL> alter database recover managed standby database disconnect from session;
Database altered.

-- Verify the MRP process
$ ps -ef|grep mrp
oracle   23455     1  0 12:58 ?        00:00:00 ora_mrp0_PROD
oracle   24125 21699  0 13:11 pts/1    00:00:00 grep mrp

Now archive-logs are applying.