Applies to:                                                                                                                                                            

Oracle Server - Enterprise Edition - Version 7.0.16.0 to 10.2.0.4 [Release 7.0 to 10.2] Oracle Server - Standard Edition - Version 7.0.16.0 to 10.2.0.4 [Release 7.0 to 10.2] Information in this document applies to any platform.

Purpose                                                                                                                                                  

Basic Steps to be Followed While Solving ORA-00600 [4194]' and provide step by step instructions for resolving such ORA-600 [4194] errors under a variety of conditions.

Troubleshooting  Steps                                                                                                                                                       

This error indicates that a mismatch has been detected between redo records and rollback (undo) records. ARGUMENTS:

Arg [a] - Maximum Undo record number in Undo block Arg [b]  -  Undo record number from Redo block

Since we are adding a new undo record to our undo block, we would expect that the new record number is equal to the maximum record number in the undo block plus one. Before Oracle can add a new undo record to the undo block it validates that this is correct. If this validation fails, then an ORA-600 [4194] will be triggered.

Oracle is unable to roll back an uncommitted transaction in its transaction table. You probably have corruption in either the rollback/undo segment or corruption in the object in which the rollback/undo segment is trying to apply the undo record.

Refer to Note: 39283.1 for more details on the description of this error.

SOLUTION

A.    Bouncing the DB General:

In general terms these are the steps to resolve most ORA-600 [4194] errors:

1.     Determine whether the DB is up or down.

2.     Check for active transactions.

3.     If there are no active transactions then drop the problem undo segment or create a new UNDO tablespace and delete the old one.

4.     If there are active transactions, then you will need to identify the object the undo needs to be applied to and drop and recreate that object.

5.     If all else fails, then perform a point in time recovery to a point prior to the corruption.

Note:

Prior to starting the restore/recovery you should always ask your client to take a full back up in case the problem worsens as a result of this action.

Detailed:

These are more detailed steps: Step 1

Confirm whether the database is up and running or not. If the database fails to start or crashes shortly after startup due to this error occurring, then try setting event 10513 at level 2 in the init.ora/spfile to disable transaction recovery and restart the instance, e.g.:

event = "10513 trace name context forever, level 2"

This may allow the database to successfully open and stay up so that the required diagnostics/actions can be performed.

Step 2

In the trace file there should be an undo segment header dump, and so check to see if the undo segment header shows an active transaction after recovery, e.g.:

TRN TBL    <---- Represents the Transaction table for the particular undo segment index state cflags wrap# uel scn dba

---------------------------------------------------------------------------------------------

0x41 9 0x80 0x35ab6 0xffff 0x0695.38f6b959 0x1081e796 0x42 9 0x80 0x35bb1 0x000e 0x0695.38f6b028 0x1081e793

0x43 9 0x80 0x35b11 0x005d 0x0695.38f6b7ae 0x1081e795

0x44 9 0x80 0x359f0 0x0036 0x0695.38f69a91 0x1081e78e

0x45 10 0x80 0x35b1b 0x0000 0x0695.3a0aba4d 0x1081e796

0x46 9 0x80 0x35bb7 0x001c 0x0695.38f69bde 0x1081e78f

===================================

State ---> This column specifies the status of the transaction 9 -----> represents a commited transaction

10 ---> Represents a active transaction Dba -----> Undo block containing the undo records

Strictly speaking this is the block at the end of the undo chain.

You can see from the transaction table that there is an active transaction for this particular rollback/undo segment after recovery. Therefore this rollback/undo segment and/or undo tablespace cannot be dropped without corrupting the database! Therefore recreating the UNDO tablespace is not an option.

Step 3

From the trace file determine the affected undo segment, e.g.: Block image after block recovery:

UNDO BLK:

xid: 0x0015.02b.0001544b seq: 0x163e cnt: 0x12 irb: 0x12 icl: 0x0 flg: 0x0000

XID ==> Undo segment no + Slot no + Sequence no Therefore, in this case the Undo Segment is:

USN# 0x15 (Hex) ==> 21 (Dec)  ==> _SYSSMU21$

So if and ONLY IF the transaction table shows no active transaction can the rollback/undo segment be offlined and dropped. Note however, that before you can confirm if the entire UNDO tablespace can be dropped, you would need to check the transaction tables of ALL active rollback/undo segments in the same wasy as the

above. The steps required to drop the rollback/undo segment are fully detailed in Note:179952.1, but are briefly listed here for completeness:

If using Automatic Undo Management

Offline the undo segment using the _OFFLINE_ROLLBACK_SEGMENTS parameter and bounce the database as follows:

1.     Create and edit the init.ora file for the instance to set the following parameters: UNDO_MANAGEMENT=MANUAL

_OFFLINE_ROLLBACK_SEGMENTS=(_SYSSMU21$)

2.     Open the database in restricted mode to prevent user access, e.g.: connect / as sysdba

startup restrict pfile = '<Full path to init.ora file>';

3.     Drop the rollback/undo segment, e.g.: drop rollback segment "_SYSSMU21";

4.     Shutdown the instance, and remove the init.ora parameters added in point 1 and restart the instance, e.g.:

shutdown immediate startup

If SMON was recovering the transaction then this may not work as we cannot open the database  if it is determined to be in an inconsistent state. I have reviewed a number of SRs where  this approach was successful, so it is important to try it first but understand that it may fail and you will have to resort to a point in time recovery or forcing open the DB and recreating it.

Step 4

Now we need to dump the undo block to see which object was affected. We noted in Step 2 that this is the active transaction (from the trace file):

TRN TBL

index state cflags wrap# uel scn dba

0x45 10 0x80 0x35b1b 0x0000 0x0695.3a0aba4d 0x1081e796

Dba----------------> Undo block containing the undo records

dba--->0x1081e796 is the block containing the active transaction .

Use the WebIV tools to convert this RDBA to block number (block#) and file number (file#), e.g.:

V SPLIT ==> DBA (Hex) = File#,Block# (Hex File#,Block#)

= ===== === ===== ============

V8 10,10 ==> 276948886 (0x1081e796) = 66,124822 (0x42 0x1e796)

So the file# is 66 and the block# is 124822, so dump the block by issuing: SQL> Alter system dump datafile 66 block 124822;

This will generate a trace file in the user_dump_dest. The following is a sample of the information in the undo block:

UNDO BLK:

xid: 0x000c.045.00035b1b seq: 0x1e14 cnt: 0x17 irb: 0x17 icl: 0x0 flg: 0x0000

Rec Offset Rec Offset Rec Offset Rec Offset Rec Offset

---------------------------------------------------------------------------

0x01 0x1f8c 0x02 0x1f30 0x03 0x1ed4 0x04 0x1e78 0x05 0x1e1c 0x06 0x1dc0 0x07 0x1d64 0x08 0x1d08 0x09 0x1cac 0x0a 0x1c50 0x0b 0x1bf4 0x0c 0x1b98 0x0d 0x1b3c 0x0e 0x1ae0 0x0f 0x1a74 0x10 0x1a18 0x11 0x19bc 0x12 0x1960 0x13 0x1904 0x14 0x187c

0x15 0x181c 0x16 0x1798 0x17 0x173c

* Rec #0x16 slt: 0x45 objn: 1485619(0x0016ab33) objd: 1485619 tblspc:  71(0x00000047)

* Layer: 11 (Row) opc: 1 rci 0x00

Undo type: Regular undo Begin trans Last buffer split: No Temp Object: No

Tablespace Undo: No rdba: 0x00000000

*-----------------------------

uba: 0x1081e796.1e14.14 ctl max scn: 0x0695.38f69853 prv tx scn: 0x0695.38f698a1 KDO undo record:

KTB Redo

op: 0x04 ver: 0x01

op: L itl: scn: 0x0019.009.00034237 uba: 0x36c0cce4.1d2f.19 flg: C--- lkc: 0 scn: 0x0695.38f6b96b

KDO Op code: URP xtype: XA bdba: 0x35406893 hdba: 0x35406892 itli: 1 ispac: 0 maxfr: 4863

tabn: 0 slot: 0(0x0) flag: 0x2c lock: 0 ckix: 0 ncol: 1 nnew: 1 size: -1

col 0: [ 4] c3 0e 36 2e

*-----------------------------

* Rec #0x17 slt: 0x45 objn: 1485619(0x0016ab33) objd: 1485619 tblspc: 71(0x00000047)

* Layer: 11 (Row) opc: 1 rci 0x16

Undo type: Regular undo Last buffer split: No Temp Object: No

Tablespace Undo: No rdba: 0x00000000

*-----------------------------

From the trace file above: UNDO BLK:

xid: 0x000c.045.00035b1b seq: 0x1e14 cnt: 0x17 irb: 0x17 icl: 0x0 flg: 0x0000

The undo segment with the active transaction is segment is 0x000c (Hex) which is 12 (Dec) as the XID is: Undo segment no + Slot no + Sequence no

This step is often skipped because it was performed earlier in step 3, however it is a good idea to do this again now to make sure that the XID from the UNDO block matches the UNDO SEGMENT HEADER, this way you have followed all the chain, from the UNDO SEGMENT to UNDO BLOCK, back and forth.  If there is a conflict here please check and make sure that the customer dumped the correct undo block.

Check for the value of irb which is an index which points you to the latest change done to the undo block. This is the point from which a rollback would begin if one was issued.

From the trace file we see: 'irb: 0x17' so this points to record 0x17, so search for this particular string i.e 0x17 and it will take you to undo record 'REC #0x17', e.g.:

* Rec #0x17 slt: 0x45 objn: 1485619(0x0016ab33) objd: 1485619 tblspc: 71(0x00000047)

* Layer: 11 (Row) opc: 1 rci 0x16

Undo type: Regular undo Last buffer split: No Temp Object: No

Tablespace Undo: No rdba: 0x00000000

*-----------------------------

Note the slot number (slt) is 0x45, the object number (objn) is the OBJECT_ID from dba_objects and data object number (objd) is the DATA_OBJECT_ID from dba_objects. These numbers may be the same but not necessarily, and so if the database is open then identify this object, e.g.:

select object_name, owner, object_type, data_object_id from dba_objects where object_id = <objn>;

This is the object, which has an active transaction. Note in the above trace file extract that rci has a value of 0x16 which means that this record is at the end of an undo chain. This means that the chain continues in another UNDO BLOCK. Please refer to unpublished Note:281504.1 for information on Undo chains.

So the next record that needs to be rolled back is present in REC #X016. If rci is 0x00 then it means that this is the first record present in the undo chain and so you can check to see if there is rdba info, e.g.:

* Rec #0x16 slt: 0x45 objn: 1485619(0x0016ab33) objd: 1485619 tblspc: 71(0x00000047)

* Layer: 11 (Row) opc: 1 rci 0x00

Undo type: Regular undo Begin trans Last buffer split: No Temp Object: No

Tablespace Undo: No rdba: 0x00000000

*-----------------------------

uba: 0x1081e796.1e14.14 ctl max scn: 0x0695.38f69853 prv tx scn: 0x0695.38f698a1 KDO undo record:

KTB Redo

op: 0x04 ver: 0x01

op: L itl: scn: 0x0019.009.00034237 uba: 0x36c0cce4.1d2f.19 flg: C--- lkc: 0 scn: 0x0695.38f6b96b

KDO Op code: URP xtype: XA bdba: 0x35406893 hdba: 0x35406892 itli: 1 ispac: 0 maxfr: 4863

tabn: 0 slot: 0(0x0) flag: 0x2c lock: 0 ckix: 0 ncol: 1 nnew: 1 size: -1

col 0: [ 4] c3 0e 36 2e

*-----------------------------

If the object is an Index, drop and recreate it. If it is a table, then again the table would need to be dropped and recreated (or truncated) so that its object number changes and hence the rollback/undo is no longer required. If this isn't possible, then you have two options:

First take a backup of the database in its current state. This is critical in case anything goes wrong and you lose the opportunity to salvage the data!

Option 1

- Restore the undo segment datafile and the datafile containing the object and perform a full recovery. This can only be done if you have all the archived redo as you will need to do full recovery on these files.

OR

Option 2

If option 1 is not possible, you can use the unsupported method, e.g.:

Specify the undo segment in the _OF LINE_ROLLBACK_SEGMENTS parameter and try to drop the rollback segment. If there is an active transaction then this is not likely to work and you will probably need to set the

_CORRUPTED_ROLLBACK_SEGMENTS parameter as well, see unpublished Note:152085.1 ROLLBACK SEGMENT CORRUPTION with _OFFLINE & _CORRUPTED_ROLLBACK_SEGMENTS.

This will cause inconsistency in the database and so You will have to rebuild the database after doing this.

SCENARIOS:

Scenario 1:

DB is open.

No active Transactions

From the trace file:

---------

ORA-00600: internal error code, arguments: [4194], [37], [3], [], [], [], TRN TBL::

index state cflags wrap# uel scn dba parent- xid nub stmt_num cmt

-----------------------------------------------------------------------------------------------

there are no active transactions

Solution:

Identify the problem undo segment. Offline it and then drop it.

Scenario 2: DB is open.

No active Transactions

UNDO_MANAGEMENT parameter is set to MANUAL Solution:

Drop ROLLBACK SEGMENTS and recreate the ROLLBACK tablespace

Scenario 3:

No active transactions.

ORA-600 [4194] occurs when Oracle tries to start a new transaction.

index state cflags wrap# uel scn dba parent-xid nub stmt_num

------------------------------------------------------------------------------------------------

0x00 9 0x00 0x15454 0x0002 0x0000.1f80e920 0x0580049a 0x0000.000.00000000 0x00000001 0x00000000

<all transactions are in status 9>

Block image after block recovery: UNDO BLK:

xid: 0x0015.02b.0001544b seq: 0x163e cnt: 0x12 irb: 0x12 icl: 0x0 flg: 0x0000

USN# 15 (Hex) ==> 21 (Dec) ==> _SYSSMU21$

Solution:

Startup database using _OFFLINE for undo segment 21 and drop the UNDO tablespace Please see above for more details.

Scenario 4:

The undo header data is not available in the trace file or the information is out of date. You will need to gather the details about the active transactions as follows:

1.     If the DB is up the issue:

alter system dump undo header <undo segment>;

2.     If the DB is down, then set event 10015 at level 10 in the init.ora and attempt to restart, e.g.: event="10015 trace name context forever, level 10"

Start the database after setting this event. This will dump each undo segment header with information before and after recovery. You will still not be able to open the database though. The trace file will be generated in the user_dump_dest where it will show the transaction table information for each segment.

Scenario 5:

We prefer do not bounce the DB, there is an alternative to treat this error if the following condition is true: In the segment header dump we see a pending transaction but:

0x1c  10 0x00  0x1087  0x0001  0x0000.03f3ec53 0x00c00174  0x0000.000.00000000 0x00000001 <==

Number of block used is 1 that show is a new transaction, therefore even was showed as active can be ignore. 1- Find the active transaction in the undo segment affected to compare these values with the found in the dump SQL> select XIDUSN, XIDSLOT, XIDSQN, USED_UBLK, USED_UREC from v$transaction where XIDUSN = #;

2-   Set the following parameter that will allow to drop the undo segment: alter system set "_smu_debug_mode"=4;

3-   Verify the status of the Undo Segment

select SEGMENT_ID, SEGMENT_NAME, status from  dba_rollback_segs where segment_id = #;