Skip to content

Oracle恢复

How to recreate idl_ub1$,idl_char$,idl_ub2$,idl_sb4$.

近期遭遇一个客户误truncate idl_ub1$表,导致数据库关闭后无法启动,在受到支援请求后在1小时内解决了此问题,遭遇了此类问题的处理方式主要如下,

以upgrade方式打开数据库,并截断相关表

startup upgrade
truncate table idl_ub1$;
truncate table idl_char$;
truncate table idl_ub2$;
truncate table idl_sb4$;

接着执行utlirp和rmjvm两个脚本,并关闭数据库

@c:\oracle\product\10.2.0\db_1\rdbms\admin\utlirp
shutdown immediate
startup upgrade
@c:\oracle\product\10.2.0\db_1\javavm\install\rmjvm
shutdown immediate
exit

 

再次启动数据库后,设置监听为临时不可用状态并依据数据库组件状态运行相关脚本

startup upgrade
alter system set "_system_trig_enabled"=false scope=memory;

@c:\oracle\product\10.2.0\db_1\javavm\install\initjvm.sql
-- 如果xml已经安装执行此脚本
@c:\oracle\product\10.2.0\db_1\xdk\admin\initxml.sql
@c:\oracle\product\10.2.0\db_1\xdk\admin\xmlja.sql
@c:\oracle\product\10.2.0\db_1\rdbms\admin\catjava.sql
--

shutdown immediate
exit

 

启动数据库并执行ultrp修复失效对象,运行编译修复后,修复完成。

startup upgrade
@c:\oracle\product\10.2.0\db_1\rdbms\admin\utlrp
exit

SQLPLUS / AS SYSDBA
shutdown immediate
startup
execute utl_recomp.recomp_serial();
exit

NOLOGGING 操作引起的坏块 – 错误解释和解决方案

如果数据段定义为 NOLOGGING 属性,当 NOLOGGING/UNRECOVERABLE 操作修改该数据段或者使用datapump import 参数 disable_archive_logging:y,联机重做日志只记录很少的日志信息,如果之后执行 RECOVERY 操作的话,会导致这些块无效。

如果这些联机重做日志/归档日志被用来恢复数据文件,那么 Oracle 会将对应的数据块标志为无效,而且下一次访问这些数据块时,会报 ORA-1578 和 ORA-26040错误。

例如:

SQL> select * from test_nologging;

ORA-01578: ORACLE data block corrupted (file # 11, block # 84)
ORA-01110: data file 4: ‘/oradata/users.dbf’
ORA-26040: Data block was loaded using the NOLOGGING option

以下数据字典视图中的 LOGGING 列记录了 NOLOGGING 属性:

DBA_TABLES, DBA_INDEXES, DBA_LOBS, DBA_TAB_PARTITIONS, DBA_LOB_PARTITIONS, DBA_TAB_SUBPARTITIONS, 等等。

LOGGING=’NO’ 表示 NOLOGGING。

接下来,这些数据块会被标志为 Soft Corrupt,当下一次访问该数据块时,会报 ORA-1578 和 ORA-26040错误。

DATAPUMP 参数 DISABLE_ARCHIVE_LOGGING

DATAPUMP impdp 参数 DISABLE_ARCHIVE_LOGGING:Y 在import时禁止 LOGGING 定义,会产生NOLOGGING操作; 如果相应的datafile被restored 和 recovered, 那么接下来的语句会报错 ORA-1578 和 ORA-26040.

“如果database是 FORCE LOGGING 模式, 那么DISABLE_ARCHIVE_LOGGING 选项不会关闭logging.

import时使用这个参数的例子:

 

impdp scott/tiger directory=DATA_PUMP_DIR dumpfile=dp transform=disable_archive_logging:y

 

DBV 检测坏块时,如果 RDBMS 版本小于 10.2.0.4,那么 DBV 打印错误 DBV-200,如果 RDBMS 版本大于或等于 10.2.0.4,那么 DBV 打印错误 DBV-201  ( Note  5031712.8 ):

DBV-00200: Block, dba 46137428, already marked corrupted
DBV-00201: Block, DBA 46137428, marked corrupt for invalid redo application

“VALIDATE” RMAN 命令用来检测NOLOGGING数据块,检查结果记录在 view v$database_block_corruption (versions lower than 12c) 和 v$nonlogged_block (12c and greater).  下面的例子中检查出datafile 4 有 933 坏块,查询v$database_block_corruption 或者 v$nonlogged_block。

RMAN> VALIDATE DATABASE;

…..
File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
—- —— ————– ———— ————— ———-
4    OK     933            1            6401            2275124
File Name: /oracle/dbs/users.dbf

RMAN 检测坏块时,如果 RDBMS 版本小于 10.2.0.5 和 11.1.0.7,RMAN 打印如下错误:

10.2.0.4 and lower, 11.1.0.6, 11.1.0.7:

RMAN validate reports it in v$database_block_corruption with CORRUPTION_TYPE=LOGICAL

如果 RDBMS 版本大于或等于 10.2.0.5 和 11.2.0.1,RMAN 报告:查看视图 v$database_block_corruption 中 CORRUPTION_TYPE=NOLOGGING 的记录。参考 Note 7396077.8 :

10.2.0.5 and 11.2.0.1+:

RMAN validate reports it in v$database_block_corruption with CORRUPTION_TYPE=NOLOGGING

 

在12c及以后版本中,RMAN validate的结果不在视图v$database_block_corruption中,而是在视图v$nonlogged_block:

12c:

RMAN validate的结果显示在视图v$nonlogged_block

 

在12.2 版本, 可以使用新的命令:”validate .. nonlogged block” 去验证nologging的block。 在以下的例子中,数据文件5,6有nologged的block:

RMAN> validate database nonlogged block;

Starting validate at …
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=133 device type=DISK
channel ORA_DISK_1: starting validation of datafile
channel ORA_DISK_1: validation complete, elapsed time: 00:00:35

List of Datafiles
=================
File Status Nonlogged Blocks Blocks Examined Blocks Skipped
—- —— —————- ————— ————–
1 OK 0 106363 0
2 OK 0 78919 0
3 OK 0 96639 0
4 OK 0 4991 0
5 OK 400 2559 0
6 OK 569 2559 0

Details of nonlogged blocks can be queried from v$nonlogged_block view

Alert log中会更新以下信息:

Started Nonlogged Block Replacement recovery(validate) on file 5 (ospid 26351 rcvid 10616970560844821494)
Finished Nonlogged Block Replacement recovery(validate) on file 5. 400 blocks found

Started Nonlogged Block Replacement recovery(validate) on file 6 (ospid 26351 rcvid 10616970560844821494)
Finished Nonlogged Block Replacement recovery(validate) on file 6. 569 blocks found

NOLOGGING 导致的坏块不会导致 RMAN 备份失败,一般来说 soft corrupt block 不会导致 RMAN 备份失败,不需要设置 MAXCORRUPT。数据库备份中就会含有 soft corrupt block,如果使用这些备份恢复数据,那么恢复的数据也含有 soft corrupt block。

除 ORA-26040 错误之外,当还有一些其他通用信息出现时,block dump 可 能会被产生,如果数据块的 block dump 内有 byte 0xff 信息或者属于某个段,ORA-1578/ORA-26040会因为介质恢复了NOLOGGING的部分导致了corruption而出现。

 

监控 NOLOGGING 操作

执行NOLOGGING操作,并且之后没有备份的情况下,RMAN 命令 “REPORT UNRECOVERABLE” 可以查询出被影响的datafile。

RMAN> report unrecoverable;

using target database control file instead of recovery catalog
Report of files that need backup due to unrecoverable operations
File Type of Backup Required Name
—- ———————– ———————————–
4    full or incremental     /oracle/dbs/users.dbf

当初始化参数db_unrecoverable_scn_tracking设置为true (默认值),那么V$DATAFILE 中以下列会被更新;
db_unrecoverable_scn_tracking参数在10g中是不可用的。请参考Oracle Database在线文档中关于V$DATAFILE中以下列信息:

UNRECOVERABLE_CHANGE#
UNRECOVERABLE_TIME
FIRST_NONLOGGED_SCN
FIRST_NONLOGGED_TIME

在11.2.0.4 or 12.1.0.2+版本中,设置event 16490的情况下,物理备库的MRP进程会检查出NOLOGGING变化,并记录在alert log。

ORA-16490 “logging invalidated blocks on standby due to invalidation redo”

“INVD_BLKS: Invalidating (file <file number>, bno <block number>)”
“fname: ‘Datafile name’. rdba: …”

识别数据块什么时候被标志为NOLOGGING

识别数据块什么时候被标志为NOLOGGING,可以将trace文件中数据块SCN或者v$database_block_coruption视图中CORRUPTION_CHANGE#值
转换为时间:

使用trace文件中数据块SCN
例如:

  Start dump data blocks tsn: 60 file#: 4 minblk 84 maxblk 84
buffer tsn: 3 rdba: 0x02c00054 (11/84)
scn: 0x0771.4fa24eb5 seq: 0xff flg: 0x04 tail: 0x4eb500ff

提取SCN值 0x0771.4fa24eb5, 删除 ‘.’ ,然后转换 0x07714fa24eb 到十进制 511453045995

 

使用v$database_block_coruption视图中CORRUPTION_CHANGE#值

如果运行RMAN validate命令后,v$database_block_coruption视图中corruption_type=’NOLOGGING’ (10.2.0.5 和 11.2.0.1+),
那么CORRUPTION_CHANGE#列的值就是十进制的SCN值。

获得SCN Timestamp

使用下面的方法获得时间:

  

select scn_to_timestamp(&&decimal_scn)
from dual;

如果运行RMAN validate:

select file#, block#, scn_to_timestamp(CORRUPTION_CHANGE#)
from v$database_block_corruption
where CORRUPTION_TYPE=’NOLOGGING’;

In 12c:

select file#, block#, scn_to_timestamp(NONLOGGED_START_CHANGE#)
from v$nonlogged_block;

 

如果查询gv$archived_log 或 gv$log_history遇到错误ORA-08181:

alter session set nls_date_format = ‘DD-MON-YY HH24:MI:SS’;

select first_time, next_time
from gv$archived_log
where &decimal_scn between first_change# and next_change#;

select first_time
from gv$log_history
where &decimal_scn between first_change# and next_change#;

如果运行RMAN validate:

alter session set nls_date_format = ‘DD-MON-YY HH24:MI:SS’;

select file#, block#, first_time, next_time
from v$archived_log, v$database_block_corruption
where CORRUPTION_CHANGE# between first_change# and next_change#
and CORRUPTION_TYPE=’NOLOGGING’;

select file#,block#,first_time
from   v$log_history, v$database_block_corruption
where  CORRUPTION_CHANGE# between first_change# and next_change#
and CORRUPTION_TYPE=’NOLOGGING’;

12c:

alter session set nls_date_format = ‘DD-MON-YY HH24:MI:SS’;

select file#, block#, first_time, next_time
from v$nonlogged_block, v$archived_log
where NONLOGGED_START_CHANGE# between first_change# and next_change#;

OR

select file#, block#, first_time
from v$nonlogged_block, v$log_history
where NONLOGGED_START_CHANGE# between first_change# and next_change#;

 

SYSAUX表空间/AWR,EM 等出现NOARCHIVELOG 和 NOLOGGING 问题

 

如果数据库版本是11.1.0.6 或 11.1.0.7 或 11.2.0.1,
对NOLOGGING对象执行过DIRECT PATH操作,并且后续执行了RECOVER DATABASE命令,及时数据库FORCE LOGGING是打开的情况下,
会出现ORA-1578 and ORA-26040错误。

 

这种问题经常发生在SYSAUX表空间中的AWR或EM对象

 

解决方法

NOLOGGING 操作引起的坏块是不能修复的,比如”Media Recovery” 或 “RMAN blockrecover”都无法修复这种坏块。可行的方法是在 NOLOGGING 操作之后立刻备份对应的数据文件。

问题是在执行RMAN DUPLICATE或RESTORE之后产生?

如果问题是执行RMAN DUPLICATE 或 RESTORE之后 ,那么在源库打开FORCE LOGGING,然后再重新运行RMAN DUPLICATE 或 RESTORE。

  • alter database force logging;

问题是发生在物理standby库?

  • 如果错误出现在物理 STANDBY 数据库, 从主库恢复被影响的数据文件 (只有当主库没有这个问题的情况下)。 在12c中可以使用RMAN选项RECOVER NONLOGGED BLOCK with DATAFILE,TABLESPACE,DATABASE。例如:
  •   
    RMAN> RECOVER DATABASE NONLOGGED BLOCK;
  • 为了避免这个问题发生,在主库强制生产日志:
  • alter database force logging;

如果同一个datafile的数据块在主库出现nologging 坏块,但是备库没有,可以通过手动跳过(dbms_repair)坏块 或者设置event 10231.
主库出现nologging 坏块可能是由于主库执行过备份恢复或者之前是备库,执行了switchover

识别被影响的Segment

找到坏块所在的对象:

  • 如果NOLOGGING数据块位于空闲数据块(dba_free_space视图可以查询到),DBVerify检查会发现这个问题,报错DBV-00201
    或者在v$database_block_corruption视图中显示.对于这种情况,我们可以等待到这个数据块被重用时,会自动格式化,或者
    手动强制格式化
  • 如果是索引,重新创建(drop/create)索引。
  • 如果是,使用 procedure DBMS_REPAIR.SKIP_CORRUPT_BLOCKS 跳过坏块。然后考虑是否重建表:移动table: alter table &table_name move;或者保存数据 (export, Create Table as Select, etc) 然后truncate 或 drop/create.
  • 如果是LOB 另行处理

如果删除有坏块的段之后,这个坏块就处于空闲状态,后续可以被分配给其他对象/段,当这个坏块被分配给他对象/段时,
这个数据块被重新格式化。 如果v$database_block_corruption视图中还是显示为坏块,那么手动运行rman validate来清除视图中的信息。

OBJ$表上索引有坏块导致部分业务失效的修复

这种问题的解决办法有好几种,根据数据量的大小以及业务是否允许停机的情况考虑,主要分以下几类救急办法:

一、坏块在索引上的位置判断

1.1如果坏块不多,而且坏块都在在索引比较靠前的位置(这个靠前的位置每个版本不一样,每个版本的对象数不一样),可以理解为在建库时候obj$自带索引的那个块最大地址就是属于考前的位置,当坏块在这个位置前面时候有一种比较便捷的处理方式,

就是通过创建一个一模一样的数据库,使用bbed copy(相比DD操作,操作起来超级简单又不容易出错)把对应坏块位置的块copy覆盖损坏的块,apply后即可(如果是在obj$表中的块那么记得修改下obj里对应的create time即可使数据库恢复正常,这个在前面有个案例描述过)。

 

1.2如果坏块是在索引靠后的位置或者坏块很多,索引的对象为后面创建的对象,那么这个时候就比较麻烦了,好在obj$的数据是完整的,

我的建议是采用2种方法,

1.2.1  采用屏蔽obj$索引的方法,具体需要修改基表,这个方法在前面文章中有描述。这个情况一般用在数据库比较大,比如数据量有几十个T,上百T或者业务不能接受长时间停机的情况

1.2.2 当数据库的数据量不大的时候,可以采用另外一种办法,使用exp来导出数据进行修复,一般这个时候exp工具是无法直接使用的,需要使用到catexp.sql这个脚本,具体的方法请查阅博客里的文章 Corruptions on OBJ$ indexes

 

Oracle bootstrap$ 说明

What is bootstrap?

Bootstrap is a technique for loading the first few instructions of a computer program into active memory and then using them to bring in the rest of the program.

What is bootstrap in Oracle ?

In Oracle, Bootstrap refers to loading of metadata (data dictionary) before we OPEN the database.Bootstrap objects are classified as the objects (tables / indexes / clusters) with the object_id below 56 as bootstrap objects.  These objects are mandatory to bring up an instance, as this contains the most important metadata of the database.

What happens on database startup?

This shall be explained by setting the SQL_TRACE while opening the database.Connect as sysdba and do the following
SQL> startup mount ;
SQL> alter session set events ‘10046 trace name context forever, level 12 ‘ ;
SQL> alter database open ;
SQL>  alter session set events ‘10046 trace name context off ‘ ;
SQL> ORADEBUG SETMYPID
SQL> ORADEBUG TRACEFILE_NAME
The sql_trace of the above process explains the following operations behind startup. The bootstrap operation happens between MOUNT stage and OPEN stage.
1.)  The first SQL after in the above trace shows the creation of the bootstrap$ table. Something similar to the following:
create table bootstrap$ ( line# number not null, obj# number not null, sql_text varchar2(4000) not null) storage (initial 50K objno 56 extents (file 1 block 377))
This sys.bootstrap$ table contains the DDL’s for other bootstrap tables (object_id below 56). Actually these tables were created internally by the time of database creation (by sql.bsq), The create DDL passed between MOUNT and OPEN stage will be executed through different driver routines. In simple words these are not standard CREATE DDLs.
While starting up the database oracle will load these objects into memory (shared_pool), (ie) it will assign the relevant object number and refer to the datafile and the block associated with that. And such operations happen only while warm startup.
 The internals of the above explained in ‘kqlb.c’.
2.)  Now a query executed against the sys.bootstrap$ table, which holds the create sql’s for other base tables.
select line#, sql_text from bootstrap$ where obj# != :1 (56)
Subsequently it will create those objects by running those queries.
Object number 0 – (System Rollback Segment)
Object number 2 to 55 (Other base tables)
Object number 1 is NOT used by any of the objects.
3.) Performs various operations to keep the bootstrap objects in consistent state.
Upon the successful completion of bootstrap the database will do the other tasks like recovery and will open the database.

Which objects are classified as bootstrap objects in oracle database?

Objects with data_object_id less than 56 are classified as core bootstrap objects.The objects are added to the bootstrap. The objects affected are :

hist_head$
histgrm$
i_hh_obj#_col#
i_hh_obj#_intcol#
i_obj#_intcol#
i_h_obj#_col#
c_obj#_intcol#
From 10.1 the following objects have been added:
fixed_obj$
tab_stats$
ind_stats$
i_fixed_obj$_obj#
i_tab_stats$_obj#
i_ind_stats$_obj#
object_usage
These additional objects shall be re-classified (or) ignored by following methods.
1. Opening the database in migrate mode
2. Using event 38003
Event 38003 affects the bootstrap process of loading the fixed cache in  kqlblfc(). Per default certain objects are marked as bootstrap objects (even though they are not defined as such in sys.bootstrap$) but by setting the event they will be left as non-bootstrapped.

What is bootstrap process failure? or  ORA-00704

This ORA-00704 error SERIOUS if reported at startup. This error refers to some problem during bootstrap operation. Any ORA-00704 error on STARTUP / RECOVER is serious, this error normally rose due to some inconsistency with the bootstrap segments (or) data corruption on bootstrap$ (or) any of the base tables below object_id  56. After this error it might not allow to open that database.

When ORA-00704 shall occur?

1. There is a probable of this error when any unsupported operations are tried to force open the database.
2. This error can also occur when system datafile has corrupted blocks. (ORA-01578)
3. In earlier releases of oracle (prior to 7.3.4 and 8.0.3) this issue shall arise due to Bug 434596
The option is to restore it from a good backup and recover it.
-> If the underlying cause is physical corruption that is due to hardware problems then do complete recovery.
-> If the issue is not relating to any physical corruption, then the problem could be due some unsupported actions on Bootstrap, and a Point In Time Recovery would be an option in such cas.

Corruptions on OBJ$ indexes。OBJ$上索引损坏的一种处理方式(数据量不大的时候,只要业务允许停机可以忽略数据量的考虑)

INTERNAL:Corruptions on OBJ$ indexes (Doc ID 39400.1)

 

To export a system when there are corruptions on indexes on OBJ$:

 

–     Take a FULL COLD BACKUP

 

–     Connect internal in SQLDBA and assuming your default tablespace is

SYSTEM and you have some free space:

 

create table pig$ as select * from obj$;

 

create unique index p_obj1 on pig$(obj#);

 

create unique index p_obj2 on pig$(owner#, name, namespace,

remoteowner, linkname);

 

 

–     Now edit the catexp.sql script (take a safe copy first).

This is generally located in the RDBMS/ADMIN directory.

 

Edit this to change ALL references to the table ‘OBJ$’

to now reference ‘PIG$’.

 

–     Connect internal in SQLDBA and run this new CATEXP.SQL

 

–     Perform a FULL export.

 

–     Shutdown the database and delete it.

 

–     Recreate the database from scratch with a CREATE DATABASE command.

Ensure you set up a second rollback segment for non-system

tablespace use on the import.

 

–     Perform a FULL import.