The Cluster Node Addition of /u01/app/11.2.0/grid was successful. Please check '/tmp/silentInstall.log' for more details. [oracle@rac1 ~]$ cd $ORACLE_HOME/oui/bin [oracle@rac1 bin]$ export IGNORE_PREADDNODE_CHECKS=Y [oracle@rac1 bin]$ ./addNode.sh -silent "CLUSTER_NEW_NODES={rac2}" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 1562 MB Passed Oracle Universal Installer, Version 11.2.0.4.0 Production Copyright (C) 1999, 2013, Oracle. All rights reserved. Performing tests to see whether nodes rac2 are available xecution setup for node rac1 using shells /usr/bin/ssh and /usr/bin/rsh File "/usr/bin/rsh" does not exist on node "rac1" ote command execution setup for node rac2 using shells /usr/bin/ssh and /usr/bin/rsh File "/usr/bin/rsh" does not exist on node "rac2" Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).). Correct the problem and try the operation again. ......................... 100% Done.

存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理 SQL> recover database until cancel; ORA-00283: 恢复会话因错误而取消 ORA-01122: 数据库文件 536 验证失败 ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora' ORA-01207: 文件比控制文件更新 - 旧的控制文件 Sun May 05 00:09:03 2024 ALTER DATABASE RECOVER database until cancel Media Recovery Start started logmerger process Sun May 05 00:09:10 2024 SUCCESS: diskgroup FRA was mounted Sun May 05 00:09:10 2024 NOTE: dependency between database orcl and diskgroup resource ora.FRA.dg is established Sun May 05 00:09:14 2024 WARNING! Recovering data file 1 from a fuzzy backup. It might be an online backup taken without entering the begin backup command. Media Recovery failed with error 1122 Slave exiting with ORA-283 exception Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_pr00_8048.trc: ORA-00283: 恢复会话因错误而取消 ORA-01122: 数据库文件 536 验证失败 ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora' ORA-01207: 文件比控制文件更新 - 旧的控制文件 Sun May 05 00:09:16 2024 Recovery Slave PR00 previously exited with exception 283 ORA-283 signalled during: ALTER DATABASE RECOVER database until cancel ... using backup controlfile进行恢复 SQL> recover database using backup controlfile until cancel; ORA-00279: 更改 18646239951 (在 04/25/2024 17:14:50 生成) 对于线程 1 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505 ORA-00280: 更改 18646239951 (用于线程 1) 在序列 #1003886 中 指定日志: {=suggested | filename | AUTO | CANCEL} auto ORA-00279: 更改 18646239951 (在 04/25/2024 17:11:40 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807 ORA-00280: 更改 18646239951 (用于线程 2) 在序列 #677876 中 ORA-00279: 更改 18646255791 (在 04/25/2024 17:16:46 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099 ORA-00280: 更改 18646255791 (用于线程 2) 在序列 #677877 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807' ORA-00279: 更改 18646295647 (在 04/25/2024 17:21:38 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677878.199379.1167240623 ORA-00280: 更改 18646295647 (用于线程 2) 在序列 #677878 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099' ORA-00279: 更改 18646331784 (在 04/25/2024 17:28:25 生成) 对于线程 1 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507 ORA-00280: 更改 18646331784 (用于线程 1) 在序列 #1003887 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505' ORA-00308: 无法打开归档日志 '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507' ORA-17503: ksfdopn: 2 未能打开文件 +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507 ORA-15012: ASM file '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507' does not exist ORA-10879: error signaled in parallel recovery slave ORA-01547: 警告: RECOVER 成功但 OPEN RESETLOGS 将出现如下错误 ORA-01194: 文件 1 需要更多的恢复来保持一致性 ORA-01110: 数据文件 1: '+DATA/orcl/system01.dbf' 通过分析,确认由于cache丢失导致thread_1_seq_1003887这个日志丢失（而且redo已经被覆盖） 数据库无法通过正常recover的思路解决.通过屏蔽一致性,强制打开数据库,alert日志报ORA-600 2662错误 Sat May 04 17:23:00 2024 Redo thread 2 internally disabled at seq 1 (CKPT) ARC1: Archiving disabled thread 2 sequence 1 Archived Log entry 2 added for thread 2 sequence 1 ID 0x0 dest 1: ARC3: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc (incident=47066): ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc: ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc: ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 3684): terminating the instance due to error 600 Instance terminated by USER, pid = 3684 ORA-1092 signalled during: alter database open resetlogs... 通过修改数据库scn,open数据库报ORA-600 4137 Sun May 05 00:12:41 2024 replication_dependency_tracking turned off (no async multimaster replication found) LOGSTDBY: Validating controlfile with logical metadata LOGSTDBY: Validation complete Completed: alter database open resetlogs Sun May 05 00:12:56 2024 Trace dumping is performing id=[cdmp_20240505001256] Sun May 05 00:12:56 2024 ORACLE Instance orcl1 (pid = 22) - Error 600 encountered while recovering transaction (28, 21). Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_smon_5896.trc: ORA-00600: ??????, ??: [4137], [28.21.42965783], [0], [0], [], [], [], [], [], [], [], [] 这个错误比较明显,由于28号回滚段异常导致,对异常回滚段进行处理,重建undo,数据库恢复主要工作完成

数据库启动报ORA-600 2131错误,查看alert日志发现是在mount过程报错 Fri May 17 20:58:28 2024 Starting ORACLE instance (normal) LICENSE_MAX_SESSION = 0 LICENSE_SESSIONS_WARNING = 0 Initial number of CPU is 16 Number of processor cores in the system is 8 Number of processor sockets in the system is 1 Picked latch-free SCN scheme 3 Autotune of undo retention is turned on. IMODE=BR ILAT =249 LICENSE_MAX_USERS = 0 SYS auditing is disabled Starting up: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options. Windows NT Version V6.2 CPU : 16 - type 8664, 8 Physical Cores Process Affinity : 0x0x0000000000000000 Memory (Avail/Total): Ph:93799M/97925M, Ph+PgF:78891M/112261M Using parameter settings in server-side spfile E:\APP\ADMINISTRATOR\PRODUCT\11.2.0\DBHOME_1\DATABASE\SPFILEorcl.ORA System parameters with non-default values: processes = 1500 sessions = 2272 nls_language = "SIMPLIFIED CHINESE" nls_territory = "CHINA" sga_target = 29440M control_files = "E:\ORADATA\orcl\CONTROL01.CTL" db_block_size = 8192 compatible = "11.2.0.4.0" log_archive_dest_1 = "LOCATION=e:\app\archivelog\" log_archive_format = "ARC%S_%R.%T" undo_tablespace = "UNDOTBS2" sec_case_sensitive_logon = FALSE remote_login_passwordfile= "EXCLUSIVE" db_domain = "" dispatchers = "(PROTOCOL=TCP) (SERVICE=orclXDB)" audit_file_dest = "E:\APP\ADMINISTRATOR\ADMIN\orcl\ADUMP" audit_trail = "NONE" db_name = "orcl" open_cursors = 300 pga_aggregate_target = 9792M diagnostic_dest = "E:\APP\ADMINISTRATOR" Fri May 17 20:58:29 2024 PMON started with pid=2, OS id=6696 Fri May 17 20:58:29 2024 PSP0 started with pid=3, OS id=2424 Fri May 17 20:58:30 2024 VKTM started with pid=4, OS id=5472 at elevated priority VKTM running at (10)millisec precision with DBRM quantum (100)ms Fri May 17 20:58:30 2024 GEN0 started with pid=5, OS id=5764 Fri May 17 20:58:30 2024 DIAG started with pid=6, OS id=372 Fri May 17 20:58:30 2024 DBRM started with pid=7, OS id=2992 Fri May 17 20:58:30 2024 DIA0 started with pid=8, OS id=4960 Fri May 17 20:58:30 2024 MMAN started with pid=9, OS id=6036 Fri May 17 20:58:30 2024 DBW0 started with pid=10, OS id=4724 Fri May 17 20:58:30 2024 DBW1 started with pid=11, OS id=2652 Fri May 17 20:58:30 2024 LGWR started with pid=12, OS id=5320 Fri May 17 20:58:30 2024 CKPT started with pid=13, OS id=5732 Fri May 17 20:58:30 2024 SMON started with pid=14, OS id=936 Fri May 17 20:58:30 2024 RECO started with pid=15, OS id=2192 Fri May 17 20:58:30 2024 MMON started with pid=16, OS id=5576 Fri May 17 20:58:30 2024 MMNL started with pid=17, OS id=5748 starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'... starting up 1 shared server(s) ... ORACLE_BASE from environment = E:\app\Administrator Fri May 17 20:58:31 2024 ALTER DATABASE MOUNT Errors in file E:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\trace\orcl_ora_5452.trc (incident=403399): ORA-00600: ??????, ??: [2131], [9], [8], [], [], [], [], [], [], [], [], [] Incident details in: E:\APP\ADMINISTRATOR\diag\rdbms\orcl\orcl\incident\incdir_403399\orcl_ora_5452_i403399.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ORA-600 signalled during: ALTER DATABASE MOUNT... 这个错误是由于controlfile损坏导致,有这个库以前部署过rman备份，解决起来比较简单，使用rman还原控制文件,并尝试recover RMAN> restore controlfile from 'E:\rmanback\rmanfile\CTL_20240517_A62R067K_1_1.RMAN'; 启动 restore 于 17-5月 -24 使用通道 ORA_DISK_1 通道 ORA_DISK_1: 正在还原控制文件 通道 ORA_DISK_1: 还原完成, 用时: 00:00:01 输出文件名=E:\ORADATA\orcl\CONTROL01.CTL 完成 restore 于 17-5月 -24 RMAN> RMAN> RMAN> alter database mount; 数据库已装载 释放的通道: ORA_DISK_1 RMAN> recover database; 启动 recover 于 17-5月 -24 分配的通道: ORA_DISK_1 通道 ORA_DISK_1: SID=996 设备类型=DISK 正在开始介质的恢复 线程 1 序列 4100 的归档日志已作为文件 E:\ORADATA\orcl\REDO02.LOG 存在于磁盘上 线程 1 序列 4101 的归档日志已作为文件 E:\ORADATA\orcl\REDO03.LOG 存在于磁盘上 线程 1 序列 4102 的归档日志已作为文件 E:\ORADATA\orcl\REDO01.LOG 存在于磁盘上 归档日志文件名=E:\APP\ARCHIVELOG\ARC0000004025_1165094245.0001 线程=1 序列=4025 归档日志文件名=E:\APP\ARCHIVELOG\ARC0000004026_1165094245.0001 线程=1 序列=4026 ………… 归档日志文件名=E:\APP\ARCHIVELOG\ARC0000004099_1165094245.0001 线程=1 序列=4099 归档日志文件名=E:\ORADATA\orcl\REDO02.LOG 线程=1 序列=4100 归档日志文件名=E:\ORADATA\orcl\REDO03.LOG 线程=1 序列=4101 归档日志文件名=E:\ORADATA\orcl\REDO01.LOG 线程=1 序列=4102 介质恢复完成, 用时: 00:00:22 完成 recover 于 17-5月 -24 RMAN> exit 恢复管理器完成。 E:\oradata\orcl> 这种恢复情况下,如果现在要打开库,需要resetlogs方式,考虑通过创建ctl直接打开（不想用resetlogs） SQL> shutdown immediate; ORA-01109: 数据库未打开 已经卸载数据库。 ORACLE 例程已经关闭。 SQL> startup nomount; ORACLE 例程已经启动。 Total System Global Area 3.0732E+10 bytes Fixed Size 2296264 bytes Variable Size 3825206840 bytes Database Buffers 2.6844E+10 bytes Redo Buffers 61206528 bytes SQL> CREATE CONTROLFILE REUSE DATABASE "orcl" NORESETLOGS ARCHIVELOG 2 MAXLOGFILES 16 3 MAXLOGMEMBERS 3 4 MAXDATAFILES 100 5 MAXINSTANCES 8 6 MAXLOGHISTORY 876 7 LOGFILE 8 GROUP 1 'E:\ORADATA\orcl\REDO01.LOG' SIZE 50M BLOCKSIZE 512, 9 GROUP 2 'E:\ORADATA\orcl\REDO02.LOG' SIZE 50M BLOCKSIZE 512, 10 GROUP 3 'E:\ORADATA\orcl\REDO03.LOG' SIZE 50M BLOCKSIZE 512 11 -- STANDBY LOGFILE 12 DATAFILE 13 'E:\ORADATA\orcl\SYSTEM01.DBF', 14 'E:\ORADATA\orcl\SYSAUX01.DBF', 15 'E:\ORADATA\orcl\USERS01.DBF', 16 'E:\ORADATA\orcl\orcl_DATA01.DBF', 17 'E:\ORADATA\orcl\orcl_INDEX01.DBF', 18 'E:\ORADATA\orcl\UNDOTBS2.DBF' 19 CHARACTER SET ZHS16GBK 20 ; 控制文件已创建。 SQL> recover database; 完成介质恢复。 SQL> alter database open; 数据库已更改。 SQL> ALTER TABLESPACE TEMP ADD TEMPFILE 'E:\ORADATA\orcl\TEMP01.DBF' REUSE; 表空间已更改。 至此本次恢复晚上,由于arch,redo和数据文件没有损坏,恢复非常完美