在 OL7.2 上安装 Oracle RAC for Linux x86_64- 打 PSU 更新版 目录 一 官方文档中的一些介绍 Memory Requirements Disk Space

Transcription

1 在 OL7.2 上安装 Oracle RAC for Linux x86_64- 打 PSU 更新版 目录 一 官方文档中的一些介绍 Memory Requirements Disk Space Requirements Package Requirements Additional Software Requirements Oracle ODBC Drivers Installing the cvuqdisk Package for Linux... 6 二 Linux 系统的几个级别... 6 三 安装 RAC 用 VirtualBox 安装 Oracle Linux 7.2 For Linux x86_64 的过程此处省略, 这里有几点需要注意一下 停以下服务 ( 两节点 ) 禁用 SELinux ( 两节点 ) 挂载 ISO 镜像文件 配置本地 YUM 源 网络命令 ifconfig 不存在, 通 YUM 源安 net-tools 软件包解决 安装桌面环境 在 /etc/hosts 文件中增加如下内容 通过 YUM 源安装 tigervnc-server ( 图形界面远程连接工具 ) 配置 VNCSERVER 并设置开机自动启动该服务 (1) 配置 VNC Service (2) 设置开机启动 VNC Service / 120

2 (3) 设置 VNC 密码 操作系统的磁盘和 ASM 共享磁盘容量规划 用户 root 的密码和两节点数据库 REDO 个数 创建用户和组 ( 两节点 ) 配置 grid 用户和 oracle 用户的环境变量 ( 两节点 ) 创建所需的目录及赋予相应的权限 ( 两节点 ) 编辑 /etc/security/limits.conf 文件, 增加如下内容 ( 两节点 ) 编辑 /etc/pam.d/login 文件, 增加如下内容 ( 两节点 ) 编辑 /etc/profile 文件, 添加如下参数 ( 两节点 ) 编辑 /etc/sysctl.conf 文件, 增加下列参数 ( 两节点 ) [Oracle Linux 7.2 下为这个文件 # vi /usr/lib/sysctl.d/00-system.conf] 配置 SSH 互信, 在图形界面点击设置, 此步不用命令 将 grid 和 oracle 的软件安装包上传到节点一的 /software 目录 用 YUM 源安装 RAC 必需的 RPM 软件包 用 VirutalBox 克隆 adccrac1 为另一台主机 adccrac2, 选 Full Clone, 并选择 重新初始化两块网卡的 MAC 地址 关闭两节点的主机, 在节点一上添加 6 块磁盘并将磁盘类型设置为 可共享, 在节点二上依次添加节点一刚刚新增的磁盘, 最后将 6 块磁盘设置为共享模式, 并开启两个节点的主机 用 UDEV 方式绑定磁盘 ( 两节点 ) 对 grid 软件打 PSU 补丁 在节点一打开 VNC 的图形桌面 由于存在 BUG , 安装 Oracle Grid Infrastructure 软件, 后面打完补丁后再配置 GI Oracle Grid Infrastructure Installation Issue 给 GI 软件打 PSU 补丁 (1) 更新 Opatch 工具 (2) 打补丁 配置 Grid (1) 克隆 grid 软件到节点 / 120

3 (2) 克隆 ORACLE_HOME 配置 (3) 进行 Cluster 配置 检查 GI ( 两节点都检查 ) 通过 asmca 命令创建 ASM 磁盘的 data back arch 3 个磁盘组 ( 只在节点一上做 ) 安装 DB ( 只在节点一上做 ) 只安装软件, 不建库 (1) 对 db 软件打补丁 (2) 安装 在 DB 上安装 PSU (1) 介质 (2) 升级 Opatch 工具 (3) 安装 PSU 使用 DBCA 命令在节点一, 创建 RAC 数据库 racdb 设置归档目录, 并重启数据库到 mount 状态, 打开归档模式, 最后打开数据库 在 grid 用户下关闭集群服务之后, 重新启动正常 一 官方文档中的一些介绍 Memory Requirements The following are the memory requirements for installing Oracle Database 11g Release 2 (11.2): Minimum: 1 GB of RAM Recommended: 2 GB of RAM or more 3 / 120

4 RAM Swap Space Between 1 GB and 2 GB 1.5 times the size of the RAM Between 2 GB and 16 GB Equal to the size of the RAM More than 16 GB 16 GB Disk Space Requirements The following are the disk space requirements for installing Oracle Database 11g Release 2 (11.2): 1 GB of space in the /tmp directory The following tables describe the disk space requirements for software files and data files for each installation type on Linux x86-64: Installation Type Requirement for Software Files (GB) Enterprise Edition 4.7 Installation Type Disk Space for Data Files (GB) Enterprise Edition 1.7 Starting with Oracle Database 11g Release 2 ( ), Oracle Linux 7 and Red Hat Enterprise Linux 7 are supported on Linux x Package Requirements Oracle Database Package Requirements for Linux x86-64 Oracle Linux 7 and Red Hat Enterprise Linux 7 The following packages (or later versions) must be installed: binutils el7.x86_64 compat-libcap el7.x86_64 compat-libstdc el7.i686 compat-libstdc el7.x86_64 4 / 120

5 gcc el7.x86_64 gcc-c el7.x86_64 glibc el7.i686 glibc el7.x86_64 glibc-devel el7.i686 glibc-devel el7.x86_64 ksh libaio el7.i686 libaio el7.x86_64 libaio-devel el7.i686 libaio-devel el7.x86_64 libgcc el7.i686 libgcc el7.x86_64 libstdc el7.i686 libstdc el7.x86_64 libstdc++-devel el7.i686 libstdc++-devel el7.x86_64 libxi el7.i686 libxi el7.x86_64 libxtst el7.i686 libxtst el7.x86_64 make el7.x86_64 sysstat el7.x86_64 5 / 120

6 2.4.5 Additional Software Requirements Oracle ODBC Drivers To use ODBC on Linux x86-64, the following additional ODBC RPMs are required: On Oracle Linux 7, and Red Hat Enterprise Linux 7: unixodbc el7.x86_64 or later unixodbc el7.i686 or later unixodbc-devel el7.x86_64 or later unixodbc-devel el7.i686 or later 2.8 Installing the cvuqdisk Package for Linux # rpm -iv cvuqdisk rpm 二 Linux 系统的几个级别 为了让 Linux 系统在启动后能直接进入到控制台字符操作模式, 我们可以配置 /etc/inittab 文件 更改 id:5: initdefault: 这一行为 id:3:initdefault: 后重新启 动系统即可 我们看到, 简简单单地将 5 改为 3, 就能实现启动时直接进入 X-Window 图形操作界面或控制台字符操作界面的转换, 这是因为 Linux 操作 系统有六种不同的运行级别 ( 见 /etc/inittab 文件中描述 ), 在不同的运行级下, 系统有着不同的状态, 这六种运行级分别为 : # Default runlevel. The runlevels used by RHS are: # 0 - halt (Do NOT set initdefault to this) 6 / 120

7 # 1 - Single user mode # 2 - Multiuser, without NFS (The same as 3, if you do not have networking) # 3 - Full multiuser mode # 4 - unused # 5 - X11 # 6 - reboot (Do NOT set initdefault to this) # 三 安装 RAC 1. 用 VirtualBox 安装 Oracle Linux 7.2 For Linux x86_64 的过程此处省略, 这里有几点需要注意一下 (1) 最小化安装 ( 大约 320 个软件包 ), 先不安装图形界面, 之后通过 YUM 源进行安装 ; (2) 安装过程中, 设置时区 ( 亚洲上海 ) 自动分区 主机名(adccrac1, 这里一定不要设置为大写, 因为在后面安装 GI 软件时,SSH 互信虽然设置成功, 但是测试总是一直报错, 后来查阅相关文档 [INS-06006] Passwordless SSH connectivity not set up 得知, 这是 的一个 Bug,12c 目前没有测试过 ) 和 ROOT 密码 (adccrac); (3) 设置网络, 两块网卡 enp0s3 和 enp0s8 的 IP 地址和子网掩码等等 ; 其中 IP 地址是手动方式 ( 不是 DHCP 自动获取 ) 设置, 网卡 enp0s3, , , 网卡 enp0s8, , , 没有网关然后将两块网卡都选择开启 (4) 在安装成功操作系统并重新启动之后, 系统默认将两块网卡设置的开机不启动 ( 虽然上一步选择的是开启 ), 下面对两块网卡的配置文件进行设置, 让其下次启动以后自动开启 [root@adccrac1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s3 TYPE=Ethernet 7 / 120

8 BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no NAME=enp0s3 UUID=f26a2d18-32d6-4a17-88de-68127c860b82 DEVICE=enp0s3 ONBOOT=yes <<== 将这个值改为 yes, 原来是 no IPADDR= PREFIX=24 GATEWAY= IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_PRIVACY=no ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8 TYPE=Ethernet BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes 8 / 120

9 IPV6_FAILURE_FATAL=no NAME=enp0s8 UUID=973d91e4-913d-47ec-b117-9bc36190cd65 DEVICE=enp0s8 ONBOOT=yes <<== 将这个值改为 yes, 原来是 no IPADDR= PREFIX=24 IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_PRIVACY=no (5) 接下来就可以使用 Xshell 的 SSH 客户端工具连接到这台 Linux 服务器进行后续的操作了 ; 2. 停以下服务 ( 两节点 ) systemctl stop NetworkManager.service systemctl disable NetworkManager.service systemctl stop firewalld.service systemctl disable firewalld.service -- 没有 ntpd 的软件包 [root@adccrac1 ~]# rpm -qa grep ntp fontpackages-filesystem el7.noarch ntpdate-4.2.6p5-22.el7.x86_64 python-ntplib el7.noarch 9 / 120

10 CentOS7.2 默认使用 firewalld; iptables 默认无法被 systemctl 控制, 需要安装 iptables-services iptables-devel 这两个依赖包 ~]# rpm -qa grep iptables iptables el7.x86_64 3. 禁用 SELinux ( 两节点 ) vi /etc/sysconfig/selinux SELINUX=disabled 然后重启主机 4. 挂载 ISO 镜像文件 [root@adccrac1 ~]# mkdir -p /media/ol7.2 用 xftp 上传 V iso 到 /media/ol7.2 [root@adccrac1 ~]# mkdir -p /mnt/iso [root@adccrac1 ~]# [root@adccrac1 ~]# mount -o loop,ro /media/ol7.2/v iso /mnt/iso/ [root@adccrac1 ~]# cat /etc/fstab 在最后新增一行 # # /etc/fstab # Created by anaconda on Fri Jul 27 19:10: # # Accessible filesystems, by reference, are maintained under '/dev/disk' 10 / 120

11 # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/ol_adcc-root / xfs defaults 0 0 UUID=54c63908-f74d fb7-93aa684e4715 /boot xfs defaults 0 0 /dev/mapper/ol_adcc-home /home xfs defaults 0 0 /dev/mapper/ol_adcc-swap swap swap defaults 0 0 /media/ol7.2/v iso /mnt/iso/ iso9660 defaults,ro,loop 配置本地 YUM 源 [root@adccrac1 ~]# cd /etc/yum.repos.d/ [root@adccrac1 yum.repos.d]# ls -l total 4 -rw-r--r--. 1 root root 2560 Nov public-yum-ol7.repo [root@adccrac1 yum.repos.d]# mv public-yum-ol7.repo public-yum-ol7.repo.bak [root@adccrac1 yum.repos.d]# vi ol7.repo 创建 repo 文件 [ol7] name=oracle Linux Server 7.2 baseurl=file:///mnt/iso #gpgkey=file:///etc/pki/rpm-gpg/rpm-gpg-key-oracle gpgcheck=0 enabled=1 [root@adccrac1 yum.repos.d]# yum clean all Loaded plugins: ulninfo Cleaning repos: ol7 11 / 120

12 Cleaning up everything yum.repos.d]# yum.repos.d]# yum makecache Loaded plugins: ulninfo ol7 3.6 kb 00:00:00 (1/4): ol7/group_gz 134 kb 00:00:00 (2/4): ol7/filelists_db 3.5 MB 00:00:00 (3/4): ol7/primary_db 4.1 MB 00:00:00 (4/4): ol7/other_db 1.4 MB 00:00:00 Metadata Cache Created yum.repos.d]# yum list... yp-tools.x86_ el7 ol7 ypbind.x86_64 3: el7 ol7 ypserv.x86_ el7 ol7 yum-langpacks.noarch el7 ol7 yum-plugin-aliases.noarch el7 ol7 yum-plugin-changelog.noarch el7 ol7 yum-plugin-ovl.noarch el7 ol7 yum-plugin-tmprepo.noarch el7 ol7 yum-plugin-verify.noarch el7 ol7 yum-plugin-versionlock.noarch el7 ol7 yum-utils.noarch el7 ol7 zenity.x86_ el7 ol7 zip.x86_ el7 ol7 12 / 120

13 zlib.i el7 ol7 zlib-devel.i el7 ol7 zlib-devel.x86_ el7 ol7 zsh.x86_ el7 ol7 zziplib.i el7 ol7 zziplib.x86_ el7 ol7 6. 网络命令 ifconfig 不存在, 通 YUM 源安 net-tools 软件包解决 ~]# ifconfig -bash: ifconfig: command not found ~]# yum install -y net-tools Loaded plugins: ulninfo Resolving Dependencies --> Running transaction check ---> Package net-tools.x86_64 0: git.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================== Package Arch Version Repository Size ================================================================================================== Installing: 13 / 120

14 net-tools x86_ git.el7 ol7 303 k Transaction Summary ================================================================================================== Install 1 Package Total download size: 303 k Installed size: 917 k Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : net-tools git.el7.x86_64 1/1 Verifying : net-tools git.el7.x86_64 1/1 Installed: net-tools.x86_64 0: git.el7 Complete! [root@adccrac1 ~]# ifconfig enp0s3: flags=4163<up,broadcast,running,multicast> mtu 1500 inet netmask broadcast inet6 fe80::a00:27ff:fef4:e8b0 prefixlen 64 scopeid 0x20<link> ether 08:00:27:f4:e8:b0 txqueuelen 1000 (Ethernet) 14 / 120

15 RX packets bytes (4.3 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets bytes (31.6 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s8: flags=4163<up,broadcast,running,multicast> mtu 1500 inet netmask broadcast inet6 fe80::a00:27ff:fe03:6ffb prefixlen 64 scopeid 0x20<link> ether 08:00:27:03:6f:fb txqueuelen 1000 (Ethernet) RX packets 71 bytes 9814 (9.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 8 bytes 564 (564.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<up,loopback,running> mtu inet netmask inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) RX packets 12 bytes 1032 (1.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 12 bytes 1032 (1.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 15 / 120

16 7. 安装桌面环境 ~]# yum grouplist Loaded plugins: ulninfo There is no installed groups file. Maybe run: yum groups mark convert (see man yum) Available Environment Groups: Minimal Install Infrastructure Server File and Print Server Basic Web Server Virtualization Host Server with GUI <<== 安装这个软件包的组 Available Groups: Compatibility Libraries Console Internet Tools Development Tools Graphical Administration Tools Legacy UNIX Compatibility Scientific Support Security Tools Smart Card Support System Administration Tools System Management Done 16 / 120

17 ~]# yum -y groupinstall "Server with GUI"... yajl.x86_64 0: el7 yelp-libs.x86_64 1: el7 yelp-xsl.noarch 0: el7 zenity.x86_64 0: el7 Complete! 现在,GNOME 的图形界面就安装成功了 8. 在 /etc/hosts 文件中增加如下内容 #public-ip adccrac adccrac2 #private-ip adccrac1-priv adccrac2-priv #vip adccrac1-vip adccrac2-vip #scan-ip adccrac-scan 17 / 120

18 9. 通过 YUM 源安装 tigervnc-server ( 图形界面远程连接工具 ) [root@adccrac1 ~]# yum install -y tigervnc-server Loaded plugins: ulninfo Resolving Dependencies --> Running transaction check ---> Package tigervnc-server.x86_64 0: el7 will be installed --> Processing Dependency: tigervnc-server-minimal for package: tigervnc-server el7.x86_64 ================================================================================================== Package Arch Version Repository Size ================================================================================================== Installing: tigervnc-server x86_ el7 ol7 202 k Installing for dependencies: tigervnc-server-minimal x86_ el7 ol7 1.0 M Transaction Summary ================================================================================================== Install 1 Package (+47 Dependent packages) Total download size: 26 M Installed size: 83 M Downloading packages: Total 68 MB/s 26 MB 00:00:00 18 / 120

19 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : tigervnc-server-minimal el7.x86_64 44/48 Verifying : tigervnc-server-minimal el7.x86_64 12/48 Verifying : tigervnc-server el7.x86_64 15/48 Installed: tigervnc-server.x86_64 0: el7 Complete! 10. 配置 VNCSERVER 并设置开机自动启动该服务 (1) 配置 VNC Service 以下方法是新方法, 以前是要配置 /etc/sysconfig/vncservers, 现在第一步是将默认提供的文件复制到 /etc/systemd/system, 命令如下 [root@adccrac1 ~]# cp /lib/systemd/system/vncserver@:1.service /etc/systemd/system/vncserver@:1.service 接下来修改该配置文件 [root@adccrac1 ~]# vim /etc/systemd/system/vncserver@:1.service 19 / 120

20 将其中 <USER> 替换为你想要的用户名, 我这里是 root, 添加设置分辨率的参数 -geometry 1280x720, 所有内容如下 # The vncserver service unit file # # Quick HowTo: # 1. Copy this file to /etc/systemd/system/vncserver@:<display>.service # 2. Edit <USER> and vncserver parameters appropriately # ("runuser -l <USER> -c /usr/bin/vncserver %i -arg1 -arg2") # 3. Run `systemctl daemon-reload` # 4. Run `systemctl enable vncserver@:<display>.service` #... [Unit] Description=Remote desktop service (VNC) After=syslog.target network.target [Service] Type=forking # Clean any existing files in /tmp/.x11-unix environment ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 :' ExecStart=/sbin/runuser -l root -c "/usr/bin/vncserver %i -geometry 1280x720" PIDFile=/root/.vnc/%H%i.pid ExecStop=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 :' 20 / 120

21 [Install] WantedBy=multi-user.target 保存文件并退出 vim, 重新加载配置 ~]# systemctl daemon-reload (2) 设置开机启动 VNC Service ~]# systemctl enable (3) 设置 VNC 密码 ~]# vncserver You will require a password to access your desktops. Password: <<== 输入 adccrac Verify: <<== 输入 adccrac xauth: file /root/.xauthority does not exist New 'adccrac1:1 (root)' desktop is adccrac1:1 Creating default startup script /root/.vnc/xstartup Starting applications specified in /root/.vnc/xstartup 21 / 120

22 Log file is /root/.vnc/adccrac1:1.log ~]# vncserver -list TigerVNC server sessions: X DISPLAY # PROCESS ID : [root@adccrac1 ~]# ps -ef grep vnc root :41? 00:00:00 /usr/bin/xvnc :1 -desktop adccrac1:1 (root) -auth /root/.xauthority -geometry 1024x768 -rfbwait rfbauth /root/.vnc/passwd -rfbport fp catalogue:/etc/x11/fontpath.d -pn root :41? 00:00:00 /usr/bin/vncconfig -iconic root :44 pts/0 00:00:00 grep --color=auto vnc 后来在服务器上又安装了 VNCVIEWER (VNCSERVER 的客户端工具 ), 主要是因为当时在 Windows 上安装的 VncViewer 没有连上这台 Linux 服务器 ( 没有关闭防火墙服务 ), 只要先关闭了防火墙服务,Windows 上的 VncViewer 就可以连接到 Linux 服务器, 所以我把关闭防火墙服务放在了步骤 操作系统的磁盘和 ASM 共享磁盘容量规划 操作系统磁盘 : /dev/sda 共 200g, 关于如何分区, 在安装操作系统时选择自动分区选项 ASM 共享磁盘 : ( 在 VitualBox 里的磁盘大小必须是固定的, 否则无法从 " 正常 " 模式变为 " 共享 " 模式 ) /dev/sdb OCRVOTE1 2g 22 / 120

23 /dev/sdc OCRVOTE2 /dev/sdd OCRVOTE3 /dev/sde DATA /dev/sdf BACK /dev/sdg ARCH 2g 2g 15g 4g 4g 12. 用户 root 的密码和两节点数据库 REDO 个数 adccrac1 root 密码 : adccrac adccrac2 root 密码 : adccrac REDO LOG 的大小设为 256M 共设 6 组线程 1 是 3 组线程 2 也是 3 组 13. 创建用户和组 ( 两节点 ) groupadd -g 1100 oinstall groupadd -g 1200 dba groupadd -g 1300 oper groupadd -g 2100 asmadmin groupadd -g 2200 asmdba groupadd -g 2300 asmoper useradd -u 777 -g oinstall -G dba,oper,asmadmin,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle 23 / 120

24 echo "oracle" passwd --stdin oracle useradd -u 888 -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c "grid Infrastructure Owner" grid echo "grid" passwd --stdin grid 14. 配置 grid 用户和 oracle 用户的环境变量 ( 两节点 ) 节点一, su - oracle vim.bash_profile export ORACLE_BASE=/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1 export ORACLE_SID=racdb1 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH export PATH=$ORACLE_HOME/bin:$PATH export LANG=en_US.UTF-8 export DISPLAY= :0.0 su - oracle vim.bash_profile export ORACLE_BASE=/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1 export ORACLE_SID=racdb2 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH export PATH=$ORACLE_HOME/bin:$PATH export LANG=en_US.UTF-8 export DISPLAY= : / 120

25 保存退出, 用 source.bash_profile 重读一下配置文件节点二, su - grid vim.bash_profile export ORACLE_BASE=/app/grid export ORACLE_HOME=/app/11.2.0/grid export ORACLE_SID=+ASM1 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH export PATH=$ORACLE_HOME/bin:$PATH export LANG=en_US.UTF-8 export DISPLAY= :0.0 su - grid vim.bash_profile export ORACLE_BASE=/app/grid export ORACLE_HOME=/app/11.2.0/grid export ORACLE_SID=+ASM2 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH export PATH=$ORACLE_HOME/bin:$PATH export LANG=en_US.UTF-8 export DISPLAY= :0.0 保存退出, 用 source.bash_profile 重读一下配置文件 25 / 120

26 15. 创建所需的目录及赋予相应的权限 ( 两节点 ) mkdir -p /app/grid mkdir -p /app/11.2.0/grid mkdir -p /app/oracle chown -R oracle:oinstall /app chown -R grid:oinstall /app/grid chown -R grid:oinstall /app/ chmod -R 775 /app 16. 编辑 /etc/security/limits.conf 文件, 增加如下内容 ( 两节点 ) oracle soft nproc 2047 oracle hard nproc oracle soft nofile 1024 oracle hard nofile grid soft nproc 2047 grid hard nproc grid soft nofile 1024 grid hard nofile / 120

27 17. 编辑 /etc/pam.d/login 文件, 增加如下内容 ( 两节点 ) session required /lib64/security/pam_limits.so session required pam_limits.so 18. 编辑 /etc/profile 文件, 添加如下参数 ( 两节点 ) if [ $USER = "oracle" ] [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p ulimit -n else ulimit -u n fi fi 保存退出, 用 source /etc/profile 重读一下配置文件 19. 编辑 /etc/sysctl.conf 文件, 增加下列参数 ( 两节点 ) [Oracle Linux 7.2 下为这个文件 # vi /usr/lib/sysctl.d/00-system.conf] 就在文件 /etc/sysctl.conf 的行尾增加就可以, 这样到最后安装形界面的预检测时不会出现内核参数的警告提示 ; 如果将下面的参数写到文件 /usr/lib/sysctl.d/00-system.conf 就会有警告提示 27 / 120

28 kernel.msgmnb = kernel.msgmax = kernel.shmmax = kernel.shmall = kernel.shmmni = 4096 fs.aio-max-nr = fs.file-max = kernel.sem = net.ipv4.ip_local_port_range = net.core.rmem_default = net.core.rmem_max = net.core.wmem_default = net.core.wmem_max = 保存退出, 之后别忘了用 sysctl -p 命令使参数生效 [Oracle Linux 7.2 下 # sysctl -p /usr/lib/sysctl.d/00-system.conf] 20. 配置 SSH 互信, 在图形界面点击设置, 此步不用命令 21. 将 grid 和 oracle 的软件安装包上传到节点一的 /software 目录 [root@adccrac1 ~]# mkdir /software [root@adccrac1 ~]# cd /software/ [root@adccrac1 software]# ls -lrht total 3.5G 28 / 120

29 -rw-r--r-- 1 root root 1.1G Aug 15 09:50 p _112040_linux-x86-64_2of7.zip -rw-r--r-- 1 root root 1.3G Aug 15 09:51 p _112040_linux-x86-64_1of7.zip -rw-r--r-- 1 root root 1.2G Aug 15 09:53 p _112040_linux-x86-64_3of7.zip 解压软件包 [root@adccrac1 ~]# chown -R oracle:oinstall /software/ [root@adccrac1 ~]# su - grid Last login: Tue Aug 14 15:06:22 CST 2018 on pts/0 [grid@adccrac1 ~]$ cp /software/p _112040_linux-x86-64_3of7.zip./ [grid@adccrac1 ~]$ unzip p _112040_linux-x86-64_3of7.zip [root@adccrac1 ~]# su - oracle Last login: Wed Aug 15 12:24:47 CST 2018 on pts/0 [oracle@adccrac1 ~]$ cp /software/p _112040_linux-x86-64_1of7.zip./ [oracle@adccrac1 ~]$ cp /software/p _112040_linux-x86-64_2of7.zip./ [oracle@adccrac1 ~]$ ls p _112040_linux-x86-64_1of7.zip p _112040_linux-x86-64_2of7.zip [oracle@adccrac1 ~]$ unzip p _112040_linux-x86-64_1of7.zip; unzip p _112040_linux-x86-64_2of7.zip 22. 用 YUM 源安装 RAC 必需的 RPM 软件包 yum install -y binutils yum install -y compat-libcap1 yum install -y compat-libstdc el7.i / 120

30 yum install -y compat-libstdc++-33 yum install -y elfutils-libelf-devel yum install -y gcc yum install -y gcc-c++ yum install -y glibc el7.i686 yum install -y glibc yum install -y glibc-devel el7.i686 yum install -y glibc-devel yum install -y ksh yum install -y libaio el7.i686 yum install -y libaio yum install -y libaio-devel el7.i686 yum install -y libaio-devel yum install -y libgcc el7.i686 yum install -y libgcc yum install -y libstdc el7.i686 yum install -y libstdc++ yum install -y libstdc++-devel el7.i686 yum install -y libstdc++-devel yum install -y libxi el7.i686 yum install -y libxi yum install -y libxtst el7.i686 yum install -y libxtst yum install -y make yum install -y sysstat yum install -y unixodbc 30 / 120

31 yum install -y unixodbc el7.i686 yum install -y unixodbc-devel yum install -y unixodbc-devel el7.i686 yum remove y ksh 上传这个 RPM 软件包 pdksh el5.x86_64.rpm 到 /software rpm ivh /software/ pdksh el5.x86_64.rpm 切换到 grid 用户, 进入 grid/rpm 目录, 有一个 RPM 文件 cvuqdisk rpm, 不完全切换到 root 用户进行安装 [grid@adccrac1 ~]$ cd grid/ [grid@adccrac1 grid]$ ls install readme.html response rpm runcluvfy.sh runinstaller sshsetup stage welcome.html [grid@adccrac1 grid]$ cd rpm [grid@adccrac1 rpm]$ ls cvuqdisk rpm [grid@adccrac1 rpm]$ su Password: [root@adccrac1 rpm]# rpm -ivh cvuqdisk rpm Preparing... ################################# [100%] Using default group oinstall to install package Updating / installing... 1:cvuqdisk ################################# [100%] 31 / 120

32 23. 用 VirutalBox 克隆 adccrac1 为另一台主机 adccrac2, 选 Full Clone, 并选择 重新初始化两块网 卡的 MAC 地址 之后启动 adccrac2, 更改主机名为 adccrac2, ~]# hostnamectl Static hostname: adccrac1 Icon name: computer-vm Chassis: vm Machine ID: cd4b43fc434a4b949feedc2c31f0da3f Boot ID: e977d29e2a764a80b14a5aa68f18627d Virtualization: kvm Operating System: Oracle Linux Server 7.2 CPE OS Name: cpe:/o:oracle:linux:7:2:server Kernel: Linux el7uek.x86_64 Architecture: x86-64 ~]# hostnamectl status Static hostname: adccrac1 Icon name: computer-vm Chassis: vm Machine ID: cd4b43fc434a4b949feedc2c31f0da3f Boot ID: e977d29e2a764a80b14a5aa68f18627d Virtualization: kvm Operating System: Oracle Linux Server / 120

33 CPE OS Name: cpe:/o:oracle:linux:7:2:server Kernel: Linux el7uek.x86_64 Architecture: x86-64 ~]# hostnamectl set-hostname adccrac2 --static ~]# hostname adccrac2 用 Xshell 重新连接, 发现主机名已变为 adccrac2 Last login: Wed Aug 15 09:04: from ~]# 更改网卡 enp0s3 和 enp0s8 的 IP 地址 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s3 TYPE=Ethernet BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no NAME=enp0s3 UUID=f26a2d18-32d6-4a17-88de-68127c860b82 DEVICE=enp0s3 33 / 120

34 ONBOOT=yes IPADDR= <<== 改为 62 PREFIX=24 GATEWAY= IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_PRIVACY=no ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8 TYPE=Ethernet BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no NAME=enp0s8 UUID=973d91e4-913d-47ec-b117-9bc36190cd65 DEVICE=enp0s8 ONBOOT=yes IPADDR= <<== 改为 62 PREFIX=24 IPV6_PEERDNS=yes IPV6_PEERROUTES=yes IPV6_PRIVACY=no 34 / 120

35 重启网络服务, 使配置生效 ~]# systemctl restart network.service ~]# ~]# systemctl status network.service?.network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network) Active: active (exited) since Wed :06:32 CST; 8s ago Docs: man:systemd-sysv-generator(8) Process: 3535 ExecStop=/etc/rc.d/init.d/network stop (code=exited, status=0/success) Process: 3781 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=0/success) Aug 15 09:06:27 adccrac1 systemd[1]: Starting LSB: Bring up/down networking... Aug 15 09:06:27 adccrac1 network[3781]: Bringing up loopback interface: [ OK ] Aug 15 09:06:29 adccrac1 network[3781]: Bringing up interface enp0s3: [ OK ] Aug 15 09:06:32 adccrac1 network[3781]: Bringing up interface enp0s8: [ OK ] Aug 15 09:06:32 adccrac1 systemd[1]: Started LSB: Bring up/down networking. [root@adccrac1 ~]# ifconfig enp0s3 enp0s3: flags=4163<up,broadcast,running,multicast> mtu 1500 inet netmask broadcast inet6 fe80::a00:27ff:fe25:e0f1 prefixlen 64 scopeid 0x20<link> ether 08:00:27:25:e0:f1 txqueuelen 1000 (Ethernet) RX packets 394 bytes (39.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 35 / 120

36 TX packets 279 bytes (36.1 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@adccrac1 ~]# ifconfig enp0s8 enp0s8: flags=4163<up,broadcast,running,multicast> mtu 1500 inet netmask broadcast inet6 fe80::a00:27ff:fe69:ed64 prefixlen 64 scopeid 0x20<link> ether 08:00:27:69:ed:64 txqueuelen 1000 (Ethernet) RX packets 18 bytes 2830 (2.7 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 64 bytes (10.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 更改用户 oracle 和 grid 的.bash_profile 中的实例名为节点二的名称 [root@adccrac2 ~]# su - oracle Last login: Tue Aug 14 15:05:33 CST 2018 on pts/0 [oracle@adccrac2 ~]$ [oracle@adccrac2 ~]$ vi.bash_profile... export ORACLE_SID=racdb2 <<== 改为 racdb2 [oracle@adccrac2 ~]$ [oracle@adccrac2 ~]$ source.bash_profile [oracle@adccrac2 ~]$ exit logout 36 / 120

37 ~]# ~]# su - grid Last login: Tue Aug 14 15:06:22 CST 2018 on pts/0 [grid@adccrac2 ~]$ [grid@adccrac2 ~]$ vi.bash_profile... export ORACLE_SID=+ASM2 <<== 改为 +ASM2 [grid@adccrac2 ~]$ [grid@adccrac2 ~]$ source.bash_profile 24. 关闭两节点的主机, 在节点一上添加 6 块磁盘并将磁盘类型设置为 可共享, 在节点二上依次 添加节点一刚刚新增的磁盘, 最后将 6 块磁盘设置为共享模式, 并开启两个节点的主机 OCRVOTE1 2g OCRVOTE2 2g OCRVOTE3 2g DATA 15g BACK 4g ARCH 4g 25. 用 UDEV 方式绑定磁盘 ( 两节点 ) [root@adccrac1 ~]# for disk in `ls /dev/sd*` > do 37 / 120

38 > echo $disk > /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=$disk > done /dev/sdb 1ATA_VBOX_HARDDISK_VB8423dba6-263dd03e /dev/sdc 1ATA_VBOX_HARDDISK_VB4163a8df-b665ad13 /dev/sdd 1ATA_VBOX_HARDDISK_VB1d5f72c3-023cb880 /dev/sde 1ATA_VBOX_HARDDISK_VBd0feeeb6-de5ed8a4 /dev/sdf 1ATA_VBOX_HARDDISK_VBd2542fa2-9f0c4e98 /dev/sdg 1ATA_VBOX_HARDDISK_VBa0072bca-68bf2cbb # vi /etc/udev/rules.d/99-oracle-asmdevices.rules KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1ATA_VBOX_HARDDISK_VB8423dba6-263dd03e", SYMLINK+="asm-ocrvote1", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1ATA_VBOX_HARDDISK_VB4163a8df-b665ad13", SYMLINK+="asm-ocrvote2", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1ATA_VBOX_HARDDISK_VB1d5f72c3-023cb880", SYMLINK+="asm-ocrvote3", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1ATA_VBOX_HARDDISK_VBd0feeeb6-de5ed8a4", SYMLINK+="asm-data", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", 38 / 120

39 RESULT=="1ATA_VBOX_HARDDISK_VBd2542fa2-9f0c4e98", SYMLINK+="asm-back", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="1ATA_VBOX_HARDDISK_VBa0072bca-68bf2cbb", SYMLINK+="asm-arch", OWNER="grid", GROUP="asmadmin", MODE="0660" 重启 UDEV 服务 # udevadm control --reload systemctl restart systemd-udev-trigger.service 已整合到 UDEV 后台服务中了. systemd-udev-settle.service loaded active exited udev Wait for Complete Device Initialization systemd-udev-trigger.service loaded active exited udev Coldplug all Devices systemd-udevd.service loaded active running udev Kernel Device Manager systemd-udevd-control.socket loaded active running udev Control Socket systemd-udevd-kernel.socket loaded active running udev Kernel Socket 查看组和权限 [root@adccrac1 ~]# ls -ld /dev/asm-* ls: cannot access /dev/asm-*: No such file or directory [root@adccrac1 ~]# systemctl restart systemd-udev-trigger.service [root@adccrac1 ~]# ls -ld /dev/asm-* lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-arch -> sdg lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-back -> sdf lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-data -> sde lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-ocrvote1 -> sdb lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-ocrvote2 -> sdc 39 / 120

40 lrwxrwxrwx 1 root root 3 Aug 15 16:38 /dev/asm-ocrvote3 -> sdd [root@adccrac1 ~]# ls -ld /dev/sd* brw-rw root disk 8, 0 Aug 15 16:38 /dev/sda brw-rw root disk 8, 1 Aug 15 16:38 /dev/sda1 brw-rw root disk 8, 2 Aug 15 16:38 /dev/sda2 brw-rw grid asmadmin 8, 16 Aug 15 16:38 /dev/sdb brw-rw grid asmadmin 8, 32 Aug 15 16:38 /dev/sdc brw-rw grid asmadmin 8, 48 Aug 15 16:38 /dev/sdd brw-rw grid asmadmin 8, 64 Aug 15 16:38 /dev/sde brw-rw grid asmadmin 8, 80 Aug 15 16:38 /dev/sdf brw-rw grid asmadmin 8, 96 Aug 15 16:38 /dev/sdg 26. 对 grid 软件打 PSU 补丁 [grid@adccrac1 ~]$ unzip /software/p _112040_linux-x86-64.zip -d /tmp/ Archive: /software/p _112040_linux-x86-64.zip creating: /tmp/b / creating: /tmp/b /client/ inflating: /tmp/b /client/cvu_prereq.xml inflating: /tmp/b /client/oracle.client_instantclient.xml creating: /tmp/b /grid/ inflating: /tmp/b /grid/cvu_prereq.xml creating: /tmp/b /examples/ inflating: /tmp/b /examples/cvu_prereq.xml creating: /tmp/b /database/ 40 / 120

41 inflating: /tmp/b /database/cvu_prereq.xml inflating: /tmp/b /readme_b txt inflating: /tmp/patchsearch.xml ~]$ cp /tmp/b /grid/cvu_prereq.xml./grid/stage/cvu/ ~]$ cd./grid/stage/cvu/ cvu]$ ls -lrht total 48K drwxr-xr-x 5 grid oinstall 44 Aug cv -rwxr-xr-x 1 grid oinstall 45K Aug 22 10:36 cvu_prereq.xml 27. 在节点一打开 VNC 的图形桌面 用 Windows 里的 VncViewer 客户端工具连到 输入 :1 输入 VNC 密码 : adccrac 28. 由于存在 BUG , 安装 Oracle Grid Infrastructure 软件, 后面打完补丁后再配置 GI 在 Oracle 的官方文档 中的 有如下的一段描述 : 41 / 120

42 7.2.3 Oracle Grid Infrastructure Installation Issue During the Oracle Grid Infrastructure installation, you must apply patch before configuring the software that is installed. The timing of applying the patch is important and is described in detail in the Note on My Oracle Support. This patch ensures that the clusterware stack is configured to use systemd for clusterware processes, as Oracle Linux 7 uses systemd for all services. This issue is tracked with Oracle bug , which was logged for release , but the patch is for release 所以, 在步骤 27 当中的 VNC 桌面窗口打开一个 terminal 窗口, 切换到 grid 用户执行下面的命令 [grid@adccrac1 grid]$./runinstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual MB Passed Checking swap space: must be greater than 150 MB. Actual 4991 MB Passed Checking monitor: must be configured to display at least 256 colors. Actual Passed Preparing to launch Oracle Universal Installer from /tmp/orainstall _ am. Please wait...[grid@adccrac1 grid]$ 弹出图形界面 在第二个界面时, 选择第四个选项, 只安装 GI 软件 最后弹出执行 2 个脚本的对话框 这里, 实际上 grid 软件只安装到节点 1, 只在节点 1 上执行 : 42 / 120

43 # /app/orainventory/orainstroot.sh # /app/11.2.0/grid/root.sh [root@adccrac1 ~]# /app/orainventory/orainstroot.sh Changing permissions of /app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /app/orainventory to oinstall. The execution of the script is complete. [root@adccrac1 ~]# [root@adccrac1 ~]# /app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file / 120

44 Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user: /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/perl/lib -I/app/11.2.0/grid/crs/install /app/11.2.0/grid/crs/install/roothas.pl To configure Grid Infrastructure for a Cluster execute the following command: /app/11.2.0/grid/crs/config/config.sh This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media. [root@adccrac1 ~]# 29. 给 GI 软件打 PSU 补丁 以下操作只需要在节点 1 上完成, 节点 2 上还没有 grid 软件 (1) 更新 Opatch 工具 [root@adccrac1 ~]# ls -ld /app/11.2.0/grid/ drwxrwxr-x 57 grid oinstall 4096 Aug 22 10:52 /app/11.2.0/grid/ 44 / 120

45 ~]$ /app/11.2.0/grid/opatch/opatch version OPatch Version: OPatch succeeded. ~]$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch_ ~]$ unzip -q /software/p _112000_linux-x86-64.zip -d $ORACLE_HOME ~]$ ls -l $ORACLE_HOME grep OPatch drwxr-x grid oinstall 4096 Apr 16 16:20 OPatch drwxr-xr-x 8 grid oinstall 4096 Aug 22 10:50 OPatch_ [grid@adccrac1 ~]$ $ORACLE_HOME/OPatch/opatch version OPatch Version: OPatch succeeded. (2) 打补丁 [grid@adccrac1 ~]$ unzip /software/p _112040_linux-x86-64.zip -d /tmp/ [grid@adccrac1 ~]$ cd /tmp [grid@adccrac1 tmp]$ [grid@adccrac1 tmp]$ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp 45 / 120

46 OCM Installation Response Generator Production Copyright (c) 2005, 2012, Oracle and/or its affiliates. All rights reserved. Provide your address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support address/user Name. Visit for details. address/user Name: 回车 You have not provided an address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y The OCM configuration response file (ocm.rsp) was successfully created. [grid@adccrac ]$ $ORACLE_HOME/OPatch/opatch apply /tmp/ / -oh $ORACLE_HOME -ocmrf /tmp/ocm.rsp Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. Oracle Home : /app/11.2.0/grid Central Inventory : /app/orainventory from : /app/11.2.0/grid/orainst.loc OPatch version : OUI version : Log file location : /app/11.2.0/grid/cfgtoollogs/opatch/opatch _ am_1.log Verifying environment and performing prerequisite checks / 120

47 OPatch continues with these patches: Do you want to proceed? [y n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/app/11.2.0/grid') Is the local system ready for patching? [y n] y User Responded with: Y Backing up files... Applying interim patch ' ' to OH '/app/11.2.0/grid' Patching component oracle.crs, Patch successfully applied. Log file location: /app/11.2.0/grid/cfgtoollogs/opatch/opatch _ am_1.log OPatch succeeded. [grid@adccrac ]$ [grid@adccrac ]$ $ORACLE_HOME/OPatch/opatch lsinventory Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. 47 / 120

48 Oracle Home : /app/11.2.0/grid Central Inventory : /app/orainventory from : /app/11.2.0/grid/orainst.loc OPatch version : OUI version : Log file location : /app/11.2.0/grid/cfgtoollogs/opatch/opatch _ am_1.log Lsinventory Output file location : /app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory _ am.txt Local Machine Information:: Hostname: adccrac1 ARU platform id: 226 ARU platform description:: Linux x86-64 Installed Top-level Products (1): Oracle Grid Infrastructure 11g There are 1 products installed in this Oracle Home. Interim patches (1) : Patch : applied on Wed Aug 22 11:25:38 CST / 120

49 Unique Patch ID: Patch description: "Grid Infrastructure Patch Set Update : x (gibugno)" Created on 15 Aug 2014, 16:14:40 hrs PST8PDT Bugs fixed: OPatch succeeded. [grid@adccrac ]$ 30. 配置 Grid (1) 克隆 grid 软件到节点 2 节点一操作, # tar -cvpf grid_soft.tar /app/11.2.0/grid/ 节点二操作, # scp grid@adccrac1:/grid_soft.tar./ # tar -xvf grid_soft.tar 节点二执行下面两个脚本 # /app/orainventory/orainstroot.sh # /app/11.2.0/grid/root.sh 49 / 120

50 (2) 克隆 ORACLE_HOME 配置 节点 1: ~]# cd /app/orainventory/ orainventory]# rm -rf * [root@adccrac1 orainventory]# su - grid Last login: Wed Aug 22 11:09:37 CST 2018 on pts/3 [grid@adccrac1 ~]$ [grid@adccrac1 ~]$ cd $ORACLE_HOME/clone/bin [grid@adccrac1 bin]$ perl clone.pl -silent -debug ORACLE_BASE=/app/grid ORACLE_HOME=/app/11.2.0/grid ORACLE_HOME_NAME=Ora11g_gridinfrahome1 INVENTORY_LOCATION=/app/oraInventory OSDBA_GROUP=oinstall OSOPER_GROUP=dba -O'"CLUSTER_NODES={adccrac1,adccrac2}"' - O'"LOCAL_NODE=adccrac1"' CRS=TRUE -ignoresysprereqs./runinstaller -clone -waitforcompletion "ORACLE_BASE=/app/grid" "ORACLE_HOME=/app/11.2.0/grid" "ORACLE_HOME_NAME=Ora11g_gridinfrahome1" "INVENTORY_LOCATION=/app/oraInventory" "oracle_install_osdba=oinstall" "oracle_install_osoper=dba" "CLUSTER_NODES={adccrac1,adccrac2}" "LOCAL_NODE=adccrac1" "CRS=TRUE" -ignoresysprereqs -debug -silent -noconfig -nowait Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4991 MB Passed Preparing to launch Oracle Universal Installer from /tmp/orainstall _ pm. Please wait... LD_LIBRARY_PATH environment variable : Total args: 36 Command line argument array elements... Arg:0:/tmp/OraInstall _ PM/jre/bin/java: 50 / 120

51 Arg:1:-Doracle.installer.library_loc=/tmp/OraInstall _ PM/oui/lib/linux64: Arg:2:-Doracle.installer.oui_loc=/tmp/OraInstall _ PM/oui: Arg:3:-Doracle.installer.bootstrap=TRUE: Arg:4:-Doracle.installer.startup_location=/app/11.2.0/grid/oui/bin: Arg:5:-Doracle.installer.jre_loc=../../jre: Arg:6:-Doracle.installer.custom_inventory=/app/oraInventory: Arg:7:-Doracle.installer.nlsEnabled="TRUE": Arg:8:-Doracle.installer.prereqConfigLoc= : Arg:9:-Doracle.installer.unixVersion= el7uek.x86_64: Arg:10:-mx150m: Arg:11:-cp: Arg:12:/tmp/OraInstall _ PM::/tmp/OraInstall _ PM/ext/jlib/instcommon.jar:/tmp/OraInstall _ PM/ext/jlib/cvu.jar:/tmp/OraInstall _ PM/ext/jlib/installcommons_1.0.0b.jar:/tmp/OraInstall _ PM/ext/jlib/OraPrereq.jar:/tmp/OraInstall _ PM/ext/jlib/OraPrereqChecks.jar:/tmp/OraInstall _ PM/ext/jlib/jsch.jar:/tmp/OraInstall _ PM/ext/jlib/orai18n-mapping.jar:/tmp/OraInstall _ PM/ext/jlib/orai18n- utility.jar:/tmp/orainstall _ pm/ext/jlib/prov_fixup.jar:/tmp/orainstall _ pm/ext/jlib/ojdbc5.jar:/tmp/orainstall _ PM/ext/jlib/instcrs.jar:/tmp/OraInstall _ PM/ext/jlib/remoteinterfaces.jar:/tmp/OraInstall _ PM/ext/jlib/ssh.jar:/tmp/OraInstall _ PM/oui/jlib/OraInstaller.jar:/tmp/OraInstall _ PM/oui/jlib/oneclick.jar:/tmp/OraInstall _ PM/oui/jlib/xmlparserv2.jar:/tmp/OraInstall _ PM/oui/jlib/share.jar:/tmp/OraInstall _ PM/oui/jlib/OraInstallerNet.jar:/tmp/OraInstall _ PM/oui/jlib/emCfg.jar:/tmp/OraInstall _ PM/oui/jlib/emocmutl.jar:/tmp/OraInstall _ PM/oui/jlib/OraPrereq.jar:/tmp/OraInstall _ PM/oui/jlib/jsch.jar:/tmp/OraInstall _ PM/oui/jlib/ssh.jar:/tmp/OraInstall _ PM/oui/jlib/remoteinterfaces.jar:/tmp/OraInstall _ PM/oui/jlib/http_client.jar:/tmp/OraInstall _ PM/oui/jlib/OraSuiteInstaller.jar:/tmp/OraInstall _ PM/OPatch/jlib/opatch.jar:/tmp/OraInstall _ PM/OPatch/jlib/opatchactions.jar:/tmp/OraInstall _ PM/OPatch/jlib/opatchprereq.jar:/tmp/OraInstall _ PM/OPatch/jlib/opatchutil.jar:/tmp/OraInstall _ / 120

52 03PM/oui/jlib/OraCheckPoint.jar:/tmp/OraInstall _ PM/oui/jlib/InstImages.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_de.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_es.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_fr.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_it.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_ja.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_ko.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_pt_BR.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_zh_CN.jar:/tmp/OraInstall _ PM/oui/jlib/InstHelp_zh_TW.jar:/tmp/OraInstall _ PM/oui/jlib/oracle_ice.jar:/tmp/OraInstall _ PM/oui/jlib/help4.jar:/tmp/OraInstall _ PM/oui/jlib/help4- nls.jar:/tmp/orainstall _ pm/oui/jlib/ewt3.jar:/tmp/orainstall _ pm/oui/jlib/ewt3-swingaccess.jar:/tmp/orainstall _ PM/oui/jlib/ewt3-nls.jar:/tmp/OraInstall _ PM/oui/jlib/swingaccess.jar::/tmp/OraInstall _ PM/oui/jlib/jewt4.jar:/tmp/OraInstall _ PM/oui/jlib/jewt4-nls.jar:/tmp/OraInstall _ PM/oui/jlib/orai18n- collation.jar:/tmp/orainstall _ pm/oui/jlib/orai18n-mapping.jar:/tmp/orainstall _ PM/oui/jlib/ojmisc.jar:/tmp/OraInstall _ PM/oui/jlib/xml.jar:/tmp/OraInstall _ PM/oui/jlib/srvm.jar:/tmp/OraInstall _ PM/oui/jlib/srvmasm.jar: Arg:13:oracle.sysman.oii.oiic.OiicInstaller: Arg:14:-scratchPath: Arg:15:/tmp/OraInstall _ PM: Arg:16:-sourceType: Arg:17:network: Arg:18:-timestamp: Arg:19: _ PM: Arg:20:-clone: Arg:21:-waitForCompletion: Arg:22:ORACLE_BASE=/app/grid: Arg:23:ORACLE_HOME=/app/11.2.0/grid: Arg:24:ORACLE_HOME_NAME=Ora11g_gridinfrahome1: Arg:25:INVENTORY_LOCATION=/app/oraInventory: 52 / 120

53 Arg:26:oracle_install_OSDBA=oinstall: Arg:27:oracle_install_OSOPER=dba: Arg:28:CLUSTER_NODES={adccrac1,adccrac2}: Arg:29:LOCAL_NODE=adccrac1: Arg:30:CRS=TRUE: Arg:31:-ignoreSysPrereqs: Arg:32:-debug: Arg:33:-silent: Arg:34:-noConfig: Arg:35:-nowait: Initializing Java Virtual Machine from /tmp/orainstall _ pm/jre/bin/java. Please wait... Oracle Universal Installer, Version Production Copyright (C) 1999, 2013, Oracle. All rights reserved. You can find the log of this install session at: /app/orainventory/logs/cloneactions _ pm.log...[main] [ :37: CST ] [QueryCluster.<init>:56] [main] [ :37: CST ] [QueryCluster.isCluster:65] Cluster existence check = false % Done. No Cluster detected Installation in progress (Wednesday, August 22, :37:41 PM CST)... 69% Done. Install successful 53 / 120

54 Linking in progress (Wednesday, August 22, :37:49 PM CST) Link successful Setup in progress (Wednesday, August 22, :39:14 PM CST) % Done. Setup successful End of install phases.(wednesday, August 22, :39:53 PM CST) WARNING: The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. /app/11.2.0/grid/root.sh #On nodes adccrac1 To execute the configuration scripts: 1. Open a terminal window 2. Log in as "root" 3. Run the scripts in each cluster node Run the script on the local node first. After successful completion, you can run the script in parallel on all the other nodes. The cloning of Ora11g_gridinfrahome1 was successful. Please check '/app/orainventory/logs/cloneactions _ pm.log' for more details. copying /app/orainventory/logs/cloneactions _ pm.log to /app/11.2.0/grid/cfgtoollogs/oui/cloneactions _ pm.log copying /app/orainventory/logs/orainstall _ pm.out to /app/11.2.0/grid/cfgtoollogs/oui/orainstall _ pm.out copying /app/orainventory/logs/orainstall _ pm.err to /app/11.2.0/grid/cfgtoollogs/oui/orainstall _ pm.err copying /app/orainventory/logs/silentinstall _ pm.log to /app/11.2.0/grid/cfgtoollogs/oui/silentinstall _ pm.log 配置完成执行 :# /app/11.2.0/grid/root.sh 54 / 120

55 ~]# /app/11.2.0/grid/root.sh Check /app/11.2.0/grid/install/root_adccrac1_ _ log for the output of root script ~]# cat /app/11.2.0/grid/install/root_adccrac1_ _ log Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /app/11.2.0/grid Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user: /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/perl/lib -I/app/11.2.0/grid/crs/install /app/11.2.0/grid/crs/install/roothas.pl To configure Grid Infrastructure for a Cluster execute the following command: /app/11.2.0/grid/crs/config/config.sh This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the 55 / 120

56 response file that is available in the installation media. ~]# 节点 2: ~]# cd /app/orainventory/ orainventory]# rm -rf * [root@adccrac2 orainventory]# su - grid Last login: Wed Aug 22 11:15:37 CST 2018 on pts/3 [grid@adccrac2~]$ [grid@adccrac2 ~]$ cd $ORACLE_HOME/clone/bin [grid@adccrac2 bin]$ perl clone.pl -silent -debug ORACLE_BASE=/app/grid ORACLE_HOME=/app/11.2.0/grid ORACLE_HOME_NAME=Ora11g_gridinfrahome1 INVENTORY_LOCATION=/app/oraInventory OSDBA_GROUP=oinstall OSOPER_GROUP=dba -O'"CLUSTER_NODES={adccrac1,adccrac2}"' - O'"LOCAL_NODE=adccrac2"' CRS=TRUE -ignoresysprereqs 执行过程和节点一的相同, 此处省略 配置完成执行 :# /app/11.2.0/grid/root.sh (3) 进行 Cluster 配置 根据上一步, 最后日志的提示 : 如果是要配置单节点的 RAC, 需要在 root 用户下执行 : 56 / 120

57 /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/perl/lib -I/app/11.2.0/grid/crs/install /app/11.2.0/grid/crs/install/roothas.pl 如果是要配置集群, 需要执行在 grid 用户下执行 : /app/11.2.0/grid/crs/config/config.sh 这里先选择配置单节点的 RAC, 看是什么提示? [grid@adccrac1 ~]$ /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/perl/lib -I/app/11.2.0/grid/crs/install /app/11.2.0/grid/crs/install/roothas.pl You must be logged in as root to run this script. Log in as root and rerun this script :59:20: Not running as authorized user Insufficient privileges to execute this script [root@adccrac1 grid]# /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/perl/lib -I/app/11.2.0/grid/crs/install /app/11.2.0/grid/crs/install/roothas.pl Using configuration parameter file: /app/11.2.0/grid/crs/install/crsconfig_params User ignored Prerequisites during installation The oracle binary is currently linked with RAC enabled. Please execute the following steps to relink oracle binary and rerun the command with RAC disabled: setenv ORACLE_HOME <oracle_restart_home> cd <oracle_restart_home>/rdbms/lib make -f ins_rdbms.mk rac_off ioracle 现在选择集群的 RAC, 57 / 120

58 ~]$ /app/11.2.0/grid/crs/config/config.sh 回车 弹出图形界面 58 / 120

59 59 / 120

60 60 / 120

61 61 / 120

62 62 / 120

63 63 / 120

64 64 / 120

65 65 / 120

66 66 / 120

67 67 / 120

68 68 / 120

69 69 / 120

70 70 / 120

71 经过一系列的配置, 最后在 15% 的进度条时, 弹出执行脚本 /app/11.2.0/grid/root.sh 的对话框 ~]# /app/11.2.0/grid/root.sh Check /app/11.2.0/grid/install/root_adccrac1_ _ log for the output of root script ~]# tail -f /app/11.2.0/grid/install/root_adccrac1_ _ log Copying coraenv to /usr/local/bin... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /app/11.2.0/grid/crs/install/crsconfig_params User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys 71 / 120

72 peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding Clusterware entries to oracle-ohasd.service <<== 当出现这一行信息时, 打开另一个 session 窗口, 查看 CRS-2672: Attempting to start 'ora.mdnsd' on 'adccrac1' CRS-2676: Start of 'ora.mdnsd' on 'adccrac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'adccrac1' CRS-2676: Start of 'ora.gpnpd' on 'adccrac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'adccrac1' CRS-2672: Attempting to start 'ora.gipcd' on 'adccrac1' CRS-2676: Start of 'ora.cssdmonitor' on 'adccrac1' succeeded CRS-2676: Start of 'ora.gipcd' on 'adccrac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'adccrac1' CRS-2672: Attempting to start 'ora.diskmon' on 'adccrac1' CRS-2676: Start of 'ora.diskmon' on 'adccrac1' succeeded CRS-2676: Start of 'ora.cssd' on 'adccrac1' succeeded 72 / 120

73 ASM created and started successfully. Disk Group OCRVOTE created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4256: Updating the profile Successful addition of voting disk 6742c5d7880c4fd2bf2301e b. Successfully replaced voting disk group with +OCRVOTE. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group ONLINE 6742c5d7880c4fd2bf2301e b (/dev/asm-ocrvote1) [OCRVOTE] Located 1 voting disk(s). CRS-2672: Attempting to start 'ora.asm' on 'adccrac1' CRS-2676: Start of 'ora.asm' on 'adccrac1' succeeded CRS-2672: Attempting to start 'ora.ocrvote.dg' on 'adccrac1' CRS-2676: Start of 'ora.ocrvote.dg' on 'adccrac1' succeeded Configure Oracle Grid Infrastructure for a Cluster... succeeded [root@adccrac1 ~]# ls -ld /etc/init.d/* -rw-r--r--. 1 root root Sep /etc/init.d/functions 73 / 120

74 -rwxr-xr-x. 1 root root 2989 Sep /etc/init.d/netconsole -rwxr-xr-x. 1 root root 6630 Sep /etc/init.d/network -rw-r--r--. 1 root root 1160 Nov /etc/init.d/readme -rwxr-xr-x. 1 root root 2437 Nov /etc/init.d/rhnsd [root@adccrac1 ~]# ls -ld /etc/init.d/* -rw-r--r--. 1 root root Sep /etc/init.d/functions -rwxr-xr-x 1 root root 8758 Aug 22 13:14 /etc/init.d/init.ohasd <<== 已有这一行, 刚才没有 -rwxr-xr-x. 1 root root 2989 Sep /etc/init.d/netconsole -rwxr-xr-x. 1 root root 6630 Sep /etc/init.d/network -rwxr-xr-x 1 root root 6709 Aug 22 13:14 /etc/init.d/ohasd <<== 已有这一行, 刚才没有 -rw-r--r--. 1 root root 1160 Nov /etc/init.d/readme -rwxr-xr-x. 1 root root 2437 Nov /etc/init.d/rhnsd [root@adccrac1 ~]# systemctl status ohasd?.ohasd.service - (null) Loaded: loaded (/etc/rc.d/init.d/ohasd) Active: inactive (dead) Docs: man:systemd-sysv-generator(8) [root@adccrac1 ~]# ps -ef grep ohasd root :14? 00:00:00 /bin/sh /etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null root :14? 00:00:04 /app/11.2.0/grid/bin/ohasd.bin reboot root :18 pts/2 00:00:00 grep --color=auto ohasd 74 / 120

75 ~]# ps -ef grep d.bin root :14? 00:00:05 /app/11.2.0/grid/bin/ohasd.bin reboot grid :16? 00:00:00 /app/11.2.0/grid/bin/oraagent.bin grid :16? 00:00:00 /app/11.2.0/grid/bin/mdnsd.bin grid :16? 00:00:00 /app/11.2.0/grid/bin/gpnpd.bin root :16? 00:00:00 /app/11.2.0/grid/bin/cssdmonitor grid :16? 00:00:01 /app/11.2.0/grid/bin/gipcd.bin root :16? 00:00:00 /app/11.2.0/grid/bin/cssdagent grid :16? 00:00:01 /app/11.2.0/grid/bin/ocssd.bin root :17? 00:00:00 /app/11.2.0/grid/bin/orarootagent.bin root :17? 00:00:00 /app/11.2.0/grid/bin/octssd.bin root :17? 00:00:02 /app/11.2.0/grid/bin/osysmond.bin root :17? 00:00:01 /app/11.2.0/grid/bin/ologgerd -M -d /app/11.2.0/grid/crf/db/adccrac1 root :18? 00:00:02 /app/11.2.0/grid/bin/crsd.bin reboot grid :18? 00:00:00 /app/11.2.0/grid/bin/evmd.bin grid :18? 00:00:00 /app/11.2.0/grid/bin/evmlogger.bin -o /app/11.2.0/grid/evm/log/evmlogger.info -l /app/11.2.0/grid/evm/log/evmlogger.log grid :18? 00:00:00 /app/11.2.0/grid/bin/oraagent.bin root :19? 00:00:00 /app/11.2.0/grid/bin/orarootagent.bin grid :19? 00:00:00 /app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit root :19 pts/0 00:00:00 sh -c /bin/su grid -c ' /app/11.2.0/grid/bin/srvctl start oc4j ' 2>&1 root :19 pts/0 00:00:00 /bin/su grid -c /app/11.2.0/grid/bin/srvctl start oc4j grid :19? 00:00:00 /bin/sh /app/11.2.0/grid/bin/srvctl start oc4j grid :19? 00:00:00 /app/11.2.0/grid/bin/scriptagent.bin grid :19? 00:00:00 /bin/sh /app/11.2.0/grid/bin/oc4jctl start grid :19? 00:00:00 /app/11.2.0/grid/perl/bin/perl -I/app/11.2.0/grid/lib /app/11.2.0/grid/bin/oc4jctl.pl start 75 / 120

76 root :19 pts/2 00:00:00 grep --color=auto d.bin 31. 检查 GI ( 两节点都检查 ) # su - grid $ crsctl check crs $ crs_stat -t $ olsnodes -n $ srvctl status listener $ srvctl status asm -a $ ocrcheck $ crsctl query css votedisk ===================================================================================================== [grid@adccrac1 ~]$ crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [grid@adccrac1 ~]$ [grid@adccrac1 ~]$ crs_stat -t Name Type Target State Host ora.arch.dg ora...up.type ONLINE ONLINE adccrac1 ora.back.dg ora...up.type ONLINE ONLINE adccrac1 76 / 120

77 ora.data.dg ora...up.type ONLINE ONLINE adccrac1 ora...er.lsnr ora...er.type ONLINE ONLINE adccrac1 ora...n1.lsnr ora...er.type ONLINE ONLINE adccrac1 ora.ocrvote.dg ora...up.type ONLINE ONLINE adccrac1 ora...sm1.asm application ONLINE ONLINE adccrac1 ora...c1.lsnr application ONLINE ONLINE adccrac1 ora...ac1.gsd application OFFLINE OFFLINE ora...ac1.ons application ONLINE ONLINE adccrac1 ora...ac1.vip ora...t1.type ONLINE ONLINE adccrac1 ora...sm2.asm application ONLINE ONLINE adccrac2 ora...c2.lsnr application ONLINE ONLINE adccrac2 ora...ac2.gsd application OFFLINE OFFLINE ora...ac2.ons application ONLINE ONLINE adccrac2 ora...ac2.vip ora...t1.type ONLINE ONLINE adccrac2 ora.asm ora.asm.type ONLINE ONLINE adccrac1 ora.cvu ora.cvu.type ONLINE ONLINE adccrac1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora...network ora...rk.type ONLINE ONLINE adccrac1 ora.oc4j ora.oc4j.type ONLINE ONLINE adccrac1 ora.ons ora.ons.type ONLINE ONLINE adccrac1 ora.scan1.vip ora...ip.type ONLINE ONLINE adccrac1 ~]$ ~]$ olsnodes -n adccrac1 1 adccrac2 2 [grid@adccrac1 ~]$ 77 / 120

78 ~]$ srvctl status listener Listener LISTENER is enabled Listener LISTENER is running on node(s): adccrac2,adccrac1 ~]$ ~]$ srvctl status asm -a ASM is running on adccrac2,adccrac1 ASM is enabled. ~]$ ~]$ ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : Used space (kbytes) : 2644 Available space (kbytes) : ID : Device/File Name : +OCRVOTE Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured 78 / 120

79 Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged user ~]$ ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group ONLINE 6742c5d7880c4fd2bf2301e b (/dev/asm-ocrvote1) [OCRVOTE] Located 1 voting disk(s). 79 / 120

80 32. 通过 asmca 命令创建 ASM 磁盘的 data back arch 3 个磁盘组 ( 只在节点一上做 ) 80 / 120

81 81 / 120

82 82 / 120

83 83 / 120

84 84 / 120

85 33. 安装 DB ( 只在节点一上做 ) 只安装软件, 不建库 (1) 对 db 软件打补丁 [root@adccrac1 ~]# su - oracle Last login: Wed Aug 22 13:05:07 CST 2018 on pts/0 [oracle@adccrac1 ~]$ [oracle@adccrac1 ~]$ cd /tmp/b / [oracle@adccrac1 b ]$ cp database/cvu_prereq.xml ~/database/stage/cvu/ (2) 安装 [root@adccrac1 ~]# su - oracle Last login: Wed Aug 22 13:32:35 CST 2018 on pts/5 [oracle@adccrac1 ~]$ [oracle@adccrac1 ~]$ cd database/ [oracle@adccrac1 database]$ ls -lrht total 48K -rwxr-xr-x 1 oracle oinstall 3.2K Aug runinstaller [oracle@adccrac1 database]$./runinstaller 到 56% 的进度条时, 弹出一个报错对话框 85 / 120

86 这个问题是 Bug , 参考 Installation of Oracle Database Software on OL7 fails with undefined reference to symbol B_DestroyKeyObject error ( 文档 ID ) Oracle 建议 Ignore, 安装完后打 Patch 其实不用, 另开一个窗口, 或者直接修改 ins_emagent.mk 文件 86 / 120

87 给文件 ins_emagent.mk 的 MK_EMAGENT_NMECTL 行尾增加参数 -lnnz11 $ vi $ORACLE_HOME/sysman/lib/ins_emagent.mk #=========================== # emdctl #=========================== $(SYSMANBIN)emdctl: $(MK_EMAGENT_NMECTL) 修改为 : #=========================== # emdctl #=========================== $(SYSMANBIN)emdctl: $(MK_EMAGENT_NMECTL) -lnnz11 然后点击 Retry 继续安装, 最后成功 34. 在 DB 上安装 PSU (1) 介质 p _112000_linux-x86-64.zip p _112040_linux-x86-64.zip p _112040_linux-x86-64.zip 87 / 120

88 都已上传到 /software 目录 由于 grid 软件 Opatch 已打, 此处不需要再升级 Grid 的 Opatch, 只需要升级 Db 的 OPatch (2) 升级 Opatch 工具 ~]$ $ORACLE_HOME/OPatch/opatch version OPatch Version: OPatch succeeded. ~]$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch_ ~]$ unzip -q /software/p _112000_linux-x86-64.zip -d $ORACLE_HOME ~]$ ~]$ $ORACLE_HOME/OPatch/opatch version OPatch Version: OPatch succeeded. 88 / 120

89 (3) 安装 PSU 先安装 PSU ~]$ $ORACLE_HOME/OPatch/opatch apply /tmp/ / -oh $ORACLE_HOME -ocmrf /tmp/ocm.rsp Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. Oracle Home : /app/oracle/product/11.2.0/db_1 Central Inventory : /app/orainventory from : /app/oracle/product/11.2.0/db_1/orainst.loc OPatch version : OUI version : Log file location : /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch _ pm_1.log Verifying environment and performing prerequisite checks... UtilSession failed: Patch requires component(s) that are not installed in OracleHome. These not-installed components are oracle.crs: , Log file location: /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch _ pm_1.log OPatch failed with error code 73 [oracle@adccrac1 ~]$ $ORACLE_HOME/OPatch/opatch apply /tmp/ /custom/server/ / -oh $ORACLE_HOME -ocmrf /tmp/ocm.rsp Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. 89 / 120

90 Oracle Home : /app/oracle/product/11.2.0/db_1 Central Inventory : /app/orainventory from : /app/oracle/product/11.2.0/db_1/orainst.loc OPatch version : OUI version : Log file location : /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch _ pm_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: Do you want to proceed? [y n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/app/oracle/product/11.2.0/db_1') Is the local system ready for patching? [y n] y User Responded with: Y Backing up files... Applying interim patch ' ' to OH '/app/oracle/product/11.2.0/db_1' 90 / 120

91 Patching component oracle.rdbms, Patch successfully applied. Log file location: /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch _ pm_1.log OPatch succeeded. ~]$ $ORACLE_HOME/OPatch/opatch lsinventory Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. Oracle Home : /app/oracle/product/11.2.0/db_1 Central Inventory : /app/orainventory from : /app/oracle/product/11.2.0/db_1/orainst.loc OPatch version : OUI version : Log file location : /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch _ pm_1.log Lsinventory Output file location : /app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory _ pm.txt Local Machine Information:: Hostname: adccrac1 ARU platform id: 226 ARU platform description:: Linux x / 120

92 Installed Top-level Products (1): Oracle Database 11g There are 1 products installed in this Oracle Home. Interim patches (1) : Patch : applied on Wed Aug 22 14:27:38 CST 2018 Unique Patch ID: Patch description: "Grid Infrastructure Patch Set Update : x (gibugno)" Created on 15 Aug 2014, 16:14:32 hrs PST8PDT Bugs fixed: OPatch succeeded. 再安装 PSU 使用 grid 用户解压 92 / 120

93 ~]$ unzip -q /software/p _112040_linux-x86-64.zip -d /tmp/ replace /tmp/patchsearch.xml? [y]es, [n]o, [A]ll, [N]one, [r]ename: y [root@adccrac1 tmp]# ls -lrht total 272K drwxr-xr-x 6 grid oinstall 87 Aug b rw-r--r-- 1 grid oinstall 267 Aug bundle.xml -rw-r--r-- 1 grid oinstall 50K Aug README.txt -rw-r--r-- 1 grid oinstall 90K Aug README.html drwxr-xr-x 5 grid oinstall 41 Aug drwxr-xr-x 5 grid oinstall 125 Jul <<== 这个补丁目录 -rw-rw-r-- 1 grid oinstall 109K Jul PatchSearch.xml drwxr-xr-x 2 grid oinstall 6 Aug 22 10:43 logs -rw-r--r-- 1 grid oinstall 621 Aug 22 11:14 ocm.rsp drwxr-xr-x 2 grid oinstall 54 Aug 22 12:22 Logs drwxr-xr-x 2 root root 6 Aug 22 13:19 hsperfdata_root drwxr-xr-x 3 grid oinstall 4.0K Aug 22 13:21 CVU_ _grid drwxr-xr-x 2 grid oinstall 18 Aug 22 13:24 hsperfdata_grid drwxr-xr-x 3 oracle oinstall 4.0K Aug 22 13:42 CVU_ _oracle drwxr-xr-x 2 oracle oinstall 6 Aug 22 14:28 hsperfdata_oracle [grid@adccrac1 ~]$ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -output /tmp/ /ocm.rsp OCM Installation Response Generator Production Copyright (c) 2005, 2012, Oracle and/or its affiliates. All rights reserved. 93 / 120

94 Provide your address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support address/user Name. Visit for details. address/user Name: 回车 You have not provided an address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y The OCM configuration response file (/tmp/ /ocm.rsp) was successfully created. [root@adccrac ]# ls -lrht total 152K drwxr-xr-x 5 grid oinstall 58 Feb drwxrwxr-x 13 grid oinstall 4.0K Jun rw-rw-r-- 1 grid oinstall 71K Jun PatchSearch.xml drwxr-xr-x 5 grid oinstall 58 Jul rw-r--r-- 1 grid oinstall 549 Jul bundle.xml -rw-rw-r-- 1 grid oinstall 63K Jul README.html -rw-rw-r-- 1 grid oinstall 24 Jul README.txt -rw-r--r-- 1 grid oinstall 621 Aug 22 14:53 ocm.rsp <<== 刚刚生成的 ocm.rsp 文件 检查补丁冲突 [grid@adccrac1 ~]$ opatch prereq CheckConflictAgainstOHWithDetail -ph /tmp/ / Oracle Interim Patch Installer version Copyright (c) 2018, Oracle Corporation. All rights reserved. 94 / 120

95 PREREQ session Oracle Home : /app/11.2.0/grid Central Inventory : /app/orainventory from : /app/11.2.0/grid/orainst.loc OPatch version : OUI version : Log file location : /app/11.2.0/grid/cfgtoollogs/opatch/opatch _ pm_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkconflictagainstohwithdetail" passed. OPatch succeeded. 在 grid 用户下使用 opatch auto 命令自动打 PSU [grid@adccrac1 ~]$ su Password: [root@adccrac1 grid]# $ORACLE_HOME/OPatch/opatch auto /tmp/ / -ocmrf /tmp/ /ocm.rsp Executing /app/11.2.0/grid/perl/bin/perl /app/11.2.0/grid/opatch/crs/patch11203.pl -patchdir /tmp -patchn ocmrf /tmp/ /ocm.rsp -paramfile /app/11.2.0/grid/crs/install/crsconfig_params This is the main log file: /app/11.2.0/grid/cfgtoollogs/opatchauto _ log 95 / 120

96 This file will show your detected configuration and all the steps that opatchauto attempted to do on your system: /app/11.2.0/grid/cfgtoollogs/opatchauto _ report.log :56:13: Starting Clusterware Patch Setup Using configuration parameter file: /app/11.2.0/grid/crs/install/crsconfig_params Stopping CRS... Stopped CRS successfully patch /tmp/ / apply successful for home /app/11.2.0/grid patch /tmp/ / apply successful for home /app/11.2.0/grid patch /tmp/ / apply successful for home /app/11.2.0/grid Starting CRS... Installing Trace File Analyzer 最后, 查看 PSU 补丁信息 ~]$ opatch lspatches ;ACFS Patch Set Update : ( ) ;OCW Patch Set Update : ( ) ;Database Patch Set Update : ( ) OPatch succeeded. 96 / 120

97 35. 使用 DBCA 命令在节点一, 创建 RAC 数据库 racdb 97 / 120

98 98 / 120

99 99 / 120

100 100 / 120

101 101 / 120

102 102 / 120

103 103 / 120

104 104 / 120

105 105 / 120

106 106 / 120

107 107 / 120

108 108 / 120

109 109 / 120

110 110 / 120

111 111 / 120

112 112 / 120

113 113 / 120

114 114 / 120

115 115 / 120

116 36. 设置归档目录, 并重启数据库到 mount 状态, 打开归档模式, 最后打开数据库 ~]$ sqlplus / as sysdba SQL> alter system set log_archive_dest_1='location=+arch'; System altered. 在节点一和节点二上查看, 已经发现归档目录已经设置好 NAME TYPE VALUE log_archive_dest_1 string location=+arch 现在关闭数据库, 依次在两个节点上进行操作, 确保整个集群数据库正常关闭节点一, SQL> shutdown immediate; 节点二, SQL> shutdown immediate; 在节点一, 启动数据库到 mount 状态, 开启归档模式, 最后正常打开数据库 SQL> startup mount; SQL> archive log list; SQL> alter database archivelog; SQL> archive log list; 116 / 120

117 SQL> alter database open; 此时, 在节点二上直接打开数据库 SQL> startup; 117 / 120

118 37. 在 grid 用户下关闭集群服务之后, 重新启动正常 118 / 120