6

Now we begin script based installation. We will run scripts one by one, and comment their action. You can see scripts sources and run scripts or reproduce the same manually..

6.1) Login to both nodes.

Became ROOT. (use ‘sux –‘ if you login as another user).

6.2) Verify CONFIG.sh once again.

This file contains configuration for our installation scripts. It is in reality standard shell profile. I marked by bold lines, which are very likely to be changed.

6.3) 010-install-orarun.sh - installs orarun, unlock oracle user and set up password.

On both nodes, run script 010 – ‘sh 010*.sh’. Enter new password for user 'oracle'.

File 010-install-orarun.sh:

#!/bin/bash

# Install orarun package

#
. CONFIG.sh

rpm -U $RPMORARUN

passwd oracle

chsh -s /bin/bash oracle

Script:

- Install orarun RPM;

- Unlocks ‘oracle’ user by setting password and changing shell to executable one.

Result:

testrac12:/INST # sh 010*
Updating etc/sysconfig/oracle...
Changing password for oracle.
New password:
Bad password: it is based on a dictionary word
Re-enter new password:
Password changed
Changing login shell for oracle.
Shell changed.

6.4) 015-edit-oraprofile.sh - copy our version of oracle profile and edit it (set up SID).

On both nodes, run script 15: sh 015*sh

File 015-edit-oraprofile.sh:

#
#       Edit /etc/profile.d/oracle.sh and oracle.sh
#
. CONFIG.sh
test -f /etc/profile.d/oracle.sh-ORIG && cp /etc/profile.d/oracle.sh /etc/profile.d/oracle.sh-ORIG
sed "s+ORACLE_SID=test1+ORACLE_SID=$SID+" FILES.d/oracle.sh > /tmp/oracle.sh && cp /tmp/oracle.sh /etc/profile.d/oracle.sh
chmod a+x /etc/profile.d/oracle.sh
chown oracle /etc/profile.d/oracle.sh
cd /etc/profile.d
if [ ! -f ~oracle/.profile ]
then
     cd ~oracle || exit 1
     cp -r /etc/skel/.??* ~oracle
     touch .profile
     find . -print | xargs chown oracle
fi

Script:

- Make reserved copy of orarun profile file;

- Install our version of this file (see appendix);

- Set up ORACLE_SID variable (notice, that it is different on different nodes);

- Make this script editable for oracle user (not required but simplify many things);

- Copy standard files into ~oracle (orarun creates user with empty directory);

- Change owner for these files.

Results (SIDs must be different):

testrac12:/INST # sh 015*sh
testrac12:/INST # grep SID /etc/profile.d/oracle.sh
ORACLE_SID=test2
export ORACLE_BASE ORACLE_HOME ORA_NLS33 ORACLE_SID PATH LD_LIBRARY_PATH CLASSPATH TNS_ADMIN

6.5) 018-edit-rcoracle.sh - copy our version of oracle startup files for Oracle10 cluster.

On both nodes, run script 018.

File 018-edit-rcoracle.sh:

#!/bin/bash
#
#       Edit /etc/init.d/oracle
#
. CONFIG.sh
if [ ! -d /etc/init.d/ORIG ]
then
    mkdir -p /etc/init.d/ORIG
    cp /etc/init.d/oracle /etc/init.d/ORIG/.
fi
if [ ! -d /etc/sysconfig/ORIG ]
then
    mkdir -p /etc/sysconfig/ORIG
    cp /etc/sysconfig/oracle /etc/sysconfig/ORIG/.
fi
#
cp FILES.d/init.oracle /etc/init.d/oracle
cp FILES.d/sysconfig.oracle /etc/sysconfig/oracle

Script:

- Makes reserved copy of init script and sysconfig file;

- Install our version of these files (see appendix);

Results:

testrac11:/INST # sh 018*sh
testrac11:/INST #

6.6) 020-start-rcoracle.sh - start rcoracle

(We want to set up system variables, because nothing is installed yet).
On both nodes, run script 020:

File 020-start-rcoracle.sh:

#!/bin/bash
#
# Run rcoracle to set up kernel parameters
#
. CONFIG.sh
rcoracle start

Result:

testrac11:/INST # sh 020*sh

#############################################################################
#                 Begin of   O R A C L E   startup section                  #
#############################################################################

ORACLE_HOME directory /opt/oracle/product/10.1.0/db_1 does not exist!
Unsetting ORACLE_HOME, will try to determine it from system...
ORACLE_HOME environment variable not set.
Check /etc/profile.d/oracle.sh and /etc/oratab
Cannot find ORACLE_HOME directory .
Environment settings are wrong! Check /etc/profile.d/oracle.sh
SETTINGS start from /etc/sysconfig/oracle
- Set Kernel Parameters for Oracle:   yes
- Start Oracle OCFS:                  yes
- Start Oracle CRS:                   yes
- Start Oracle EM:                    yes
Can't find needed file: emctl - Setting START_ORACLE_DB_EM = no
Can't find needed file: init.crs - Setting START_ORACLE_DB_CRS = no
Can't find needed file: /sbin/load_ocfs - Setting START_ORACLE_DB_OCFS = no

Setting kernel parameters for Oracle, see file
/etc/sysconfig/oracle for explanations.

Shared memory:      SHMMAX=3294967296 SHMMNI=4096 SHMALL=2097152
Semaphore values:   SEMMSL=1250 SEMMNS=32000 SEMOPM=100 SEMMNI=256
Other values:       FILE_MAX_KERNEL=131072 IP_LOCAL_PORT_RANGE=1024 65000

ULIMIT values:      MAX_CORE_FILE_SIZE_SHELL=unlimited
                    FILE_MAX_SHELL=65536 PROCESSES_MAX_SHELL=16384

Kernel parameters set for Oracle:                               done

- Starting Oracle Cluster Filesystem...                       skipped
- Starting Oracle CRS...                                      skipped

- Starting Oracle EM dbconsole...                             skipped

#############################################################################
#                      End of   O R A C L E   section                       #
#############################################################################

testrac11:/INST #

Script runs rcoracle to set up kernel variable. Do not surprise, if it complain about absent applications – you did not install anything yet.

6.7) 030-check-oracle-user.sh - this script verify Oracle user and location of Oracle installation disk.

Run it on all nodes.

File 030-check-oracle-user.sh:

#!/bin/bash

# Check oracle user setting

. CONFIG.sh

if [ ! -x $ORA10CD1/runInstaller ]

then

echo -e "\33[31m *** Wrong oracle disk location - no file $ORA10CD1/runInstaller found \33[30m"

exit 1

if [ ! -x $CRS10CD1/runInstaller ]

then

echo -e "\33[31m *** Wrong CRS disk location - no file $CRS10CD1/runInstaller found \33[30m"

exit 1

echo "*** CHECK ORACLE VARIABLES *** "

sux - -c "env" oracle | grep ORACLE

Results:

testrac11:/INST # sh 030*sh
*** CHECK ORACLE VARIABLES ***
xauth: creating new authority file /opt/oracle/.Xauthority
ORACLE_SID=test1
ORACLE_BASE=/opt/oracle
ORACLE_HOME=/opt/oracle/product/10.1.0/db_1

ORACLE_SID must be different on different nodes.

(Here we run NFS mounting script, when installed OracleRAC on NetApp NFS. We skip it in case of ASM.)

6.8) 060-setup-xntp.sh - set up XNTP service (important to synchronize clock on all nodes).

It starts yast2 ntp-client. Please, configure xntp and ensure that it is started in boot time.

File 060-setup-xntp.sh:

#
# Now we use this script only to start xntpd
#
. CONFIG.sh
yast2 xntp-client

/etc/init.d/xntpd status

Just setting XNTP server. It is important to have time synchronized on all RAC nodes.

6.9) 070-edit-services.sh - script removes conflicting records from /etc/services

(I am not sure if it is still necessary. Anyway, we will run it. Be careful, if you use NIS or LDAP, update them after running script and verify, that they do not keep removed records.).
Run on all nodes.

File 070-edit-services.sh:

#
. CONFIG.sh
sed '/net8-cman/d' /etc/services > /etc/services- && cp /etc/services- /etc/services && rm /etc/services-
ls -l /etc/services

Script removes net8-cman records from /etc/services.

Results:

testrac11:/INST # sh 070*sh
-rw-r--r-- 1 root root 596317 Feb 28 22:17 /etc/services

6.10) 080-ssh-genkeys.sh - set up password-less access between nodes.

Oracle RAC installer require, that user oracle can run ‘ssh’ between RAC nodes without password and without any warnings, prompts and additional output. This sript set up such access, using RCA keys.Do not use this script if you expose your Oracle system to public internet.

File 080-ssh-genkeys.sh:

#
#       Generate ssh rsa and dsa keys for password-less access
#       You may want to remove this after you wil complete oracle installation
#       Or use ssh-agent and passphrase for better security
#
#
. CONFIG.sh
#
# Check DNS or HOSTS first
#
for i in $NODES $NODESP
do
    echo testing $i
    if ! ping -q -c 1 $i
    then
        echo "*** BAD NODE NAME $i, fix DNS, hosts or NODES list"
        exit 1
    fi
done
su - oracle -c "ssh-keygen -t rsa -b 2048"
su - oracle -c "ssh-keygen -t dsa -b 2048"
#
#
for i in $NODES
do
echo Copying keys to node $i
cat ~oracle/.ssh/*pub | ssh $i 'su - oracle -c "mkdir -p .ssh;touch .ssh/authorized_keys; cat >> .ssh/authorized_keys"'
su - oracle -c "ssh $i date"
echo done
done

Run script on both nodes (first on node1, then on node2), enter EMPTY passphrase (4 times), enter root password (2 times), and answer 'yes' on requests.
Example (I marked answers as <answer> - you will need to enter root password few times. Do not enter passphrase, keep it empty):

testrac11:/INST # sh 080*sh
testing testrac11
PING testrac11 (10.25.32.111) 56(84) bytes of data.

--- testrac11 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
testing testrac12
PING testrac12 (10.25.32.112) 56(84) bytes of data.

--- testrac12 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.311/0.311/0.311/0.000 ms
testing testrac11-1
PING testrac11-1 (10.253.32.111) 56(84) bytes of data.

--- testrac11-1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.040/0.040/0.040/0.000 ms
testing testrac12-1
PING testrac12-1 (10.253.32.112) 56(84) bytes of data.

--- testrac12-1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.208/1.208/1.208/0.000 ms
Generating public/private rsa key pair.
Enter file in which to save the key (/opt/oracle/.ssh/id_rsa):      <ENTER>
Created directory '/opt/oracle/.ssh'.
Enter passphrase (empty for no passphrase):                <ENTER>
Enter same passphrase again:                               <ENTER>
Your identification has been saved in /opt/oracle/.ssh/id_rsa.
Your public key has been saved in /opt/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
ad:db:92:f3:40:26:d6:d5:6d:95:89:b8:03:64:b8:7d oracle@testrac11
Generating public/private dsa key pair.
Enter file in which to save the key (/opt/oracle/.ssh/id_dsa):      <ENTER>
Enter passphrase (empty for no passphrase):                <ENTER>
Enter same passphrase again:                               <ENTER>
Your identification has been saved in /opt/oracle/.ssh/id_dsa.
Your public key has been saved in /opt/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
70:87:ea:cd:1c:12:c6:f1:fd:5e:49:ab:9d:fe:a8:5d oracle@testrac11
Copying keys to node testrac11
Password:                                                  <ROOT PASSWORD>
The authenticity of host 'testrac11 (10.25.32.111)' can't be established.
RSA key fingerprint is fe:87:5c:2c:6f:fd:d9:0b:e5:19:dd:f3:f9:72:64:7e.
Are you sure you want to continue connecting (yes/no)? yes <yes>
Warning: Permanently added 'testrac11,10.25.32.111' (RSA) to the list of known hosts.
Mon Feb 28 22:21:28 PST 2005
done
Copying keys to node testrac12
Password:                                                  <ROOT PASSWORD>
The authenticity of host 'testrac12 (10.25.32.112)' can't be established.
RSA key fingerprint is e2:8b:a8:a7:13:43:4b:66:b4:b5:9f:26:49:e7:06:d4.
Are you sure you want to continue connecting (yes/no)? yes <yes>
Warning: Permanently added 'testrac12,10.25.32.112' (RSA) to the list of known hosts.
Mon Feb 28 22:21:35 PST 2005
done
testrac11:/INST #

ATTENTION. Script can fail, if root or oracle user have old .ssh directory with wrong keys (for example, if you did another installation and did not reinstall everything). If it fail, you can try to remove these keys and repeat script again:

testrac11# rm -rf ~root/.ssh ~oracle/.ssh

testrac11# sh 080*sh

6.11) 090-verify-ssh-access.sh - verify results (password-less access between nodes).

Run these script on all nodes: sh 090-verify-ssh-access.sh. It is IMPORTANT script - it sets up identity on the very first call of ssh, so do not skip it.

File 090-verify-ssh-access.sh:

#
# Now, verify oracle password-access between nodes
#
#
#
. CONFIG.sh
#
for i in $NODES $NODESP
do
    echo -e testing $i ", \033[31m Answer yes very first time, please \033[m"
    su - oracle -c "ssh -n $i echo TEST"
    lines=`su - oracle -c "ssh -n $i echo TEST" | wc -l`
    if [ x$lines != x1 ]
    then
        echo ATTENTION. Something wrong with oracle 'ssh $i' , check it manually
    else
        echo $i OK
    fi
done

Output:

testrac11:/INST # sh 090*sh
testing testrac11 , Answer yes very first time, please
TEST
testrac11 OK
testing testrac12 , Answer yes very first time, please
TEST
testrac12 OK
testing testrac11-1 , Answer yes very first time, please
The authenticity of host 'testrac11-1 (10.253.32.111)' can't be established.
RSA key fingerprint is fe:87:5c:2c:6f:fd:d9:0b:e5:19:dd:f3:f9:72:64:7e.
Are you sure you want to continue connecting (yes/no)? yes <yes>
Warning: Permanently added 'testrac11-1,10.253.32.111' (RSA) to the list of known hosts.
TEST
testrac11-1 OK
testing testrac12-1 , Answer yes very first time, please
The authenticity of host 'testrac12-1 (10.253.32.112)' can't be established.
RSA key fingerprint is e2:8b:a8:a7:13:43:4b:66:b4:b5:9f:26:49:e7:06:d4.
Are you sure you want to continue connecting (yes/no)? yes <yes>
Warning: Permanently added 'testrac12-1,10.253.32.112' (RSA) to the list of known hosts.
TEST
testrac12-1 OK

6.12) Edit and verify /etc/hosts:

Edit this file, if necessary, to make it EXACTLY as in our example (replace hostnames and domain). System installation creates (usually) wrong records for private interface.

Wrong /etc/hosts (for example, using full domain name in first field) is most common source of RAC installation problems. So, double check and verify that you have perfect host names.

Example:

testrac11# vi /etc/hosts
testrac11# cat /etc/hosts

10.23.32.111 testrac11 testrac11.mydomain.com

10.254.32.111 testrac11-1 testrac11-1.mydomain.com

10.23.32.112 testrac12 testrac12.mydomain.com

10.254.32.112 testrac12-1 testrac12-1.mydomain.com

10.23.32.113 testrac11-vip testrac11-vip.mydomain.com

10.23.32.114 testrac12-vip testrac12-vip.mydomain.com

7. Install Cluster Ready Services.

7.1) 100-verify-rawfiles.sh – verify that oracle have access to shared files/

Run script 100 on all nodes, and CHECK OUTPUT carefully.

File 100-verify-rawfiles.sh:

# 100) Verify that we have access to OCRFile and CSSFile

. CONFIG.sh

. /etc/profile.d/oracle.sh

# IMPORTANT. LD_ASSUME_KERNEL must be 2.4.5 here

if [ "$LD_ASSUME_KERNEL" = "" ]

then

echo -e "\033[31mATTENTION. LD_ASSUME_KERNEL variable absent, set it up first\033[m"

exit 1

ls -lL $OCRFile $CSSFile

if su - oracle -c "test -c $OCRFile -a -w $OCRFile"

then

echo $OCRFile OK, continue...

else

echo "Oracle can not write OCRFile=$OCRFile or file is not raw device. Aborted"

echo "CORRECT ERROR BEFORE ANY ACTIONS"

exit 1

if su - oracle -c "test -c $CSSFile -a -w $CSSFile"

then

echo $CSSFile OK, continue...

else

echo "Oracle can not write CSSFile=$CSSFile or file is not raw device. Aborted"

echo "CORRECT ERROR BEFORE ANY ACTIONS"

exit 1

echo "OK, you can continue with CRS installation"

testrac12:/INST # sh 100*sh

crw-rw---- 1 root disk 162, 3 Jun 30 2004 /dev/rawnames/CSSFile

crw-rw---- 1 root disk 162, 2 Jun 30 2004 /dev/rawnames/OCRFile

/dev/rawnames/OCRFile OK, continue...

/dev/rawnames/CSSFile OK, continue...

OK, you can continue with CRS installation

testrac12:/INST #

Script verifies, that Oracle user have write access to /dev/rawnames/OCRFile and /dev/rawnames/CSSFile (names are configured in CONFIG.sh) and that these files are raw devices. If it fail, you can not install CRS and must fix problem (most likely, access permissions and iSCSI not running) first. Oracle must be a member of ‘disks’ group to have write access into these files.

7.2) 200-InstCRS.1sh - installing Cluster Ready Services. It is first real INSTALLATION step.

I use this script for CRS installation. Remember - when oracle universal installer (OUI) asks you to run root.sh files, you better use our scripts (210* and 220*) instead - they fix some access permissions problems, which exists in Oracle installer.

Here it is. Run it on node-1 only.

File 200-InstCRS.1sh:

# 200) Install Oracle cluster manager

# Enter /u02/OCRFile and /u02/VCSFile as shared files

. CONFIG.sh

. /etc/profile.d/oracle.sh

# IMPORTANT. LD_ASSUME_KERNEL must be 2.4.5 here

if [ "$LD_ASSUME_KERNEL" = "" ]

then

echo -e "\033[31mATTENTION. LD_ASSUME_KERNEL variable absent, set it up first\033[m"

exit 1

CRS=$CRS10CD1

if [ ! -x $CRS/runInstaller ]

then

echo "Can not find CRS installation disk: $CRS10CD1"

echo "verify variable CRS10CD1 in CONFIG.sh and location of CRS installation files"

exit 1

sux - -c "mkdir -p $ORACLE_HOME;ORACLE_HOME=$ORACLE_BASE/product/10.1.0/crs_1;mkdir -p $ORACLE_HOME;export ORACLE_HOME; echo $ORACLE+HOME;cd $CRS; ./runInstaller" oracle

sleep 5

echo -e "\033[32mWait for runInstaller.

================================

Registry file - $OCRFile

Voting file - $CSSFile

=================================

\033[m

Script:

- Change user to oracle;

- Change ORACLE_HOME to CRS home (should be different from Database home);

- Verify, that $OCRFile and $CSSFile (from CONFIG.sh) do exists, are raw devices, and are writable by ‘oracle’ (oracle must be a member of group ‘disks’);

- Start Oracle installer for CRS and reminds user to expected names for CRS control files.

Oracle installer works in background so scripts returns short before you will see Oracle Universal Installer window.

Output:

testrac11:/INST # !sh

sh 200*sh

crw-rw---- 1 root disk 162, 3 Mar 9 19:54 /dev/rawnames/CSSFile

crw-rw---- 1 root disk 162, 2 Mar 16 18:41 /dev/rawnames/OCRFile

/dev/rawnames/OCRFile OK, continue...

/dev/rawnames/CSSFile OK, continue...

+HOME

Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be UnitedLinux-1.0, redhat-3 or SuSE-9

Passed

All installer requirements met.

Checking Temp space: must be greater than 80 MB. Actual 27914 MB Passed

Checking swap space: must be greater than 150 MB. Actual 8193MB Passed

Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2005-04-06_02-25-50PM. Please wait ...Oracle Universal Installer, Version 10.1.0.3.0 Production

Wait for runInstaller.

================================

Registry file - /dev/rawnames/OCRFile

Voting file - /dev/rawnames/CSSFile

=================================

testrac11:/INST #

It starts OUI in background. Select Next, Next. Now Oracle will ask you to run first root script:

Run script 210*sh

File 210-runInventory.sh

# Run it on every node

. CONFIG.sh

. /etc/profile.d/oracle.sh

sh $ORACLE_BASE/oraInventory/orainstRoot.sh

Results:

testrac11:/INST # sh 210*sh

Creating the Oracle inventory pointer file (/etc/oraInst.loc)

Changing groupname of /opt/oracle/oraInventory to oinstall.

testrac11:/INST #

Notice. If script shows error message, then (most likely) something wrong with $ORACLE_HOME for CRS – Oracle installer and our scripts have different setting.

Click Continue’ when script is completed. You run this particular script on first node, for now.

Confirm few selections, and you will see CRS installation screen:

Continue with Next.

Fill in public and private names for all nodes:

On next screen, specify eth0 as public and eth1 as private interfaces:

(Click on empty field to see set of possible selections).[1]

Specify OCRFile location:

Specify CSSFile location:

Now Oracle will ask you to run orainstRoot.sh script on ALL nodes. Run script 210*.sh on first and second node:

File 210-runInventory.sh:

# Run it on every node

. CONFIG.sh

. /etc/profile.d/oracle.sh

sh $ORACLE_BASE/oraInventory/orainstRoot.sh

Output:

testrac11:/INST # sh 210*sh

Creating the Oracle inventory pointer file (/etc/oraInst.loc)

Changing groupname of /opt/oracle/oraInventory to oinstall.

testrac11:/INST #

and

testrac12:/INST # sh 210*sh

Creating the Oracle inventory pointer file (/etc/oraInst.loc)

Changing groupname of /opt/oracle/oraInventory to oinstall.

testrac12:/INST #

Click Continue, check installation list, and Oracle will start to install CRS:

Now, Oracle requests you to run root.sh script on ALL nodes. It is key step of CRS installation, because this script configure and start CRS, and if something is wrong (host name, disk sharing, etc) CRS will not come.

7.3) Running root.sh scripts (by 220-runRootsh.sh) on all nodes:

This is very important step (you can repeat it in future, if you lost your CRS shared files), it creates shared files, configure CRS and add it into /etc/inittab so that system now starts CRS automatically. You can use /etc/init.d/init.crs script to start / stop CRS, when it all is completed (rcoracle do the same).

Run our script 220-runRootsh.sh first on node-1, then on node-2:

File 220-runRootsh.sh:

# Run this on every node

. CONFIG.sh

mkdir -p /opt/oracle/product/10.1.0/crs_1/log && chown oracle:dba /opt/oracle/product/10.1.0/crs_1/log

sh /opt/oracle/product/10.1.0/crs_1/root.sh

/opt/oracle/product/10.1.0/crs_1/bin/olsnodes -n

/opt/oracle/product/10.1.0/crs_1/bin/crs_stat

echo 'export PATH=$PATH:/opt/oracle/product/10.1.0/crs_1/bin' >> ~root/.bashrc

Results:

testrac11:/INST # sh 220*sh
Running Oracle10 root.sh script...
\nThe following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /opt/oracle/product/10.1.0/crs_1
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Checking to see if Oracle CRS stack is already up...
/etc/oracle does not exist. Creating it now.
Setting the permissions on OCR backup directory
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/opt/oracle/product/10.1.0' is not owned by root
WARNING: directory '/opt/oracle/product' is not owned by root
WARNING: directory '/opt/oracle' is not owned by root
assigning default hostname testrac11 for node 1.
assigning default hostname testrac12 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: testrac11 testrac11-1 testrac11
node 2: testrac12 testrac12-1 testrac12
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /volcrs/CRS/CSSFile
Read -1 bytes of 512 at offset 872603648 in voting device (CSSFile)
Successful in setting block0 for voting disk.
Format complete.
Adding daemons to inittab
Preparing Oracle Cluster Ready Services (CRS):
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        testrac11
CSS is inactive on these nodes.
        testrac12
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.
testrac11       1
testrac12       2

AND on second node:

testrac12:/INST # sh 220*sh

Running Oracle10 root.sh script...

\nThe following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /opt/oracle/product/10.1.0/crs_1

Finished running generic part of root.sh script.

Now product-specific root actions will be performed.

Checking to see if Oracle CRS stack is already up...

Setting the permissions on OCR backup directory

Oracle Cluster Registry configuration upgraded successfully

WARNING: directory '/opt/oracle/product/10.1.0' is not owned by root

WARNING: directory '/opt/oracle/product' is not owned by root

WARNING: directory '/opt/oracle' is not owned by root

clscfg: EXISTING configuration version 2 detected.

clscfg: version 2 is 10G Release 1.

assigning default hostname testrac11 for node 1.

assigning default hostname testrac12 for node 2.

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: testrac11 testrac11-1 testrac11

node 2: testrac12 testrac12-1 testrac12

clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.

-force is destructive and will destroy any previous cluster

configuration.

Oracle Cluster Registry for cluster has already been initialized

Adding daemons to inittab

Preparing Oracle Cluster Ready Services (CRS):

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

testrac11

testrac12

CSS is active on all nodes.

Waiting for the Oracle CRSD and EVMD to start

Oracle CRS stack installed and running under init(1M)

testrac11 1

testrac12 2

CRS-0202: No resources are registered.

You can click Continue (after seen last message).

And see report about successful installation:

7.4) Running 230-check-crs.sh - verify crs.

Now, verify that CRS is really started on ALL nodes. Use script 230:

File 230-check-crs.sh:

#
# 230) Now, check if crs works
#
. ~oracle/.profile
. ~/.bashrc
rm /var/log/crs
ln -s ~oracle/product/*/crs_1/crs/log/ /var/log/crs

crsctl check install
tail /var/log/crs/*

Script:

- Creates shortcut to CRS logs - /var/log/crs, so that you can always watch CRS logs by simple typing tail –f /var/log/crs;

- Show you recent CRS logs.

testrac11:/INST # sh 230*sh

rm: cannot remove `/var/log/crs': No such file or directory

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

testrac11

testrac12

CSS is active on all nodes.

2005-04-06 16:18:08.532: CRSD locked during state recovery, please wait.

2005-04-06 16:18:08.534: CRSD recovered, unlocked.

2005-04-06 16:18:08.535: QS socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=ora_crsqs))

2005-04-06 16:18:08.536: UI socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=testrac11_testrac_caa))

2005-04-06 16:18:08.541: E2E socket on: (ADDRESS=(PROTOCOL=tcp)(HOST=testrac11-1)(PORT=49896))

2005-04-06 16:18:08.541: Starting Threads

2005-04-06 16:18:08.541: CRS Daemon Started.

2005-04-06 15:55:50.374: CRSD-1: [CMDMAIN:2540914048] Restart waiting for Oracle CRSD to start

2005-04-06 15:56:00.632: CRSD-1: [CMDMAIN:2540914048] Restart waiting for Oracle CRSD to start

2005-04-06 16:18:09.247: CRSD-1: Complete Restart Application Request

AND, on node 2:

testrac11:/INST # sh 230*sh

rm: cannot remove `/var/log/crs': No such file or directory

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

testrac11

testrac12

CSS is active on all nodes.

2005-04-06 16:18:08.532: CRSD locked during state recovery, please wait.

2005-04-06 16:18:08.534: CRSD recovered, unlocked.

2005-04-06 16:18:08.535: QS socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=ora_crsqs))

2005-04-06 16:18:08.536: UI socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=testrac11_testrac_caa))

2005-04-06 16:18:08.541: E2E socket on: (ADDRESS=(PROTOCOL=tcp)(HOST=testrac11-1)(PORT=49896))

2005-04-06 16:18:08.541: Starting Threads

2005-04-06 16:18:08.541: CRS Daemon Started.

2005-04-06 15:55:50.374: CRSD-1: [CMDMAIN:2540914048] Restart waiting for Oracle CRSD to start

2005-04-06 15:56:00.632: CRSD-1: [CMDMAIN:2540914048] Restart waiting for Oracle CRSD to start

2005-04-06 16:18:09.247: CRSD-1: Complete Restart Application Request

Index

Next: Database server installation

[1] You must use THE SAME interface names for private and public interfaces on all nodes – you can not, for example, use eth0 as public on testrac11 and eth1 as public on testrac12.