TABLE OF CONTENT:
10.3) Starting, stopping and managing.
10.4) Verify system shutdown and startup.
You can watch CRS activity by
looking into CRS log file (/opt/oracle/product/10.1.0/crs_1/crs/logs/*, which we symlinked into
/var/log/crs for simplicity):
tail f /var/log/crs/*
Output:
testrac11:/INST # tail -20 /var/log/crs/*
2005-04-08 15:55:44.038: Start of `ora.testrac12.LISTENER_TESTRAC12.lsnr` on member `testrac12` succeeded.
2005-04-08 18:07:21.027: [RESOURCE:1397131616] Resource Registered: ora.testrac11.ASM1.asm
2005-04-08 18:07:21.978: Attempting to start `ora.testrac11.ASM1.asm` on member `testrac11`
2005-04-08 18:07:23.045: Start of `ora.testrac11.ASM1.asm` on member `testrac11` succeeded.
2005-04-08 18:07:24.115: [RESOURCE:1397131616] Resource Registered: ora.testrac12.ASM2.asm
2005-04-08 18:07:25.372: Attempting to start `ora.testrac12.ASM2.asm` on member `testrac12`
2005-04-08 18:07:29.415: Start of `ora.testrac12.ASM2.asm` on member `testrac12` succeeded.
2005-04-08 18:45:40.011: [RESOURCE:1455851872] Resource Registered: ora.test.db
2005-04-08 18:45:41.685: [RESOURCE:1455851872] Resource Registered: ora.test.test1.inst
2005-04-08 18:45:43.415: [RESOURCE:1455851872] Resource Registered: ora.test.test2.inst
2005-04-08 18:48:27.172: [RESOURCE:1439074656] Resource Registered: ora.test.test.cs
2005-04-08 18:48:28.418: [RESOURCE:1439074656] Resource Registered: ora.test.test.test2.srv
2005-04-08 18:48:29.426: [RESOURCE:1439074656] Resource Registered: ora.test.test.test1.srv
2005-04-08 18:55:35.455: `ora.test.test1.inst` is already OFFLINE.
2005-04-08 18:55:42.744: Attempting to start `ora.test.test1.inst` on member `testrac11`
2005-04-08 18:55:42.893: Attempting to start `ora.test.test2.inst` on member `testrac12`
2005-04-08 18:56:12.085: Start of `ora.test.test2.inst` on member `testrac12` succeeded.
2005-04-08 18:56:20.990: Start of `ora.test.test1.inst` on member `testrac11` succeeded.
2005-04-08 18:56:21.498: Attempting to start `ora.test.test.cs` on member `testrac12`
2005-04-08 18:56:22.679: Start of `ora.test.test.cs` on member `testrac12` succeeded.
testrac11:/INST #
DM console is web interface to Oracle Enterprise Management , running (for first database) on port 5500, path /em, (dbca reports port number for other databases). It is pretty useful for everything, except emergency works, recoveries and other activities performed with database in non-functional mode (because OEM depends on database for functioning). I recommend even to create small test database, never touch it, and use it to run EM, which provides you access to ASM instance as well as to test instance. See example above (in previous section).
Clustered database is started and
stopped by Cluster Ready Services (CRS) manager, so our start / stop script only
starts and stops CRS processes. Moreover, Cluster Ready Services and 2 other
components are installed by Oracle installation into /etc/inittab, and
are automatically started by system init when it completes runlevel3 and
runlevel5 initialisation:
h1:35:respawn:/etc/init.d/init.evmd run >/dev/null 2>&1
</dev/null
h2:35:respawn:/etc/init.d/init.cssd fatal >/dev/null 2>&1
</dev/null
h3:35:respawn:/etc/init.d/init.crsd run >/dev/null 2>&1
</dev/null
-
evmd
is process
responsible for event exchange between cluster members;
-
cssd is cache synchronisation services
daemon;
-
crsd is cluster ready services daemon,
responsible for starting / stopping everything.
If init cannot start any of these
daemons few times in a line, it suspends future attempts for 5 minutes. System
script, /etc/init.d/init.crs, control flags, allowing these 3 processes to
start, and sends a signal to running scripts when they are requested to stop.
As a result, when you stopped CRS (or when it dies by any reason) and want to
restart it, after restart is allowed, you must kill 1 1 to speed-up
process respawning.
I recommend to link script, provided by Oracle, with root command rccrs:
ln
-s /etc/init.d/init.crs /sbin/rccrs
Do not run other 3 (/etc/init.d/init.*)
scripts provided by Oracle they are started / stopped by init and controlled with init.crs
script.
We already linked log directory for crs
as /var/log/crs, so we can watch crs logs as
tail
f /var/log/crs/*
You should know at least srvctl oracle
command for very initial RAC management. For example, run:
testrac11:~ # srvctl status database -d test
Instance test1 is running on node testrac11
Instance test2 is running on node testrac12
You can use our (modifying) rcoracle
(/etc/init.d/oracle) script to start, stop and status-request database:
rcoracle stop stops CRS and all cluster
database instances;
rcoracle start starts CRS (which will start
instances).
rcoracle status shows status;
rcoracle kill dumb brute force killing all
oracle processes including any oracle user processes. Used as a last resort
chance to release busy resource (NFS disk for example).
Verify system shutdown and startup
in a few scenarios BEFORE using newrly created database for any real data. Do
not forget to test all scenarios, for example:
o
when 1
node reboots while other node runs normally;
o
when
both nodes reboots at the same time.
We installed (by orarun
package and then by our script) rcoracle script (link to /etc/init.d/oracle),
which is controlled by /etc/sysconfig/oracle configuration. Try stopping
and starting database on one of the servers: