This is very common task – running 64 bit server with numerous Oracle components, but need to install Oracle application (such as a database OR Grid Control Server) which is not released for x86_64 platform, but works on i386 platform.
The decision is to create a new oracle32 user, with special environment, and run installations under this user.
Notice – mixing 64 and 32 bits oracle components complicate system support and is not recommended (it is much easier to keep all oracle components in 32 bit or 64 bit). But sometimes, you cannot avoid it.
I used SuSe Linux Enterprise server, Servicepack3, but idea will work (with small modifications) on any other Linux.
Modify gcc compiler so that it can recognize 32 bit accounts and run in 32-bit mode when necessary. We will use GCC_FLAGS and LD_FLAGS for this purpose, and create a special shell.
Always verify, that system upgrade did not break these scripts, and make this modification again if necessary.
Script is here (you can do the same without script if you want). See content of gcc and ld scripts in bold.
#!/bin/sh
#
# This script modifies gcc and ld and create bash32 shell
#
if [ –x /usr/bin/gcc.bin ]
then
echo “gcc.bin already exists. Verify that you want to move gcc into gcc.bin ,”
echo “ remove gcc.bin, and run script again”
exit 1
fi
cd /usr/bin
mv gcc gcc.bin
cat > gcc <<\EOF
#!/bin/sh
exec /usr/bin/gcc.bin $GCC_FLAGS “$@”
EOF
chmod a+x gcc
mv ld ld.bin
cat > ld <<\EOF
#!/bin/sh
exec /usr/bin/ld.bin $LD_FLAGS “$@”
EOF
cat > bash32 <<\EOF
#!/bin/sh
export GCC_FLAGS=-m32
export LD_FLAGS=-melf_i386
exec linux /bin/bash “$@”
EOF
Now, when we create oracle32 user with shell /usr/bin/bash32, it will run gcc and ld in 32 bit mode and will report our platform, uname –i , as i386.
Now it’s time to create oracle32 user in new 32-bit environment. Properties must be (user id is not important; you can even try the same user id as used for the oracle itself):
User ID: 61
Group: oinstall
Additional groups: dba, disk
Shell: /usr/bin/bash32
Now, create .bashrc file for this user. Remember – bash32 wil not process /etc/profile.d/*.sh init files, so if you convert orarun into oracle32 environment, you must create .bashrc in home directory.
.bash.rc file (home: /opt/oracle32, sid: util).
# Modify this,m if yoiu did not installed orarun.
. /etc/sysconfig/oracle
# This 2 lines are specific (a little) for oracle32 (but you can try without them).
export LD_ASSUME_KERNEL=2.4.21
export LD_PRELOAD_32=/usr/lib/libInternalSymbols.so
# The rest is standard.
ORACLE_SID=util
ORACLE_BASE=/opt/oracle32
ORACLE_HOME=$ORACLE_BASE/OracleHomes/oms10g
TNS_ADMIN=$ORACLE_HOME/network/admin
# Set ORA_NLSxx depending on 9i or 10g
test -d $ORACLE_HOME/ocommon/nls/admin/data &&\
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
test -d $ORACLE_HOME/nls/data && export ORA_NLS10=$ORACLE_HOME/nls/data
PATH=$ORACLE_BASE/bin32:$PATH:$ORACLE_HOME/bin
LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$ORACLE_HOME/lib:$ORACLE_HOME/ctx/lib
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib
export ORACLE_BASE ORACLE_HOME ORACLE_SID
export PATH LD_LIBRARY_PATH CLASSPATH TNS_ADMIN
ulimit -c ${MAX_CORE_FILE_SIZE_SHELL:-0} 2>/dev/null
# max number of processes for user
ulimit -u ${PROCESSES_MAX_SHELL:-16384} 2>/dev/null
# max number of open files for user
ulimit -n ${FILE_MAX_SHELL:-65536} 2>/dev/null
Now, login and verify, that everything configured properly. I did it by compiling hello_word programm:
oracle32@orabe01:~> cat > hello.c
#include <stdio.h>
main(ac,av) {
printf("hello, world!\n");
}
^D
oracle32@orabe01:~> gcc hello.c
oracle32@orabe01:~> file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2
.2.5, dynamically linked (uses shared libs), not stripped
Attention:
When
you run in ‘oracle32’ user, system returns ‘i386’ on ‘uname –i’ request. It
makes many scripts, including different root scripts, to think that they are on
32 bit linux. Never run ‘su’ or ‘su oracle’ (if ‘oracle’ is 64-bits) from
oracle32 user, except when you are 100% sure, that you want to run root in
32-bit environment.
I recommend editing /etc/security/limits.conf file and change hard limits here, to allow correct limit setting in oracle user. Setting limits in /etc/init.d/oracle is not enough for emagent, which opens many files at once (and so can run out of limits easily).
If you run both, oracle and oracle32, users, verify that they have mutual access to the inventory directories (I recommend to set up chmod g+rw for all inventofy files and g+rwx for al directories). Possible choice is to use exactly the same user id in oracle and oracle32, but it wil not allow you to distinguish processes on CPU.
After step2 is completed, you can run any Oracle installers in ‘oracle32’ user as you do it in Linux on 32 bit platform. I tested Oracle 10.1.0.4 installation, and OMS installation with and without the database, and Agent10g installation for i386. All works, with few exceptions:
- Oracle 10.2 Grid installation from Oracle have a bug in the media.
- You must disable ipv6 localhost in /etc/hosts.
Oracle Enterprise Manager (OMS) installation for SLES9 Linux is well described here:
http://www.nextre.it/oracledocs/gridcontrolonsles9.html
Everything was straightforward, after few preparation steps.
You must comment out ipv6 lines in /etc/hosts, else you will have an error on OMC configuration step (very difficult to troubleshoot). I commented out this line:
#::1 localhost ipv6-localhost ipv6-loopback
Don’t forget to install all required 32 bit rpm’s. For grid control, install
db1-32bit-9-200407011229
Here is full list of 32bit components, which I have (but some are not required for oracle32 – I just had not time to sort them out).
libart_lgpl-32bit
cdparanoia-32bit
flex-32bit
libxml2-32bit
atk-32bit
libgcj-32bit
kdebase3-32bit
kdelibs3-32bit
nss_ldap-32bit
freetype2-devel-32bit
heimdal-lib-32bit
cracklib-32bit
audiofile-32bit
libvorbis-32bit
libjasper-32bit
tk-32bit
gnome-vfs2-32bit
libpng-32bit
imlib-32bit
libattr-32bit
glib-32bit
pcre-32bit
curl-32bit
glibc-devel-32bit
libmikmod-32bit
libogg-32bit
liblcms-32bit
utempter-32bit
libtool-32bit
libjpeg-32bit
capi4linux-32bit
libmng-32bit
libusb-32bit
libidl-32bit
aalib-32bit
pam-32bit
xaw3d-32bit
SDL-32bit
openmotif-libs-32bit
resmgr-32bit
alsa-32bit
file-32bit
pango-32bit
fam-32bit
e2fsprogs-32bit
libgcj-devel-32bit
cups-libs-32bit
orbit2-32bit
freetype2-32bit
gtk2-32bit
openssl-32bit
gdbm-devel-32bit
ncurses-devel-32bit
bzip2-32bit
pam_ldap-32bit
libraw1394-32bit
mad-32bit
slang-32bit
tcl-32bit
gdbm-32bit
expat-32bit
esound-32bit
libbonobo-32bit
gtk-32bit
freeglut-32bit
thinkeramik-32bit
readline-32bit
cyrus-sasl-32bit
libaio-devel-32bit
zlib-devel-32bit
libelf-32bit
XFree86-Mesa-32bit
XFree86-libs-32bit
binutils-32bit
libsmbclient-32bit
glibc-locale-32bit
bind-utils-32bit
pam-modules-32bit
libstdc++-devel-32bit
fontconfig-devel-32bit
bison-32bit
XFree86-devel-32bit
perl-32bit
libtiff-32bit
glibc-32bit
libidn-32bit
libxcrypt-32bit
compat-32bit
fontconfig-32bit
db-32bit
gconf2-32bit
xmms-32bit
gettext-32bit
libacl-32bit
qt3-32bit
libungif-32bit
openldap2-client-32bit
libobjc-32bit
arts-32bit
ncurses-32bit
libselinux-32bit
gpm-32bit
flac-32bit
libaio-32bit
strace-32bit
popt-32bit
glib2-32bit
zlib-32bit
libxslt-32bit
ltrace-32bit
db1-32bit
Oracle 10.2 Grid installation media on oracle site is broken (as of October 2006) – it uses wrong Java version in product configuration (so you cannot instal it in ‘use existing database’ mode). I create a patch, which modifies this media (after you unzip all downloaded files), and correct this problem, BUT this patch will not work with any future version (through I think, that Oracle wil fix it in future version). So, be careful, and try without the patch first. Here is a patch – run it from the base directory of installation media (where you unzipped all Enterprise Manager files, downloaded from Oracle):
#
# This patch change Oracle 10R2 Grid installation source
# fixing bug wuth the wrong Java version
#
# Cd to the root installation directory (which contains runInstaller
# and where you unzipped al files) and run this script
# (I use -noleaf because I am on Windows NFS)
for i in `find . -noleaf -name products.xml -print`
do
cp $i $i-orig
sed 's/1.4.1.0.3/1.4.2.0.8/g' $i > $i- && cp $i- $i && rm -f $i-
ls -l $i
done
I find few configurations pretty easy to install:
1. OMS in 32bit mode only. Create database as described in Oracle documentation and install OMS in ‘Use existing database’ mode. This is recommended here, because you prevent mixing 32 and 64 bit database engines (which means that you must configure listener, be very careful starting up oracle, and complicate things). I had only one minor warning during installation (Oracle asked about Java 1.3, which but I ignored it – looks as one more bug in OMS installation), and everything come without a glitch.
2. You run only oracle32 environment (oracle32 user) on the server and install OMS with the database. To do it, I installed orarun and then modified shell (and moved /etc/init.d/oracle.sh into ~/.barhrc, because bash32 shell will not run files from /etc/profile.d). It worked out of the box as well.
I did not find any specifics in these installations (different from any other multihome and multi-user-id installations).
As I said in previous step, nothing specific here – it worked with one minor warning. Possible problems:
- You select ‘Use existing database’ and see installer error, saying about Java – apply patch.
- You select oracle home base, and system says that home exists. It can happen if you removed some previous installation – oui don’t remove records about homes, by some reason. I edited inventory (for example, /opt/oracle/oraInventory/ContentsXML/inventory.xml) manually after I removed oracle components.
- Be careful when running root.sh file – you can su from oracle or oracle32, but better always do it from ‘oracle’ or any 64 bit user.
After you install OMS, fix startup files. Oracle (as usual) supply primitive and broken inut files, written without any understanding of Linux. So, edit file /etc/init.d/gcstartup and add header into it:
/etc/init.d/gcstartup:
#!/bin/sh
#
# Add this lines.
#
### BEGIN INIT INFO
# Provides: gcsstartup
# Required-Start: $network $syslog $remote_fs raw oracle
# Required-Stop:
# Should-Start: $ALL
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Description: Start the Oracle32 database and OEM
### END INIT INFO
#
# Here is content from oracle, with small modification.
#
#Source Function Library
if [ -f /etc/rc.status ]
then
. /etc/rc.status
else
if [ -f /etc/rc.d/init.d/functions ]
then
. /etc/rc.d/init.d/functions
else
exit 0;
fi
fi
#
# Add ulimit commands here if necessary.
#
if [ -f /opt/oracle32/OracleHomes/oms10g/install/unix/scripts/omsstup ]; then
. /opt/oracle32/OracleHomes/oms10g/install/unix/scripts/omsstup
fi
if [ -f /opt/oracle32/OracleHomes/agent10g/install/unix/scripts/agentstup ]; then
. /opt/oracle32/OracleHomes/agent10g/install/unix/scripts/agentstup
fi
Oracle installer adds lines into this file on each installation, so, if you try few times, edit it and remove duplicated lines.
And that’s all. To my great surprise, everything worked pretty well after all, and I was even able to configure few agents, with a few grid control, run 32 and 64 bit agents together, and so on.