Exadata on Vmware Step by step

Step 1 : VM setup 1. Create a VM with 8gb Ram and 1 cpu with 1 core 2. add 2 Nat to the vm while creation install linux 6.8 3. change the IP for eth1 to static Hosts file config :
[root@Exacell Desktop]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6


10.0.0.100 Exacell.sultan.net Exacell

Step 2 : Needful OS setting :

[root@Exacell Desktop]# ## Install VMWare Tool

## Update Linux parameters

2.
## /etc/sysctl.conf

fs.file-max = 655360 in /etc/sysctl.conf

3.
## /etc/security/limits.conf

· soft nofile 655360

· hard nofile 655360

4.
## /etc/grub.conf

default=1

5.
## /etc/selinux/config

SELINUX=disabled

6.
## /etc/bashrc

export DISPLAY=:0

7. ## system setting: add below parameter

[root@Exacell cellbits]# modprobe rds

[root@Exacell cellbits]# modprobe rds_tcp

[root@Exacell cellbits]# modprobe rds_rdma

8.
cat /etc/modprobe.d/rds.conf

install rds /sbin/modprobe --ignore-install rds && /sbin/modprobe rds_tcp && /sbin/modprobe rds_rdma

Step 3 : Add 18 disks to vm with 2 GB of size each




Step 4 : Check the ifconfig status for the given IP's

[root@Exacell Desktop]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:5A:18:2A
inet addr:192.168.37.140 Bcast:192.168.37.255 Mask:255.255.255.0
inet6 addr: 2405:204:a602:3b5:20c:29ff:fe5a:182a/64 Scope:Global
inet6 addr: fe80::20c:29ff:fe5a:182a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000

RX bytes:724 (724.0 b) TX bytes:990 (990.0 b)

eth1 Link encap:Ethernet HWaddr 00:0C:29:5A:18:34
inet addr:10.0.0.100 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe5a:1834/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:276 (276.0 b) TX bytes:1008 (1008.0 b)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:3173 errors:0 dropped:0 overruns:0 frame:0
TX packets:3173 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:195225 (190.6 KiB) TX bytes:195225 (190.6 KiB)

Step 5 : Download and unzip of software

download the software for exadata cell storage : V42777-01.zip

you can get the software from https://edelivery.oracle.com/

unzip the software V42777-01.zip

you will get 3 output as below

[root@Exacell Cell]# ls -ltr
total 1527648
-rwxrw-rw-. 1 root root 1564306563 Jul 17 2018 V42777-01.zip

[root@Exacell Cell]# unzip --q V42777-01.zip
Archive: V42777-01.zip
inflating: README.txt
inflating: cellImageMaker_12.1.1.1.0_LINUX.X64_131219-1.x86_64.tar

[root@Exacell oler]# tar -xvf cellImageMaker_12.1.1.1.0_LINUX.X64_131219-1.x86_64.tar
Directory will be created

[root@Exacell Cell]# ls -ltr
total 3792240
drwxrwxr-x. 8 root root 4096 Dec 19 2013 dl180

[root@Exacell cellbits]# pwd
/oracle/Cell/dl180/boot/cellbits

[root@Exacell cellbits]# unzip cell.bin
Archive: cell.bin
warning [cell.bin]: 20118 extra bytes at beginning or within zipfile
(attempting to process anyway)
inflating: cell-12.1.1.1.0_LINUX.X64_131219-1.x86_64.rpm


[root@Exacell cellbits]# tar -xvf cellrpms.tbz
cellrpmlist
exadata-firmware-cell-12.1.1.1.0.131219-1.x86_64.rpm
exadata-validations-cell-12.1.1.1.0.131219-1.x86_64.rpm
extradepsrpmlist
image_functions
imagerpmlist
install.sh
jdk-1.7.0_25-fcs.x86_64.rpm
mdadm-2.6.9-5.el5.x86_64.rpm
python-numeric-23.7-2.2.2.el5_6.1.x86_64.rpm
scsi-target-utils-1.0.14-2.el5.x86_64.rpm
selinux-policy-2.4.6-338.el5.noarch.rpm
selinux-policy-targeted-2.4.6-338.el5.noarch.rpm

Step 6 : Install JDK

[root@Exacell cellbits]# rpm -ivh jdk-1.7.0_25-fcs.x86_64.rpm
warning: jdk-1.7.0_25-fcs.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing... ########################################### [100%]
1:jdk1.7.0_121 ########################################### [100%]
Unpacking JAR files...
tools.jar...
plugin.jar...
javaws.jar...
deploy.jar...
rt.jar...
jsse.jar...
charsets.jar...
localedata.jar...

Step 7 : Needed RPM install

[root@Exacell Packages]# rpm -ivh net-snmp-5.5-57.el6.x86_64.rpm 
warning: net-snmp-5.5-57.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing... ########################################### [100%]
package net-snmp-1:5.5-57.el6.x86_64 is already installed

[root@Exacell Packages]#

[root@Exacell Packages]# rpm -ivh net-snmp-utils-5.5-57.el6.x86_64.rpm
warning: net-snmp-utils-5.5-57.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing... ########################################### [100%]
package net-snmp-utils-1:5.5-57.el6.x86_64 is already installed

[root@Exacell Packages]# rpm -ivh perl-XML-NamespaceSupport-1.10-3.el6.noarch.rpm
warning: perl-XML-NamespaceSupport-1.10-3.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing... ########################################### [100%]
1:perl-XML-NamespaceSuppo########################################### [100%]

[root@Exacell yum.repos.d]# yum install perl-XML-SAX
Loaded plugins: product-id, refresh-packagekit, search-disabled-repos, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package perl-XML-SAX.noarch 0:0.96-7.el6 will be installed
--> Processing Dependency: perl(XML::LibXML::Common) for package: perl-XML-SAX-0.96-7.el6.noarch
--> Processing Dependency: perl(XML::LibXML) for package: perl-XML-SAX-0.96-7.el6.noarch
--> Running transaction check
---> Package perl-XML-LibXML.x86_64 1:1.70-5.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================
Package Arch Version Repository Size

==============================================================================================================================================

Installing:
perl-XML-SAX noarch 0.96-7.el6 oellabrepo 78 k
Installing for dependencies:
perl-XML-LibXML x86_64 1:1.70-5.el6 oellabrepo 364 k
Transaction Summary

==============================================================================================================================================
Install 2 Package(s)
Total download size: 442 k
Installed size: 1.2 M
Is this ok [y/N]: y
Downloading Packages:
----------------------------------------------------------------------------------------------------------------------------------------------

Total 133 MB/s | 442 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-XML-SAX-0.96-7.el6.noarch 1/2
Installing : 1:perl-XML-LibXML-1.70-5.el6.x86_64 2/2
Verifying : 1:perl-XML-LibXML-1.70-5.el6.x86_64 1/2
Verifying : perl-XML-SAX-0.96-7.el6.noarch 2/2
Installed:
perl-XML-SAX.noarch 0:0.96-7.el6
Dependency Installed:
perl-XML-LibXML.x86_64 1:1.70-5.el6
Complete!

[root@Exacell oracle]# rpm -ivh perl-XML-SAX-Expat-0.50-1.of.el6.noarch.rpm
warning: perl-XML-SAX-Expat-0.50-1.of.el6.noarch.rpm: Header V4 RSA/SHA1 Signature, key ID a0030e2c: NOKEY
Preparing... ########################################### [100%]
1:perl-XML-SAX-Expat ########################################### [100%]

[root@Exacell oracle]# rpm -ivh perl-XML-Simple-2.20-1.of.el6.noarch.rpm
warning: perl-XML-Simple-2.20-1.of.el6.noarch.rpm: Header V4 RSA/SHA1 Signature, key ID a0030e2c: NOKEY
Preparing... ########################################### [100%]
1:perl-XML-Simple ########################################### [100%]

Step 8 : Creating needed Directories

[root@Exacell cellbits]# mkdir /var/log/oracle
[root@Exacell cellbits]# chmod 775 /var/log/oracle

Step 9 : Install Cell Software (Important)

[root@Exacell cellbits]# rpm -ivh cell-12.1.1.1.0_LINUX.X64_131219-1.x86_64.rpm 
Preparing... ########################################### [100%]
Pre Installation steps in progress ...
1:cell ########################################### [100%]
Post Installation steps in progress ...
Set cellusers group for /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/cellsrv/deploy/log directory
Set 775 permissions for /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/cellsrv/deploy/log directory
/opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/cellsrv/deploy
/opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/cellsrv/deploy
/opt/oracle/cell12.1.1.1.0_LINUX.X64_131219
Installation SUCCESSFUL.
Starting RS and MS... as user celladmin
Done. Please Login as user celladmin and create cell to startup CELLSRV to complete cell configuration.
If this is a manual installation, please stop and restart ExaWatcher to pick up newly installed binaries.
You can run "/opt/oracle.ExaWatcher/ExaWatcher.sh --stop" and then "/opt/oracle.ExaWatcher/ExaWatcher.sh --fromconf" to stop and restart ExaWatcher.
Logout and then re-login to use the new cell environment.

Step 10 : Check the disk by below command

[root@Exacell cellbits]# fdisk -l 2>/dev/null |grep 'B,'
Disk /dev/sda: 107.4 GB, 107374182400 bytes
Disk /dev/sdb: 2147 MB, 2147483648 bytes
Disk /dev/sdc: 2147 MB, 2147483648 bytes
Disk /dev/sdd: 2147 MB, 2147483648 bytes
Disk /dev/sde: 2147 MB, 2147483648 bytes
Disk /dev/sdf: 2147 MB, 2147483648 bytes
Disk /dev/sdg: 2147 MB, 2147483648 bytes
Disk /dev/sdh: 2147 MB, 2147483648 bytes
Disk /dev/sdi: 2147 MB, 2147483648 bytes
Disk /dev/sdj: 2147 MB, 2147483648 bytes
Disk /dev/sdk: 2147 MB, 2147483648 bytes
Disk /dev/sdm: 2147 MB, 2147483648 bytes
Disk /dev/sdl: 2147 MB, 2147483648 bytes
Disk /dev/sdn: 1610 MB, 1610612736 bytes
Disk /dev/sdo: 1610 MB, 1610612736 bytes
Disk /dev/sdp: 1610 MB, 1610612736 bytes
Disk /dev/sdq: 1610 MB, 1610612736 bytes
Disk /dev/sdr: 1610 MB, 1610612736 bytes
Disk /dev/sds: 1610 MB, 1610612736 bytes

Step 11 : Create path for Raw Disks

[root@Exacell ~]# cd /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/
[root@Exacell cell12.1.1.1.0_LINUX.X64_131219]# mkdir -p disks/raw

Step 12 : Move in the created directory and create soft links for disks

[root@Exacell raw]# pwd
/opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/disks/raw
[root@Exacell raw]# ln -s /dev/sdb cell12_DISK01
[root@Exacell raw]# ln -s /dev/sdc cell12_DISK02
[root@Exacell raw]# ln -s /dev/sdd cell12_DISK03
[root@Exacell raw]# ln -s /dev/sde cell12_DISK04
[root@Exacell raw]# ln -s /dev/sdf cell12_DISK05
[root@Exacell raw]# ln -s /dev/sdg cell12_DISK06
[root@Exacell raw]# ln -s /dev/sdh cell12_DISK07
[root@Exacell raw]# ln -s /dev/sdi cell12_DISK08
[root@Exacell raw]# ln -s /dev/sdj cell12_DISK09
[root@Exacell raw]# ln -s /dev/sdk cell12_DISK10
[root@Exacell raw]# ln -s /dev/sdl cell12_DISK11
[root@Exacell raw]# ln -s /dev/sdm cell12_DISK12
[root@Exacell raw]# ln -s /dev/sdn cell12_FLASH01
[root@Exacell raw]# ln -s /dev/sdo cell12_FLASH02
[root@Exacell raw]# ln -s /dev/sdp cell12_FLASH03
[root@Exacell raw]# ln -s /dev/sdq cell12_FLASH04
[root@Exacell raw]# ln -s /dev/sdr cell12_FLASH05
[root@Exacell raw]# ln -s /dev/sds cell12_FLASH06

[root@Exacell raw]# ls -ltr
total 0
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK01 -> /dev/sdb
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK02 -> /dev/sdc
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK03 -> /dev/sdd
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK04 -> /dev/sde
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK05 -> /dev/sdf
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK06 -> /dev/sdg
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK07 -> /dev/sdh
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK08 -> /dev/sdi
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK09 -> /dev/sdj
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK10 -> /dev/sdk
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK11 -> /dev/sdl
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_DISK12 -> /dev/sdm
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH01 -> /dev/sdn
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH02 -> /dev/sdo
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH03 -> /dev/sdp
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH04 -> /dev/sdq
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH05 -> /dev/sdr
lrwxrwxrwx. 1 root root 8 Apr 2 14:25 cell12_FLASH06 -> /dev/sds

Step 13 : Check the celld status from root

[root@Exacell raw]# service celld status
rsStatus: running
msStatus: running
cellsrvStatus: stopped

Step 14 : Now switch to Cell Admin user created by cell rpm install step and create cell

[root@Exacell raw]# su - celladmin

[celladmin@Exacell ~]$ cellcli -e alter cell restart services all
Stopping the RS, CELLSRV, and MS services...
The SHUTDOWN of services was successful.
Starting the RS, CELLSRV, and MS services...
Getting the state of RS services... running
Starting CELLSRV services...
The STARTUP of CELLSRV services was not successful.
CELL-01547: CELLSRV startup failed due to unknown reasons.
Starting MS services...
The STARTUP of MS services was successful.

[celladmin@Exacell ~]$ cellcli -e create cell exacel01 interconnect1=eth1
Cell exacel01 successfully created
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Flash cell disks, FlashCache, and FlashLog will be created...
CellDisk FD_00_exacel01 successfully created
CellDisk FD_01_exacel01 successfully created
CellDisk FD_02_exacel01 successfully created
CellDisk FD_03_exacel01 successfully created
CellDisk FD_04_exacel01 successfully created
CellDisk FD_05_exacel01 successfully created
Flash log exacel01_FLASHLOG successfully created
Flash cache exacel01_FLASHCACHE successfully created

[celladmin@Exacell ~]$ cellcli -e alter cell restart services all
Stopping the RS, CELLSRV, and MS services...
The SHUTDOWN of services was successful.
Starting the RS, CELLSRV, and MS services...
Getting the state of RS services... running
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Starting MS services...
The STARTUP of MS services was successful.

Step 15 : Create cell disk and Grid disk

[celladmin@Exacell ~]$ cellcli -e create celldisk all
CellDisk CD_DISK01_exacel01 successfully created
CellDisk CD_DISK02_exacel01 successfully created
CellDisk CD_DISK03_exacel01 successfully created
CellDisk CD_DISK04_exacel01 successfully created
CellDisk CD_DISK05_exacel01 successfully created
CellDisk CD_DISK06_exacel01 successfully created
CellDisk CD_DISK07_exacel01 successfully created
CellDisk CD_DISK08_exacel01 successfully created
CellDisk CD_DISK09_exacel01 successfully created
CellDisk CD_DISK10_exacel01 successfully created
CellDisk CD_DISK11_exacel01 successfully created
CellDisk CD_DISK12_exacel01 successfully created

[celladmin@Exacell ~]$ cellcli -e create griddisk all harddisk prefix=DATA
GridDisk DATA_CD_DISK01_exacel01 successfully created
GridDisk DATA_CD_DISK02_exacel01 successfully created
GridDisk DATA_CD_DISK03_exacel01 successfully created
GridDisk DATA_CD_DISK04_exacel01 successfully created
GridDisk DATA_CD_DISK05_exacel01 successfully created
GridDisk DATA_CD_DISK06_exacel01 successfully created
GridDisk DATA_CD_DISK07_exacel01 successfully created
GridDisk DATA_CD_DISK08_exacel01 successfully created
GridDisk DATA_CD_DISK09_exacel01 successfully created
GridDisk DATA_CD_DISK10_exacel01 successfully created
GridDisk DATA_CD_DISK11_exacel01 successfully created
GridDisk DATA_CD_DISK12_exacel01 successfully created

[celladmin@Exacell ~]$ cellcli -e list griddisk
DATA_CD_DISK01_exacel01 active
DATA_CD_DISK02_exacel01 active
DATA_CD_DISK03_exacel01 active
DATA_CD_DISK04_exacel01 active
DATA_CD_DISK05_exacel01 active
DATA_CD_DISK06_exacel01 active
DATA_CD_DISK07_exacel01 active
DATA_CD_DISK08_exacel01 active
DATA_CD_DISK09_exacel01 active
DATA_CD_DISK10_exacel01 active
DATA_CD_DISK11_exacel01 active
DATA_CD_DISK12_exacel01 active

Step 16 : Cell status :

CellCLI> list cell detail
name: exacel01
bbuChargeThreshold: 800
bbuTempThreshold: 60
bmcType: absent
cellVersion: OSS_12.1.1.1.0_LINUX.X64_131219
cpuCount: 1
diagHistoryDays: 7
fanCount: 0/0
fanStatus: normal
flashCacheMode: WriteThrough
id: 6c71a8a8-c3b4-446e-9682-ecd268ef0adf
interconnectCount: 2
interconnect1: eth1
iormBoost: 0.0
ipaddress1: 10.0.0.100/24
kernelVersion: 2.6.32-642.el6.x86_64
makeModel: Fake hardware
metricHistoryDays: 7
offloadGroupEvents:
offloadEfficiency: 1.0
powerCount: 0/0
powerStatus: normal
releaseVersion: 12.1.1.1.0
releaseTrackingBug: 16980054
status: online
temperatureReading: 0.0
temperatureStatus: normal
upTime: 0 days, 0:06
cellsrvStatus: running
msStatus: running
rsStatus: running

Cell Configuration completed , Follow the below link for DB server step by step config for Exadata compute node

EXADATA DB COMPUTE NODE CONFIG

If you receive error CELL-01547

CELL-01547: CELLSRV STARTUP FAILED DUE TO UNKNOWN REASONS. ORA-600 [LINUXBLOCKIO::INIT]

Oldest

5 comments

Click here for comments
Unknown
admin
January 17, 2021 at 9:50 PM ×

Really simple steps to build exadata on vm

Reply
avatar
Harshit
admin
January 17, 2021 at 11:12 PM ×

Great steps was able to test exa storage on vm

Reply
avatar
saad
admin
January 22, 2021 at 1:12 AM ×

Hi
Thank you for this great informations.
I can't start celld. I have the error below

Getting the state of RS services... running
Starting CELLSRV services...
The STARTUP of CELLSRV services was not successful.
CELL-01537: Unable to read the cell_disk_config.xml file because the file is missing or empty.
Starting MS services...

I tried all solution without success:

Reply
avatar
kalyan
admin
June 28, 2021 at 5:59 PM ×

Hi, nice post. Do we have steps to create exadata simulator on OEL 7/8.

Reply
avatar
Sultan Khan
admin
October 29, 2021 at 4:06 PM ×

@kalyan Thanks for reading the post , will soon post the steps for the specific said versions

@Saad will post the solution for your error

Reply
avatar