Dung (Donny) Nguyen

Senior Software Engineer

Installing and Setting up Oracle Grid Infrastructure

Oracle Grid Infrastructure is a foundational component for Oracle RAC, providing cluster management and storage solutions like ASM (Automatic Storage Management). Here’s a step-by-step guide to installing and setting up Oracle Grid Infrastructure:

1. Prerequisites

a. Hardware and System Requirements

Supported operating systems (e.g., Oracle Linux, Red Hat Enterprise Linux).
Sufficient CPU, memory, and disk space for Grid Infrastructure and database.
Private interconnect network for cluster communication.

b. Shared Storage

Configure shared storage (e.g., ASM disks, NFS, or EBS in cloud environments).
Ensure shared disks are accessible by all nodes in the cluster.

c. Software Requirements

Download the appropriate Oracle Grid Infrastructure installation media from Oracle’s website.
Ensure the operating system is updated with necessary packages using tools like yum.

d. Accounts and Permissions

Create the required operating system groups (oinstall, dba, asmadmin, etc.).
Create the grid and oracle users with appropriate group memberships.

2. Prepare the Environment

a. Set Kernel Parameters

Update /etc/sysctl.conf with the necessary kernel parameters:

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

Apply changes:

sysctl -p

b. Set User Limits

Update /etc/security/limits.conf for the grid user:

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240

c. Configure SSH Equivalence

Enable passwordless SSH for cluster communication:

ssh-keygen -t rsa
ssh-copy-id <other_node>

3. Installation Steps

a. Run the Installer

Extract the Grid Infrastructure Installation Media:

unzip grid_infrastructure.zip
cd grid_infrastructure

Launch the Installer:
```
./gridSetup.sh
```
Choose Installation Type:
- Select Configure Oracle Grid Infrastructure for a Cluster.
- Choose either Configure ASM or Cluster File System for storage.

b. Cluster Configuration

Specify the nodes in the cluster (provide node names and IP addresses).
Verify cluster node connectivity during installation.

c. Network Configuration

Assign network interfaces for the public network and private interconnect.

d. ASM Configuration

Choose Automatic Storage Management as the storage option.
Configure ASM disk groups (e.g., DATA and FRA) and assign redundancy levels (e.g., normal, high).

e. Run Configuration Scripts

At the end of the installer, run the provided scripts (root.sh) on each node:

 sudo /u01/app/oraInventory/orainstRoot.sh
 sudo /u01/app/12.2.0/grid/root.sh

f. Verify Installation

Use the cluvfy utility to validate the cluster configuration:

 /u01/app/12.2.0/grid/bin/cluvfy stage -post crsinst -n <node1,node2>

4. Post-Installation Configuration

a. Create ASM Disk Groups

Use the ASM Configuration Assistant (asmca) to create additional disk groups if needed:

/u01/app/12.2.0/grid/bin/asmca

b. Configure Oracle Clusterware Services

Add databases or other services to the cluster using srvctl:

 srvctl add database -db <db_name> -spfile +DATA/<db_name>/spfile.ora -startoption OPEN

c. Setup Automatic Startup

Ensure Clusterware starts automatically during system boot:

crsctl enable crs

d. Monitor the Cluster

Check cluster status:
```
 crsctl status resource -t
```
Monitor ASM instances:
```
 asmcmd lsdg
```

5. Best Practices

Use Oracle Enterprise Manager (OEM) for cluster management and monitoring.
Regularly patch Grid Infrastructure using Oracle’s OPatch utility.
Test high availability and failover scenarios.

This process sets up Oracle Grid Infrastructure, enabling you to use features like Oracle RAC or ASM for high availability and scalable storage.