Solaris Useful Commands

find command

 

1.- Sort the 40 largest files :

#find /apps/ora11/product -xdev -ls | sort -nrk 7 | head -40

2.-Find a file name with a specific name :

#find . -name “*.jar" -print

3.-Another variant could be without specifying the file extension :

find . -name  jar  -print

du command

du provides output in a readable format, easier to be understood. You will be able to see the information in Bytes, Kilobytes, Megabytes, Gigabytes, etc.

1.- Check all directories and files size in gigas.

#du -sh *
1.5G   agent11g
819M   agent12c
120M   apex
294M   assistants
414M   bin
27M   ccr
3K   cdata
2.4M   cfgtoollogs
42K   clone
1K   config
7.8M   crs
17K   csmig
156K   css
326M   ctx
2.8M   cv

2.-check size for 20 biggest files ordered by size

#du -a | sort -n -r| head -20

iostat command

 

1.-Check disk performance usage in solaris :

#iostat
tty        md10          md11          md12          md20           cpu
tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
0  153  14   2   12    9   2   10    9   2    8    0   0   11   11  2  0 86

The command display all the devices

#iostat -xn 5

extended device statistics
r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
81.0   12.5 10131.5  847.7  0.0  2.4    0.0   25.3   0  42 ssd163
0.4    0.3   30.7    6.1  0.0  0.0    0.0   14.6   0   0 ssd164
54.0    8.9 6738.1  532.1  0.0  0.7    0.0   11.3   0  16 ssd165
0.9    2.5   12.6   14.4  0.0  0.0    0.0    1.9   0   0 ssd166
0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.3   0   0 ssd167
2.3    3.0  149.9   29.9  0.0  0.0    0.0    5.0   0   1 ssd168
0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.3   0   0 ssd169
0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.2   0   0 ssd170
0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.3   0   0 ssd171
162.6   16.3 20423.9 1181.9  0.0  1.8    0.0   10.3   0  29 ssd172
133.9   15.8 16853.3 1128.4  0.0  1.2    0.0    8.0   0  22 ssd173
162.6   16.3 20423.9 1181.9  0.0  1.8    0.0   10.3   0  29 ssd172
133.9   15.8 16853.3 1128.4  0.0  1.2    0.0    8.0   0  22 ssd173
85.8    8.5 10808.2  510.2  0.0  1.0    0.0   11.0   0  19 ssd198
44.9    8.1 5599.1  467.6  0.0  0.6    0.0   11.5   0  14 ssd199
44.6    8.1 5558.4  467.7  0.0  0.6    0.0   12.0   0  15 ssd200
126.5   22.9 15817.3 1806.7  0.0  1.2    0.0    7.9   0  21 ssd201

Important Pay special attention to the %w  %b devicecolumns.
%w : Is the wait is the expected rate and should not exceed
%b : Is the disk busy percentage. The most important is that if to a high percentaje, it get free quickly.
device : Is the device disk name.

#iostat -n
tty        md10          md11          md12          md20           cpu
tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
0  153  14   2   12    9   2   10    9   2    8    0   0   11   11  2  0 86

Print logical disk names

#iostat -en

—- errors —
s/w h/w trn tot device
0   0   0   0 md10
0   0   0   0 md11
0   0   0   0 md12
0   0   0   0 md20
0   0   0   0 md21
0   0   0   0 md22
0   0   0   0 md50
0   0   0   0 md51
0   0   0   0 md52
0   0   0   0 sd0
0   0   0   0 sd1
0   0   0   0 sd2
0   0   0   0 ssd72
0   0   0   0 ssd194
0   0   0   0 ssd195
0   0   0   0 ssd196
0   0   0   0 ssd197
0   0   0   0 ssd198
0   0   0   0 ssd199
0   0   0   0 ssd200
0   0   0   0 ssd201

Soft error      : A disk sector fails the CRC check and needs to be re-read
Hard error      : Re-read fails several times for CRC check
Transport error : Errors reported by I/O bus
Total errors    : Soft error + Hard error + Transport errors

#iostat -E

sd0       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: H106030SDSUN300G Revision: A2B0 Serial No: 1219NV1M6B
Size: 300.00GB <300000000000 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd1       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: TEAC     Product: DV-W28SS-V       Revision: 1.0B Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 2 Predictive Failure Analysis: 0
sd2       Soft Errors: 0 Hard Errors: 0 Transport Errors: 0

The errors details can be displayed with -e option.

prstat command

prstat -a :  The prstat commands show statistics and projects that currently are running on your system. With -a option shows the sum the total for each column for each user group by number of processes.
#prstat -a
NPROC USERNAME  SWAP   RSS MEMORY      TIME  CPU
1016 ora11      62G   38G    30% 171:39:20  17%
13 remedyad 3517M 3104M   2.4%  62:36:47 0.6%
30 root      819M  509M   0.4%  16:47:34 0.0%
7 xymon    4608K 6696K   0.0%   0:01:32 0.0%
1 nagios    536K 3616K   0.0%   0:08:11 0.0%
1 noaccess   86M   98M   0.1%   1:01:23 0.0%
1 smmsp    2264K 7400K   0.0%   0:00:24 0.0%
6 daemon   8608K 8976K   0.0%   0:01:27 0.0%

Total: 1075 processes, 12238 lwps, load averages: 7.16, 6.77, 6.56

At the end, you will be able to see the load averages, That is an average for 5, 10 and 15 minutes respectively.

#prstat -J : Same above but group by project :

PROJID    NPROC  SWAP   RSS MEMORY      TIME  CPU PROJECT
100     1011   62G   38G    30% 171:52:09  17% group.dba
3       20 3522M 3109M   2.4%  62:39:44 0.8% default
0       39  916M  613M   0.5%  17:59:02 0.0% system

#prstat -T :  Group by task id :

TASKID    NPROC  SWAP   RSS MEMORY      TIME  CPU PROJECT
225      941   61G   38G    29% 148:11:16  16% group.dba
59726       13 3517M 3104M   2.4%  62:38:24 0.7% default
222224        3 5256K   13M   0.0%   0:00:06 0.6% group.dba
10542       37   14G 7049M   5.4%  12:42:34 0.0% group.dba
216        2  148M  106M   0.1%   4:44:04 0.0% system
162        5  556M  294M   0.2%   9:42:22 0.0% system
855       32   16G   11G   8.5%  11:02:48 0.0% group.dba
177        1 9264K 9264K   0.0%   0:50:29 0.0% system
201        1 8808K   11M   0.0%   0:00:52 0.0% system
206        2 1872K 4568K   0.0%   0:00:16 0.0% system
194        1  816K 2352K   0.0%   0:00:00 0.0% system

Total: 1072 processes, 12209 lwps, load averages: 7.31, 6.75, 6.57

id Command

The id command display the OS current username and group and with -p option prints the current project ID in addition to the user and group IDs

#id -p
uid=1000(ora11) gid=1000(dba) projid=100(group.dba)

 

http://unixaddiction.blogspot.com/2015/09/useful-commands-in-unix.html

 

 

hpux check disk

Check bad sector

more /var/adm/syslog/syslog.log
And most important: make sure the online diagnostics are loaded and running. They will monitor disk error messages and can be configured to notify you when there are error messages. The file: /var/adm/syslog/syslog.log will also have disk errors logged.

ioscan –fCdisk
Use the ioscan command to check the S/W state of the disk. Only disks in state CLAIMED are currently accessible by the system. Disks in other states such as NO_HW or disks that are completely missing from the ioscan output are suspicious. If the disk is marked as CLAIMED, then its controller is responding. For example: # ioscan –fCdisk 
Class I H/W Path Driver S/W State H/W Type Description
======
disk 0 8/4.5.0 sdisk CLAIMED DEVICE SEAGATE ST34572WC
disk 1 8/4.8.0 sdisk UNCLAIMED UNKNOWN SEAGATE ST34572WC
disk 2 8/16/5.2.0 sdisk CLAIMED DEVICE TOSHIBA CD-ROM XM-5401TA
In this example the disk at hardware path 8/4.8.0 is not accessible. If the disk has multiple hardware paths, be sure to check all the paths.


diskinfo –v /dev/rdsk/cXtYdZ
The diskinfo for this disk showing 0 bytes , it is probably bad.

dd if=dev/rdsk/cXtYdZ of=/dev/null bs=1024k
if this exits without I/O error, the disk is assumed OK
For example, the following shows a successful read of the first 64 megabytes of the disk:
dd if=/dev/rdsk/c0t5d0 of=/dev/null bs=1024k count=64
64+0 records in
64+0 records out
The following shows an unsuccessful read of the whole disk:
dd if=/dev/rdsk/c1t3d0 of=/dev/null bs=1024k
dd read error: I/O error
0+0 records in
0+0 records out



Check LVM Errors 
Sometimes LVM commands, such as vgdisplay, return an error suggesting that a disk has problems. For example:
vgdisplay –v | more 

— Physical volumes —
PV Name /dev/dsk/c0t3d0
PV Status unavailable 
Total PE 1023
Free PE 173

The physical volume status of unavailable indicates that LVM is having problems with the disk. You can get the same status information from pvdisplay.
The next two examples are warnings from vgdisplay and vgchange indicating that LVM has no contact with a disk:
vgdisplay -v vg vgdisplay: Warning: couldn’t query physical volume “/dev/dsk/c0t3d0″: The specified path does not correspond to physical volume attached to this volume group vgdisplay: Warning: couldn’t query all of the physical volumes.
vgchange -a y /dev/vg01 vgchange: Warning: Couldn’t attach to the volume group physical volume “/dev/dsk/c0t3d0″: A component of the path of the physical volume does not exist. Volume group “/dev/vg01″ has been successfully changed.
Another sign that you might have a disk problem is seeing stale extents in the output from lvdisplay. If you have stale extents on a logical volume even after running the vgsync or lvsync commands, you might have an issue with an I/O path or one of the disks used by the logical volume, but not necessarily the disk showing stale extents. For example:
lvdisplay –v /dev/vg01/lvol3 | more 

LV Status available/stale 

— Logical extents — LE PV1 PE1 Status 1 PV2 PE2 Status 2 0000 /dev/dsk/c0t3d0 0000 current /dev/dsk/c1t3d0 0100 current 0001 /dev/dsk/c0t3d0 0001 current /dev/dsk/c1t3d0 0101 current 0002 /dev/dsk/c0t3d0 0002 current /dev/dsk/c1t3d0 0102 stale 0003 /dev/dsk/c0t3d0 0003 current /dev/dsk/c1t3d0 0103 stale … 
All LVM error messages tell you which device file is associated with the problematic disk


Check File system corrupt(no disk problem)
umount /fs
fsck -F vxfs -y -o full,nolog /dev/vg01/rlvol1
mount /dev/vg01/rlvol1

check file system consistency check and interactive repair and not fix any errors found
http://kmit4u.blogspot.hk/2010/06/unix-hp-ux-how-to-check-bad-disk.html

 

Datafile Size

SET PAUSE ON
SET PAUSE 'Press Return to Continue'
SET PAGESIZE 60
SET LINESIZE 300
COLUMN "Tablespace Name" FORMAT A20
COLUMN "File Name" FORMAT A80
 
SELECT  Substr(df.tablespace_name,1,20) "Tablespace Name",
        Substr(df.file_name,1,80) "File Name",
        Round(df.bytes/1024/1024,0) "Size (M)",
        decode(e.used_bytes,NULL,0,Round(e.used_bytes/1024/1024,0)) "Used (M)",
        decode(f.free_bytes,NULL,0,Round(f.free_bytes/1024/1024,0)) "Free (M)",
        decode(e.used_bytes,NULL,0,Round((e.used_bytes/df.bytes)*100,0)) "% Used"
FROM    DBA_DATA_FILES DF,
       (SELECT file_id,
               sum(bytes) used_bytes
        FROM dba_extents
        GROUP by file_id) E,
       (SELECT Max(bytes) free_bytes,
               file_id
        FROM dba_free_space
        GROUP BY file_id) f
WHERE    e.file_id (+) = df.file_id
AND      df.file_id  = f.file_id (+)
ORDER BY df.tablespace_name,
         df.file_name
/

ASM Datafiles Commands

Datafiles

Extending an ASM Datafile

The following example shows how to extend a datafile to the APP_DATA tablespace in the +DATA ASM group.

1. Identify the datafiles for the tablespace you want to increase.

SQL> select file_name,bytes/1024/1024 mb
aaaaaafrom dba_data_files
aaaaaawhere tablespace_name = ‘APP_DATA’
aaaaaaorder by file_name;

FILE_NAMEaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaMB
———————————————————— ——-
+DATA/SID/datafile/app_data.259.669898683aaaaaaaaaaaaaaaa20000
+DATA/SID/datafile/app_data.267.636216519aaaaaaaaaaaaaaaa28100

2. Extened the Datafile

SQL> alter database datafile ‘+DATA/SID/datafile/app_data.259.669898683’ resize 30000m;

Tablespace altered.


Adding an ASM Datafile

The following example shows how to add a datafile to the APP_DATA tablespace in the +DATA ASM group.

1. Find out where the current datafiles are defined. In this example the ASM group is +STORE1.

SQL> select file_name,bytes/1024/1024 mb
aaaaaafrom dba_data_files
aaaaaawhere tablespace_name = ‘APP_DATA’
aaaaaaorder by file_name;

2. Add the datafile.

SQL> alter tablespace APP_DATA add datafile ‘+DATA’ size 20000m;

Tablespace altered.

Oracle RAC Logs

The Cluster Ready Services Daemon (crsd) Log Files

Log files for the CRSD process (crsd) can be found in the following directories:

                 CRS home/log/hostname/crsd

Oracle Cluster Registry (OCR) Log Files

The Oracle Cluster Registry (OCR) records log information in the following location:

                CRS Home/log/hostname/client

Cluster Synchronization Services (CSS) Log Files

You can find CSS information that the OCSSD generates in log files in the following locations:

                CRS Home/log/hostname/cssd

Event Manager (EVM) Log Files

Event Manager (EVM) information generated by evmd is recorded in log files in the following locations:

                CRS Home/log/hostname/evmd

RACG Log Files

The Oracle RAC high availability trace files are located in the following two locations:

CRS home/log/hostname/racg

$ORACLE_HOME/log/hostname/racg

Core files are in the sub-directories of the log directories. Each RACG executable has a sub-directory assigned exclusively for that executable. The name of the RACG executable sub-directory is the same as the name of the executable.

You can follow below table which define locations of logs files:

Oracle Clusterware log files

Cluster Ready Services Daemon (crsd) Log Files:
$CRS_HOME/log/hostname/crsd

Cluster Synchronization Services (CSS):
$CRS_HOME/log/hostname/cssd

Event Manager (EVM) information generated by evmd:
$CRS_HOME/log/hostname/evmd

Oracle RAC RACG:
$CRS_HOME/log/hostname/racg
$ORACLE_HOME/log/hostname/racg

Oracle RAC 11g Release 2 log files

Clusterware alert log:
$GRID_HOME/log/<host>/alert<host>.log

Disk Monitor daemon:
$GRID_HOME/log/<host>/diskmon

OCRDUMP, OCRCHECK, OCRCONFIG, CRSCTL:
$GRID_HOME/log/<host>/client

Cluster Time Synchronization Service:
$GRID_HOME/log/<host>/ctssd

Grid Interprocess Communication daemon:
$GRID_HOME/log/<host>/gipcd

Oracle High Availability Services daemon:
$GRID_HOME/log/<host>/ohasd

Cluster Ready Services daemon:
$GRID_HOME/log/<host>/crsd

Grid Plug and Play daemon:
$GRID_HOME/log/<host>/gpnpd:

Mulitcast Domain Name Service daemon:
$GRID_HOME/log/<host>/mdnsd

Event Manager daemon:
$GRID_HOME/log/<host>/evmd

RAC RACG (only used if pre-11.1 database is installed):
$GRID_HOME/log/<host>/racg

Cluster Synchronization Service daemon:
$GRID_HOME/log/<host>/cssd

Server Manager:
$GRID_HOME/log/<host>/srvm

HA Service Daemon Agent:
$GRID_HOME/log/<host>/agent/ohasd/oraagent_oracle11

HA Service Daemon CSS Agent:
$GRID_HOME/log/<host>/agent/ohasd/oracssdagent_root

HA Service Daemon ocssd Monitor Agent:
$GRID_HOME/log/<host>/agent/ohasd/oracssdmonitor_root

HA Service Daemon Oracle Root Agent:
$GRID_HOME/log/<host>/agent/ohasd/orarootagent_root

CRS Daemon Oracle Agent:
$GRID_HOME/log/<host>/agent/crsd/oraagent_oracle11

CRS Daemon Oracle Root Agent:
$GRID_HOME/log/<host> agent/crsd/orarootagent_root

Grid Naming Service daemon:
$GRID_HOME/log/<host>/gnsd

RAC Cheatsheet

RAC Cheatsheet

This is a quick and dirty cheatsheet on Oracle RAC 10g, as my experience with RAC grows I will update this section, below is a beginners guide on the commands and information that you will require to administer Oracle RAC.

Acronyms

Acronyms
GCS Global Cache Services in memory database containing current locks and awaiting locks, also known as PCM
GES Global Enqueue Services coordinates the requests of all global enqueues uses the GCS, also known as non-PCM
GRD Global Resource Directory all resources available to the cluster (formed and managed by GCS and GES), see GRDfor more details
GRM Global Resource Manager helps to coordinate and communicate the locks requests between Oracle processes
GSD Global Services Daemon runs on each node with one GSD process per node. The GSD coordinates with the cluster manager to receive requests from clients such as the DBCA, EM, and the SRVCTL utility to execute administrative job tasks such as instance startup or shutdown. The GSD is not an Oracle instance background process and is therefore not started with the Oracle instance
PCM (IDLM) Parallel Cache Management formly know as (integrated) Distributed Lock Manager, its another name for GCS
Resource n/a it is a identifiable entity it basically has a name or a reference, it can be a area in memory, a disk file or an abstract entity
Resource (Global) n/a a resource that can be accessed by all the nodes within the cluster examples would be the following

  • Data Buffer Cache Block
  • Transaction Enqueue
  • Database Data Structures
LVB Lock Value Block contains a small amount of data regarding the lock
TRFC Traffic Controller controls the DLM traffic between instances (messaging tickets)

Files and Directories

Files and Directories
$ORA_CRS_HOME/cdata/<cluster_name> OCR backups (default location)
$ORA_HOME/log/<hostname>/client/ocrconfig_<pid>.log OCR command log file
$ORA_CRS_HOME/crs/log contains trace files for the CRS resources
$ORA_CRS_HOME/crs/init contains trace files for the CRS daemon during startup, a good place to start
$ORA_CRS_HOME/css/log contains cluster reconfigurations, missed check-ins, connects and disconnects from the client CSS listener. Look here to obtain when reboots occur
$ORA_CRS_HOME/css/init contains core dumps from the cluster synchronization service daemon (OCSd)
$ORA_CRS_HOME/evm/log logfiles for the event volume manager and eventlogger daemon
$ORA_CRS_HOME/evm/init pid and lock files for EVM
$ORA_CRS_HOME/srvm/log logfiles for Oracle Cluster Registry (OCR)
$ORA_CRS_HOME/log log fles for Oracle clusterware which contains diagnostic messages at the Oracle cluster level

Useful Views/Tables

GCS and Cache Fusion Diagnostics
v$cache contains information about every cached block in the buffer cache
v$cache_transfer contains information from the block headers in SGA that have been pinged at least once
v$instance_cache_transfer contains information about the transfer of cache blocks through the interconnect
v$cr_block_server contains statistics about CR block transfer across the instances
v$current_block_server contains statistics about current block transfer across the instances
v$gc_element contains one-to-one information for each global cache resource used by the buffer cache
GES diagnostics
v$lock contains information about locks held within a database and outstanding requests for locks and latches
v$ges_blocking_enqueue contains information about locks that are being blocked or blocking others and locks that are known to the lock manager
v$enqueue_statistics contains details about enqueue statistics in the instance
v$resource_limits display enqueue statistics
v$locked_object contains information about DML locks acquired by different transactions in databases with their mode held
v$ges_statistics contains miscellaneous statistics for GES
v$ges_enqueue contains information about all locks known to the lock manager
v$ges_convert_local contains information about all local GES operations
v$ges_convert_remote contains information about all remote GES operations
v$ges_resource contains information about all resources known to the lock manager
v$ges_misc contains information about messaging traffic information
v$ges_traffic_controller contains information about the message ticket usage
Dynamic Resource Remastering
v$hvmaster_info contains information about current and previous master instances of GES resources in relation to hash value ID of resource
v$gcshvmaster_info the same as above but globally
v$gcspfmaster_info conatins information about current and previous masters about GCS resources belonging to files mapped to a particular master, including the number of times the resource has remastered
Cluster Interconnect
v$cluster_interconnects contains information about interconnects that are being used for cluster communication
v$configured_interconnects same as above but also contains interconnects that AC is aware off that are not being used
Miscellanous
v$service services running on an instance
x$kjmsdp display LMS daemon statistics
x$kjmddp display LMD daemon statistics

Useful Parameters

Parameters
cluster_interconnects specify a specific IP address to use for the inetrconnect
_gcs_fast_config enables fast reconfiguration for gcs locks (true|false)
_lm_master_weight controls which instance will hold or (re)master more resources than others
_gcs_resources controls the number of resources an instance will master at a time
_lm_tickets controls the number of message tickets
_lm_ticket_active_sendback controls the number of message tickets (aggressive messaging)
_db_block_max_cr_dba limits the number of CR copies per DBA on the buffer cache (see grd)
_fairness_threshold used when too many CR requested arrive for a particular buffer and the block becomes disowned (see grd)
_gc_affinity_time specifies interval minutes for reamstering
_gc_affinity_limit defines the number of times a instance access the resource before remastering
_gc_affinity_minimum defines the minimum number of times a instance access the resource before remastering
_lm_file_affinity disables dynamic remastering for the objects belonging to those files
_lm_dynamic_remastering enable or disable remastering
_gc_defer_time define the time by which an instance deferred downgrading a lock (see Cache Fusion)
_lgwr_async_broadcast change the SCN boardcast method (see troubleshooting)

Processes

Oracle RAC Daemons and Processes
OPROCd Process Monitor provides basic cluster integrity services
EVMd Event Management spawns a child process event logger and generates callouts
OCSSd Cluster Synchronization Services basic node membership, group services, basic locking
CRSd Cluster Ready Services resource monitoring, failover and node recovery
LMSn
Lock Manager Server process – GCS this is the cache fusion part, it handles the consistent copies of blocks that are tranferred between instances. It receives requests from LMD to perform lock requests. I rools back any uncommitted transactions. There can be upto ten LMS processes running and can be started dynamically if demand requires it.

they manage lock manager service requests for GCS resources and send them to a service queue to be handled by the LMSn process. It also handles global deadlock detection and monitors for lock conversion timeouts.

LMON
Lock Monitor Process – GES this process manages the GES, it maintains consistency of GCS memory in case of process death. It is also responsible for cluster reconfiguration and locks reconfiguration (node joining or leaving), it checks for instance deaths and listens for local messaging.

A detailed log file is created that tracks any reconfigurations that have happened.

LMD
Lock Manager Daemon – GES this manages the enqueue manager service requests for the GCS. It also handles deadlock detention and remote resource requests from other instances.
LCK0
Lock Process – GES manages instance resource requests and cross-instance call operations for shared resources. It builds a list of invalid lock elements and validates lock elements during recovery.
DIAG
Diagnostic Daemon This is a lightweight process, it uses the DIAG framework to monitor the healt of the cluster. It captures information for later diagnosis in the event of failures. It will perform any neccessary recovery if an operational hang is detected.

General Administration

Managing the Cluster
starting /etc/init.d/init.crs start

crsctl start crs

stopping /etc/init.d/init.crs stop

crsctl stop crs

enable/disable at boot time /etc/init.d/init.crs enable
/etc/init.d/init.crs disable

crsctl enable crs
crsctl disable crs

Managing the database configuration with SRVCTL
start all instances srvctl start database -d <database> -o <option>

Note: starts listeners if not already running, you can use the -o option to specify startup/shutdown options

force
open
mount
nomount

stop all instances srvctl stop database -d <database> -o <option>

Note: the listeners are not stopped, you can use the -o option to specify startup/shutdown options

immediate
abort
normal
transactional

start/stop particular instance srvctl [start|stop] database -d <database> -i <instance>,<instance>
display the registered databases srvctl config database
status srvctl status database -d <database>
srvctl status instance -d <database> -i <instance>,<instance>
srvctl status service -d <database>
srvctl status nodeapps -n <node>
srvctl status asm -n <node>
stopping/starting srvctl stop database -d <database>
srvctl stop instance -d <database> -i <instance>,<instance>
srvctl stop service -d <database> -s <service>,<service> -i <instance>,<instance>
srvctl stop nodeapps -n <node>
srvctl stop asm -n <node>

srvctl start database -d <database>
srvctl start instance -d <database> -i <instance>,<instance>
srvctl start service -d <database> -s <service>,<service> -i <instance>,<instance>
srvctl start nodeapps -n <node>
srvctl start asm -n <node>

adding/removing srvctl add database -d <database> -o <oracle_home>
srvctl add instance -d <database> -i <instance> -n <node>
srvctl add service -d <database> -s <service> -r <preferred_list>
srvctl add nodeapps -n <node> -o <oracle_home> -A <name|ip>/network
srvctl add asm -n <node> -i <asm_instance> -o <oracle_home>

srvctl remove database -d <database> -o <oracle_home>
srvctl remove instance -d <database> -i <instance> -n <node>
srvctl remove service -d <database> -s <service> -r <preferred_list>
srvctl remove nodeapps -n <node> -o <oracle_home> -A <name|ip>/network
srvctl asm remove -n <node>

OCR utilities
log file $ORA_HOME/log/<hostname>/client/ocrconfig_<pid>.log
checking ocrcheck

Note: will return the OCR version, total space allocated, space used, free space, location of each device and the result of the integrity check

dump contents ocrdump -backupfile <file>

Note: by default it dumps the contents into a file named OCRDUMP in the current directory

export/import ocrconfig -export <file>

ocrconfig -restore <file>

backup/restore # show backups
ocrconfig -showbackup

# to change the location of the backup, you can even specify a ASM disk
ocrconfig -backuploc <path|+asm>

# perform a backup, will use the location specified by the -backuploc location
ocrconfig -manualbackup

# perform a restore
ocrconfig -restore <file>

# delete a backup
orcconfig -delete <file>

Note: there are many more option so see the ocrconfig man page

add/remove/replace ## add/relocate the ocrmirror file to the specified location
ocrconfig -replace ocrmirror ‘/ocfs2/ocr2.dbf’

## relocate an existing OCR file
ocrconfig -replace ocr ‘/ocfs1/ocr_new.dbf’

## remove the OCR or OCRMirror file
ocrconfig -replace ocr
ocrconfig -replace ocrmirror

CRS Administration

CRS Administration
starting ## Starting CRS using Oracle 10g R1
not possible

## Starting CRS using Oracle 10g R2
$ORA_CRS_HOME/bin/crsctl start crs

stopping ## Stopping CRS using Oracle 10g R1
srvctl stop database -d <database>
srvctl stop asm -n <node>
srvctl stop nodeapps -n <node>
/etc/init.d/init.crs stop

## Stopping CRS using Oracle 10g R2
$ORA_CRS_HOME/bin/crsctl stop crs

disabling/enabling ## use to stop CRS restarting after a reboot

## Oracle 10g R1
/etc/init.d/init.crs [disable|enable]

## Oracle 10g R2
$ORA_CRS_HOME/bin/crsctl [disable|enable] crs

checking $ORA_CRS_HOME/bin/crsctl check crs
$ORA_CRS_HOME/bin/crsctl check evmd
$ORA_CRS_HOME/bin/crsctl check cssd
$ORA_CRS_HOME/bin/crsctl check crsd
$ORA_CRS_HOME/bin/crsctl check install -wait 600
Resource Applications (CRS Utilities)
status $ORA_CRS_HOME/bin/crs_stat
create profile $ORA_CRS_HOME/bin/crs_profile
register/unregister application $ORA_CRS_HOME/bin/crs_register
$ORA_CRS_HOME/bin/crs_unregister
Start/Stop an application $ORA_CRS_HOME/bin/crs_start
$ORA_CRS_HOME/bin/crs_stop
Resource permissions $ORA_CRS_HOME/bin/crs_getparam
$ORA_CRS_HOME/bin/crs_setparam
Relocate a resource $ORA_CRS_HOME/bin/crs_relocate
Nodes
member number/name olsnodes -n
local node name olsnodes -l
activates logging olsnodes -g
Oracle Interfaces
display oifcfg getif
delete oicfg delig -global
set oicfg setif -global <interface name>/<subnet>:public
oicfg setif -global <interface name>/<subnet>:cluster_interconnect
Global Services Daemon Control
starting gsdctl start
stopping gsdctl stop
status gsdctl status
Cluster Configuration (clscfg is used during installation)
create a new configuration clscfg -install
upgrade or downgrade and existing configuration clscfg -upgrade
clscfg -downgrade
add or delete a node from the configuration clscfg -add
clscfg -delete
create a special single-node configuration for ASM clscfg -local
brief listing of terminology used in the other nodes clscfg -concepts
used for tracing clscfg -trace
help clscfg -h
Cluster Name Check
print cluster name cemulto -n

Note: in Oracle 9i the ulity was called “cemutls"

print the clusterware version cemulto -w

Note: in Oracle 9i the ulity was called “cemutls"

Node Scripts
Add Node addnode.sh

Note: see adding and deleting nodes

Delete Node deletenode.sh

Note: see adding and deleting nodes

Enqueues
displaying statistics SQL> column current_utilization heading current
SQL> column max_utilization heading max_usage
SQL> column initial_allocation heading initial
SQL> column resource_limit format a23;

SQL> select * from v$resource_limit;

Messaging (tickets)
ticket usage select local_nid local, remote_nid remote, tckt_avail avail, tckt_limit limit, snd_q_len send_queue, tckt_wait waiting from v$ges_traffic_controller;
dump ticket information SQL> oradebug setmypid
SQL> oradebug unlimit
SQL> oradebug lkdebug -t
Lighwork Rule and Fairness Threshold
downconvert select cr_requests, light_works, data_requests, fairness_down_converts from v$cr_block_server;

Note: lower the _fairness_threshold if the ratio goes above 40%, set to 0 if the instance is a query only instance.

Remastering
force dynamic remastering (DRM) ## Obtain the OBJECT_ID form the below table
SQL> select * from v$gcspfmaster_info;

## Determine who masters it
SQL> oradebug setmypid
SQL> oradebug lkdebug -a <OBJECT_ID>

## Now remaster the resource
SQL> oradebug setmypid
SQL> oradebug lkdebug -m pkey <OBJECT_ID>

GRD, SRVCTL, GSD and SRVCONFIG Tracing
Enable tracing  $ export SRVM_TRACE=true
Disable tracing  $ export SRVM_TRACE=""

Voting Disk

adding crsctl add css votedisk <file>
deleting crsctl delete css votedisk <file>
querying crsctl query css votedisk

Commonly used srvctl commands

Commonly used srvctl commands

What is srvctl? it is the Server Control Utility, we use SRVCTL to start and stop the database and instances, manage configuration information, and to add, move or remove instances and services.

These are some of the srvctl commands I frequently use, this is not a complete reference guide.

I organized the commands as follow:

  • To start a rac database
  • To stop a rac database
  • To check status and configurations
  • To start and stop instances
  • To start, stop and manage services

Start a rac database (order: nodeapps – asm – database)

srvctl start nodeapps -n nodename
srvctl start asm -n nodename
srvctl start database -d dbname
options are: srvctl start database -d dbname -o open | -o mount | -o nomount

Stop a rac database (order: database – asm – nodeapps)
srvctl stop database -d dbname -o immediate
options are: srvctl stop database -d dbname -o normal | -o transactional | -o immediate | -o abort

srvctl stop asm -n nodename
options are: srvctl stop asm -n nodename -o immediate

srvctl stop nodeapps -n nodename

To check status and configurations
Nodeapps:
srvctl status nodeapps -n nodename

srvctl config nodeapps -n nodename

ASM:
srvctl status asm -n nodename
srvctl config asm -n nodename

Database:
srvctl status database -d dbname
srvctl config database -d dbname (shows instances name, node and oracle home)

Instance:
srvctl status instance -d dbname -i instancename

Services:
srvctl status service -d dbname

To start and stop instances

srvctl start instance -d dbname -i instancename
srvctl stop instance -d dbname -i instancename

To start, stop and manage services
srvctl status service -d dbname
srvctl config service -d dbname
srvctl start service -d dbname -s servicename
srvctl stop service -d dbname -s servicename

srvctl relocate service -d dbname -s servicename -i instancename -t newinstancename [-f]

srvctl documentation for Oracle 10G R2 and 11G