Exadata Storage Cell – Administrating the Disks – Part 5

Exadata Storage server uses the cell software to manage the disks. Like volume manager, we need to build couple of virtual layers to get the grid disks.These griddisk will be used  to create the ASM disk group on the database level . In this article, we will see that how we can create/delete the celldisk, griddisk,flashcache & flashlog using the  cellcli utility as well as Linux command line.As i said earlier, we can also use flash disk to create the griddisks for high write intensive databases. But in most of the cases, we will be using those flash disks for flashcache and flashlog purposes due to the storage limitation.

Exadata Storage Architecture

The below diagram will explain that how the virtual storage objects are built on exadata storage server .

 

1. Login to the  exadata storage server celladmin and start cellcli utility.

[celladmin@uaexacell1 ~]$ id
uid=1000(celladmin) gid=500(celladmin) groups=500(celladmin),502(cellusers)
[celladmin@uaexacell1 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 - Production on Sun Nov 16 22:19:23 GMT+05:30 2014

Copyright (c) 2007, 2012, Oracle.  All rights reserved.
Cell Efficiency Ratio: 1

CellCLI>

2.List the physical disks. It lists all the attached harddisks and flash drives.

CellCLI> list physicaldisk
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK12    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK12    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK13    /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK13    normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH00   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH00   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH01   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH01   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH02   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH02   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH03   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH03   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH04   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH04   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH05   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH05   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH06   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH06   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH07   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH07   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH08   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH08   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH09   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH09   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH10   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH10   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH11   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH11   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH12   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH12   normal
         /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH13   /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH13   normal

CellCLI>

3.Check the existing celldisks.

CellCLI> LIST CELLDISK

CellCLI>

4. Create the celldisks on all disks. (we  normally do this)

CellCLI> CREATE CELLDISK ALL
CellDisk CD_DISK00_uaexacell1 successfully created
CellDisk CD_DISK01_uaexacell1 successfully created
CellDisk CD_DISK02_uaexacell1 successfully created
CellDisk CD_DISK03_uaexacell1 successfully created
CellDisk CD_DISK04_uaexacell1 successfully created
CellDisk CD_DISK05_uaexacell1 successfully created
CellDisk CD_DISK06_uaexacell1 successfully created
CellDisk CD_DISK07_uaexacell1 successfully created
CellDisk CD_DISK08_uaexacell1 successfully created
CellDisk CD_DISK09_uaexacell1 successfully created
CellDisk CD_DISK10_uaexacell1 successfully created
CellDisk CD_DISK11_uaexacell1 successfully created
CellDisk CD_DISK12_uaexacell1 successfully created
CellDisk CD_DISK13_uaexacell1 successfully created
CellDisk FD_00_uaexacell1 successfully created
CellDisk FD_01_uaexacell1 successfully created
CellDisk FD_02_uaexacell1 successfully created
CellDisk FD_03_uaexacell1 successfully created
CellDisk FD_04_uaexacell1 successfully created
CellDisk FD_05_uaexacell1 successfully created
CellDisk FD_06_uaexacell1 successfully created
CellDisk FD_07_uaexacell1 successfully created
CellDisk FD_08_uaexacell1 successfully created
CellDisk FD_09_uaexacell1 successfully created
CellDisk FD_10_uaexacell1 successfully created
CellDisk FD_11_uaexacell1 successfully created
CellDisk FD_12_uaexacell1 successfully created
CellDisk FD_13_uaexacell1 successfully created

CellCLI> LIST CELLDISK
         CD_DISK00_uaexacell1    normal
         CD_DISK01_uaexacell1    normal
         CD_DISK02_uaexacell1    normal
         CD_DISK03_uaexacell1    normal
         CD_DISK04_uaexacell1    normal
         CD_DISK05_uaexacell1    normal
         CD_DISK06_uaexacell1    normal
         CD_DISK07_uaexacell1    normal
         CD_DISK08_uaexacell1    normal
         CD_DISK09_uaexacell1    normal
         CD_DISK10_uaexacell1    normal
         CD_DISK11_uaexacell1    normal
         CD_DISK12_uaexacell1    normal
         CD_DISK13_uaexacell1    normal
         FD_00_uaexacell1        normal
         FD_01_uaexacell1        normal
         FD_02_uaexacell1        normal
         FD_03_uaexacell1        normal
         FD_04_uaexacell1        normal
         FD_05_uaexacell1        normal
         FD_06_uaexacell1        normal
         FD_07_uaexacell1        normal
         FD_08_uaexacell1        normal
         FD_09_uaexacell1        normal
         FD_10_uaexacell1        normal
         FD_11_uaexacell1        normal
         FD_12_uaexacell1        normal
         FD_13_uaexacell1        normal

CellCLI>

We have successfully created the celldisks on all the harddisks and flashdisks. This is one time activity and you no need to perform celldisk creation unless you replace any faulty drives.

5.To create the griddisk on all the harddisks , use the below command.

CellCLI> create griddisk ALL HARDDISK  PREFIX=CD_DISK
GridDisk CD_DISK_CD_DISK00_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK01_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK02_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK03_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK04_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK05_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK06_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK07_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK08_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK09_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK10_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK11_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK12_uaexacell1 successfully created
GridDisk CD_DISK_CD_DISK13_uaexacell1 successfully created

CellCLI>

6.If you want to create the griddisk with specific size & name,  use the below syntax,

CellCLI> CREATE GRIDDISK DATA01_DG celldisk = CD_DISK00_uaexacell1, size =100M
GridDisk DATA01_DG successfully created

CellCLI> list griddisk
         DATA01_DG       active

CellCLI> list griddisk detail
         name:                   DATA01_DG
         availableTo:
         cachingPolicy:          default
         cellDisk:               CD_DISK00_uaexacell1
         comment:
         creationTime:           2014-11-16T22:27:50+05:30
         diskType:               HardDisk
         errorCount:             0
         id:                     d681708b-9717-41fc-afad-78d61ca2f476
         offset:                 48M
         size:                   96M
         status:                 active

CellCLI>

If you are having the Exadata quarter rack , you need to create the same size grid disks on all the exadata storage cells. Oracle ASM will mirror across all the cell nodes for redundancy. When Database requires the additional space , its highly recommended to create the griddisk with existing griddisk size.

7.How to delete the griddisk ? Drop (delete) the specific griddisk using the below syntax

CellCLI>  list griddisk DATA01_DG
         DATA01_DG       active

CellCLI> drop  griddisk DATA01_DG
GridDisk DATA01_DG successfully dropped

CellCLI>  list griddisk DATA01_DG

CELL-02007: Grid disk does not exist: DATA01_DG
CellCLI>

8.You can also drop the bunch of grid disks using the prefix.Please see the below syntax.

CellCLI> list griddisk
         CD_DISK_CD_DISK00_uaexacell1    active
         CD_DISK_CD_DISK01_uaexacell1    active
         CD_DISK_CD_DISK02_uaexacell1    active
         CD_DISK_CD_DISK03_uaexacell1    active
         CD_DISK_CD_DISK04_uaexacell1    active
         CD_DISK_CD_DISK05_uaexacell1    active
         CD_DISK_CD_DISK06_uaexacell1    active
         CD_DISK_CD_DISK07_uaexacell1    active
         CD_DISK_CD_DISK08_uaexacell1    active
         CD_DISK_CD_DISK09_uaexacell1    active
         CD_DISK_CD_DISK10_uaexacell1    active
         CD_DISK_CD_DISK11_uaexacell1    active
         CD_DISK_CD_DISK12_uaexacell1    active
         CD_DISK_CD_DISK13_uaexacell1    active

CellCLI> drop griddisk all prefix=CD_DISK
GridDisk CD_DISK_CD_DISK00_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK01_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK02_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK03_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK04_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK05_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK06_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK07_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK08_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK09_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK10_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK11_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK12_uaexacell1 successfully dropped
GridDisk CD_DISK_CD_DISK13_uaexacell1 successfully dropped

CellCLI>

The above command deletes the griddisk which name starts from “CD_DISK”.

9. How to drop specific celldisk ? Drop  the specific celldisk using the below syntax.

CellCLI> list celldisk CD_DISK00_uaexacell1
         CD_DISK00_uaexacell1    normal

CellCLI> drop celldisk CD_DISK00_uaexacell1
CellDisk CD_DISK00_uaexacell1 successfully dropped

CellCLI> list celldisk CD_DISK00_uaexacell1

CELL-02525: Unknown cell disk: CD_DISK00_uaexacell1

CellCLI>

 Playing the Flashdisks

1. List the flashdisks

CellCLI> LIST CELLDISK where disktype=flashdisk
         FD_00_uaexacell1        normal
         FD_01_uaexacell1        normal
         FD_02_uaexacell1        normal
         FD_03_uaexacell1        normal
         FD_04_uaexacell1        normal
         FD_05_uaexacell1        normal
         FD_06_uaexacell1        normal
         FD_07_uaexacell1        normal
         FD_08_uaexacell1        normal
         FD_09_uaexacell1        normal
         FD_10_uaexacell1        normal
         FD_11_uaexacell1        normal
         FD_12_uaexacell1        normal
         FD_13_uaexacell1        normal

CellCLI>

Flashdisks will commonly used to create the flashcache and flashlog.

2.Configuring  specific  flashdisks as flashlog.

CellCLI>  CREATE FLASHLOG celldisk='FD_00_uaexacell1,FD_01_uaexacell1' , SIZE=100M
Flash log uaexacell1_FLASHLOG successfully created

CellCLI> LIST FLASHLOG
         uaexacell1_FLASHLOG     normal

CellCLI> LIST FLASHLOG DETAIL
         name:                   uaexacell1_FLASHLOG
         cellDisk:               FD_00_uaexacell1,FD_01_uaexacell1
         creationTime:           2014-11-16T23:02:50+05:30
         degradedCelldisks:
         effectiveSize:          96M
         efficiency:             100.0
         id:                     a12265f9-f80b-491b-a0e5-518b2143eede
         size:                   96M
         status:                 normal

CellCLI>

3.Configuring flashcache on specific flashdisks.

CellCLI> CREATE FLASHCACHE celldisk='FD_03_uaexacell1,FD_04_uaexacell1' , SIZE=100M
Flash cache uaexacell1_FLASHCACHE successfully created

CellCLI> LIST FLASHCACHE
         uaexacell1_FLASHCACHE   normal

CellCLI> LIST FLASHCACHE DETAIL
         name:                   uaexacell1_FLASHCACHE
         cellDisk:               FD_04_uaexacell1,FD_03_uaexacell1
         creationTime:           2014-11-16T23:04:50+05:30
         degradedCelldisks:
         effectiveCacheSize:     96M
         id:                     fe936779-abfc-4b70-a0d0-5146523cef48
         size:                   96M
         status:                 normal

CellCLI>

4.Deleting the flashlog.

CellCLI> DROP FLASHLOG
Flash log uaexacell1_FLASHLOG successfully dropped

CellCLI> LIST FLASHLOG

CellCLI>

5.Deleting the flashcache.

CellCLI> LIST FLASHCACHE
         uaexacell1_FLASHCACHE   normal

CellCLI> DROP FLASHCACHE
Flash cache uaexacell1_FLASHCACHE successfully dropped

CellCLI> LIST FLASHCACHE

CellCLI>

We need to invoke cellcli utility to manage the virtual storage objects. Is it possible manage the storage from command line ? Yes. You can manage the storage from linux command line. The below example will show that all the cellcli commands can be executed from the command line.you need to provide the command along with “cellcli -e” .

[celladmin@uaexacell1 ~]$  cellcli -e create griddisk all harddisk  prefix=UADB
GridDisk UADB_CD_DISK01_uaexacell1 successfully created
GridDisk UADB_CD_DISK02_uaexacell1 successfully created
GridDisk UADB_CD_DISK03_uaexacell1 successfully created
GridDisk UADB_CD_DISK04_uaexacell1 successfully created
GridDisk UADB_CD_DISK05_uaexacell1 successfully created
GridDisk UADB_CD_DISK06_uaexacell1 successfully created
GridDisk UADB_CD_DISK07_uaexacell1 successfully created
GridDisk UADB_CD_DISK08_uaexacell1 successfully created
GridDisk UADB_CD_DISK09_uaexacell1 successfully created
GridDisk UADB_CD_DISK10_uaexacell1 successfully created
GridDisk UADB_CD_DISK11_uaexacell1 successfully created
GridDisk UADB_CD_DISK12_uaexacell1 successfully created
GridDisk UADB_CD_DISK13_uaexacell1 successfully created
[celladmin@uaexacell1 ~]$  cellcli -e list griddisk where disktype=harddisk
         UADB_CD_DISK01_uaexacell1       active
         UADB_CD_DISK02_uaexacell1       active
         UADB_CD_DISK03_uaexacell1       active
         UADB_CD_DISK04_uaexacell1       active
         UADB_CD_DISK05_uaexacell1       active
         UADB_CD_DISK06_uaexacell1       active
         UADB_CD_DISK07_uaexacell1       active
         UADB_CD_DISK08_uaexacell1       active
         UADB_CD_DISK09_uaexacell1       active
         UADB_CD_DISK10_uaexacell1       active
         UADB_CD_DISK11_uaexacell1       active
         UADB_CD_DISK12_uaexacell1       active
         UADB_CD_DISK13_uaexacell1       active
[celladmin@uaexacell1 ~]$

Hope this article is informative to you.  Share it ! Comment it ! Be Sociable !!!