cdot

NetApp cDOT – clustershell

Let’s go a little bit deeper into clustershell. In my last post (NetApp cDOT – Clustered Data ONTAP Command Line) I have described clustershell as a default CLI shell you will use when managing Clustered Data ONTAP. What I haven’t described yet, is how it actually works, what additional features it has – comparing to 7-mode admin shell.

Command directories

In Clustered Data ONTAP commands are organized into a hierarchy by command directories. If you are familiar with managing NetApp 7-mode, you are aware that there are, for instance, bunch of commands related with volume, such us ‘vol create’, ‘vol rename’, ‘vol status’ etc. NetApp cDOT has very similar structure, but it went one step futher creating a directories of commands. Don’t worry – it’s actually useful feature. You can still use the commands “old fasion way” – by entering the full command path (for example vol create), but you can also navigate through the directory structure. I think it will be best shown on the example:

cDOT01::> volume show
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cDOT01-01 vol0         aggr0        online     RW      851.5MB    584.4MB   31%

cDOT01::>

In the first example I have entered t he full command volume show. As you can see currently my cluster cDOT01 has only one volume – vol0. Now let’s do the same by navigating through the directory structure:

cDOT01::> volume

cDOT01::volume> ?
  autosize                    Set/Display the autosize settings of the flexible volume.
  clone>                      Manage FlexClones
  copy>                       Manage volume copy operations
  create                      Create a new volume
  delete                      Delete an existing volume
  efficiency>                 Manage volume efficiency
  file>                       File related commands
  flexcache>                  Manage FlexCache
  modify                      Modify volume attributes
  mount                       Mount a volume on another volume with a junction-path
  move>                       Manage volume move operations
  offline                     Take an existing volume offline
  online                      Bring an existing volume online
  qtree>                      Manage qtrees
  quota>                      Manage Quotas, Policies, Rules and Reports
  rename                      Rename an existing volume
  restrict                    Restrict an existing volume
  show                        Display a list of volumes
  show-footprint              Display a list of volumes and their data and metadata footprints in their associated aggregate.
  show-space                  Display space usage for volume(s)
  size                        Set/Display the size of the volume.
  snapshot>                   Manage snapshots
  unmount                     Unmount a volume

cDOT01::volume> show
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cDOT01-01 vol0         aggr0        online     RW      851.5MB    584.4MB   31%

cDOT01::volume> top

cDOT01::>

Few things you should notice. First, when I entered a command volume the Data ONTAP went into volume directory (you can notice the change of the prompt from cdOT01::> to cDOT01::volume>). Next, by typing ? I have listed all available choices inside that directory. Some of the choices were another directories – you can notice them with > sign at the end, for example qtree> or quota>. Then I have executed command show, which gave me the same output as “volume show” executed from the root directory. Once I was finished with executing commands from volume directory I have moved to root directory with command top.

Question Mark

As shown in the example above, typing question mark (?) key is very useful to show the command directories and commands that are available at that command level. You can also type ? if you don’t know how to continue with the command, for example:

cDOT01::> volume create ?
    -vserver <vserver name>                                                Vserver Name
   [-volume] <volume name>                                                 Volume Name
   [-aggregate] <aggregate name>                                           Aggregate Name
  [[-size] {<integer>[KB|MB|GB|TB|PB]}]                                    Volume Size
  [ -state {online|restricted|offline|force-online|force-offline|mixed} ]  Volume State (default: online)
  [ -type {RW|DP|DC} ]                                                     Volume Type (default: RW)
  [ -policy <text> ]                                                       Export Policy
  [ -user <user name> ]                                                    User ID
  [ -group <group name> ]                                                  Group ID
  [ -security-style {unix|ntfs|mixed|unified} ]                            Security Style
  [ -unix-permissions <unix perm> ]                                        UNIX Permissions (default: ---rwxr-xr-x)
  [ -junction-path <junction path> ]                                       Junction Path
  [ -comment <text (size 0..1023)> ]                                       Comment
  { [ -max-autosize {<integer>[KB|MB|GB|TB|PB]} ]                          Maximum Autosize (for flexvols only)
    [ -autosize-increment|-i {<integer>[KB|MB|GB|TB|PB]} ]                 Autosize Increment (for flexvols only)
    [ -autosize-increment-percent|-p <percent> ]                           Autosize Increment Percent (for flexvols only)
    [ -min-autosize {<integer>[KB|MB|GB|TB|PB]} ]                          Minimum Autosize
    [ -autosize-grow-threshold-percent <percent> ]                         Autosize Grow Threshold Percentage
    [ -autosize-shrink-threshold-percent <percent> ]                       Autosize Shrink Threshold Percentage
    { [ -autosize-mode {off|grow|grow_shrink} ]                            Autosize Mode
    | [ -autosize {true|false} ] }                                         Autosize Enabled (for flexvols only)
  [ -space-guarantee|-s {none|volume|file} ]                               Space Guarantee Style (default: volume)
  [ -percent-snapshot-space <percent> ]                                    Space Reserved for Snapshots
  [ -snapshot-policy <snapshot policy> ]                                   Snapshot Policy (default: default)
  [ -language <language_code> ]                                            Language
  [ -foreground {true|false} ]                                             Foreground Process (default: true)
  [ -nvfail {on|off} ]                                                     NVFAIL Option
  [ -flexcache-origin-volume <volume name> ]                               FlexCache Origin Volume Name
  [ -enable-snapdiff {true|false} ]                                        Create Namespace Mirror Constituents For SnapDiff Use
  [ -qos-policy-group <text> ]                                             QoS Policy Group Name

cDOT01::> volume create

Question mark has printed all the options that are available for volume create command.

Tab Completion

This feature will be difficult to show on the written article, but stick with me for a second. Do you have any experience with linux bash shell? If you do, I’m sure you know what an extra feature is to press TAB key during command completion. It will complete the command or give you the possible options. Same is with NetApp c-mode (cDOT)!

cDOT01::> volume  <TAB pressed - p1>
    autosize       clone          copy           create         delete
    efficiency     file           flexcache      modify         mount
    move           offline        online         qtree          quota
    rename         restrict       show           show-footprint show-space
    size           snapshot       unmount

cDOT01::> volume show<TAB pressed - p2>
    show           show-footprint show-space

cDOT01::> volume show <TAB pressed - p3>
cDOT01::> volume show -vserver cDOT01 <TAB pressed p4>
    cDOT01    cDOT01-01
cDOT01::> volume show -vserver cDOT01 -volume
  <volume name>               Volume Name

On the above example I have used TAB key three times. Let me walk you through them:

  1. I have entered volume command and pressed TAB. That have listed all available command from volume directory.
  2. Based on my options from the first step I have chosen show and pressed TAB again. Since I haven’t put space after command show, NetApp has listed me all available commands that start with show, such as show itself, show-footprint and show-space.
  3. I decided to go with volume show command, so after the command I have pressed space and hit TAB again. NetApp has automatically completed next parameter -vserver, listed all of my available vservers (cDOT01 and cDOT01-01). If you are a little bit confused of what vserver is – do not worry, I will explain that in future post, for now just think about it as storage virtual machine.
  4. One I have chosen the vserver name clustershell has completed my command with next paramenter -volume and gave me a brief explaination what ouput is needed for that parameter.

Other features

Clustershell has some other, useful features as well, such as

Abbreviation

You can use only the shortest unambiguous sequences of characters that specify to the command you want to use, for example you can go with “volume show” command, but you can also stick with “vo show” command, since “vo” already points only to one available command family:

cDOT01::> vo show
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cDOT01-01 vol0         aggr0        online     RW      851.5MB    584.2MB   31%

The full command was given by NetApp in brackets.

Running commands outside the directories

To make life easier for 7-mode administrators (that’s my guess 🙂 ) NetApp has enabled to run some commands outside the directories. For example, within 7-mode when you wanted to do any operations on qtrees, you executed qtree <subcommand>. As you may have already noticed in above examples within NetApp cDOT command qtree is placed under volume directory, so You should go with the command: volume qtree <subcommand>. However, I guess to make life easier for us, NetApp allows us to run some commands outside the directories, for example:

cDOT01::> qtree show
This table is currently empty.

cDOT01::> qtree

cDOT01::volume qtree> show
This table is currently empty.

cDOT01::volume qtree>

My cluster currently doesn’t have any qtree, that’s why the output states that this table is empty 🙂 . As you can noticed when I executed just qtree command it took me to the right directory which is  volume > qtree.

Queries with patterns and wildcards

You can run some queries with patterns or wildcards. For example if you want to list all Your aggregates that are larger than 100MB, and then larger than 1GB:

cDOT01::> aggr show -size >100m
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr0        900MB   43.55MB   95% online       1 cDOT01-01        raid_dp,
                                                                   normal
aggr1       2.64GB    2.64GB    0% online       0 cDOT01-01        raid_dp,
                                                                   normal
2 entries were displayed.

cDOT01::> aggr show -size >1g
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr1       2.64GB    2.64GB    0% online       0 cDOT01-01        raid_dp,
                                                                   normal

History

You can also type the history command to list all your commands executed during current session, or use ‘up arrow’ key to go through commands previously executed.

 

As I mentioned before, some of those features are a little bit hard to show on the written text, since clustershell is much more dynamic that 7-mode admin shell was. In upcomming days I will create a short screencast to give you a brief introduction to the clustershell.

NetApp cDOT – Clustered Data ONTAP Command Line

I would like to continue on introduction to Clustered Data ONTAP (often referred as NetApp cDOT, or NetApp c-mode). Today I would give you a short introduction to  CLI (command line interface). There is a good chance you have some experience with 7-mode NetApp – if not – do not worry. But just as a reminder. Within 7-mode Data ONTAP you could login as an administrator. This is the primary 7-mode CLI and it’s called the admin shell. You could set different privilege levels with priv set command (such as advanced, diag, etc.). Within diag user you could access also systemshell (I have described this process in NetApp OnTAP – add ssh keys post). Keep in mind – this is quite advance topic and in normal administration tasks you would never need to access systemshell, but it’s good to be aware it is possible and how to access it.

Command prompt 7-mode

7-mode command prompt is like:

netapp01>
netapp01> uptime
  9:00pm up  9:43 0 NFS ops, 13490 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops

Where netapp01 is a hostname in our storage array. This, primary 7-mode CLI is called the admin shell. No more than two sessions are allowed (one through ssh, second through management port).

Command prompt c-mode (cDOT)

NetApp cDOT (clustered Data ONTAP) prompt is a little bit different. the Clustered Data ONTAP has three different shells for CLI commands. Do not worry, in 99% of situations you are going to use one, default clustershell. However it’s good to be aware there is also a nodeshell and systemshell available.

clustershell

netapp01::> 
netapp01::> version
NetApp Release 8.2.1 Cluster-Mode: Fri Sep 16 13:30:32 PDT 2016

This prompt with ::>  is reserved for clustershell mode. This is the default shell, and this mode you will use for all your normal administrative tasks. All comands from clustershell are executed towards the cluster itself (which points to all nodes in the cluster). This shell provides all commands you need to configure and manage the cluster.

Clustershell has three privilege levels – similar to NetApp 7-mode. Those are:

  • admin  – this id the default level
  • advanced
  • diag

If you access either advanced or diag level, you can notice change of your prompt to:

netapp01::> set -privilege advanced
Warning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.
Do you want to continue? {y|n}: y
netapp01::*>

Those additional privileges levels gives you access to some advanced commands, which will be discussed in later posts.

nodeshell

If you wish to point and execute commands to single node in the cluster you have to use the node shell. You can either enter the node shell for an interactive session or execute individual commands with system node run <nodename> <command>.  If you access interactive session the prompt will change as well:

netapp01::> system node run -node node01
Type 'exit' or 'Ctrl-D' to return to the CLI
node01> hostname
node01
node01>

Please keep in mind netapp01 is the name of the cluster, where-as node01 is a name of the node within that cluster.

To execute individual commands:

netapp01::> system node run -node node01 hostname
node01
netapp01::>

systemshell

This is a low-level shell. This shell is used only for diagnostic and troubleshooting purposes – in most cases only NetApp support is asking to access this shell. The prompt for system shell looks like:

node01%

You can only enter systemshell for a specific node, not the whole cluster. If you find it a little bit confusing, do not worry. I can guarantee you will almost always need only standard, default clustershell. This post is only to show you there are different levels you can access within NetApp cDOT.

 

NetApp – Clustered vs 7-mode Scaling

Within this post I would like to start series of articles about Clustered Data ONTAP (sometimes shorten cDOT, sometimes c-mode). I will start with explaining high level differences between Clustered-mode and 7-mode. If you are completely new to NetApp, don’t worry, this comparison will be also useful to You, because cDOT is strongly connected to 7-mode, and even if you plan to work only with new cDOT environment, it is really good to know at least basics about 7-mode.

Data ONTAP Scaling

In this section I would like to share one of the most important difference between 7-mode and c-mode. As you may already figured that out from the name itself, c-mode gives much better scaling options.

7-mode vs c-mode scaling

7-mode vs c-mode scaling

As you can see, 7-mode normally is configured in 2-node HA (High-Availability) configuration. cDOT is also configured into HA pairs, however within cDOT you can connect up to 12 pairs (in NAS configuration) or up to 4 pairs (in SAN configuration) in one cluster! (Please note this numbers are correct for Data ONTAP 8.3, it is very possible that within newer releases, those numbers will increase). Does it mean that you have to do so? Of course not! You can configure 7-mode NetApp with only one Filer (NetApp array), the same you can do using cDOT configuration. In fact you can quite often meet cDOT configuration made out-of one HA pair.

In fact that might be the main feature of Clustered Data ONTAP – you can start with 2-node Cluster, and once your environment grows you can easily non-disruptively add new nodes to your cluster. More details will come in more advance articles.

Hardware differences

Look at the title of this article. It basically says ‘c-mode vs 7-mode’ ONTAP. What is ONTAP? Well.. It is operating system. So it’s more-or-less independent from hardware layout. Does it mean you can transfer Your 7-mode NetApp to Clustered mode? In most cases the answer is yes! Not all arrays (especially older ones) are supported though. For more information about specific models you can always go to NetApp Hardware Universe. As for some details, you can choose from FAS2200, or FAS2500 series for your midsize projects, or for FAS6200/FAS8000 series for enterprise storage.

However, if you plan to connect more than two Filers into once cluster, there is one hardware difference. For Single HA configuration, there is no physical difference between 7-mode and c-dot configuration. It is sometimes called Switchless Two-Node Cluster. However if you with to configure more HA pairs, you need to apply Switched Cluster configuration. For that you require an separate Ethernet network with dedicated switch (best practice will be two switches, to make your network redundant).

Single-Node Cluster

Single-Node Cluster is the cheapest, and easiest configuration. However it has strong disadvantage: storage system is the single point of failure. If our controller fails, customers loose access to the data.

Single-Node Cluster

Single-Node Cluster

Such configuration also does not enable any non-disruptive operations. Every controller (node) reboot is equal to downtime from customer point of view.

Switchless 2-node Cluster

Such configuration is very similar to 7-mode HA configuration. The only small difference is that within c-mode configuration there is additional cluster interconnect. Cluster interconnect, as a standard Ethernet infrastructure, provides the scalability with a networking infrastructure. It is required for cluster traffic.

switchless 2Node Cluster

switchless 2Node Cluster

Such solution it is fault tolerance. Even if one node fails, second node can takeover its responsibilities. Each partner is mirroring the data for each NVRAM, so even in sudden failure of one node that is no risk of data loss.

Clustered Data ONTAP configuration

As mentioned before, clusters of more than two nodse need a supported switch to attach each node to the cluster interconnect.

Switched Cluster

Switched Cluster

Of course you cannot choose whatever switch you would like. It has to be supported cluster interconnect switch, as for example NetApp CN1610, or Cisco Nexus 5596. If you have a 2-node cluster, but you plan an expanstion in the future, it is a good approach to install Cluster Interconnect Switch for your 2-node configuration to enable future online expanstion.