Month: November 2013

NetApp OnTAP – add ssh keys

As we all know there is basically a limit of one ssh session to the filer. If you connect to your filer thru some unix/linux host it is a good practice to send each command as separate ssh command instead of constant ssh session. What do I mean by that? Instead of:

tomek@debian:~$ ssh -l root netapp01
root@netapp01’s password:

netapp01> df -Ag
Aggregate                total       used      avail capacity
aggr_root                  0GB        0GB        0GB      95%
aggr_root/.snapshot        0GB        0GB        0GB     —%
aggr0                     10GB        1GB        9GB      10%
aggr0/.snapshot            0GB        0GB        0GB     —%
netapp01> uptime
2:52pm up 18:31 47 NFS ops, 0 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops
netapp01> Connection to netapp01 closed by remote host.
Connection to netapp01 closed.

Let’s go with:

tomek@debian:~$ ssh netapp01 df -Ag
Aggregate                total       used      avail capacity
aggr_root                  0GB        0GB        0GB      95%
aggr_root/.snapshot        0GB        0GB        0GB     —%
aggr0                     10GB        1GB        9GB      10%
aggr0/.snapshot            0GB        0GB        0GB     —%

tomek@debian:~$ ssh netapp01 uptime
  2:52pm up 18:31 47 NFS ops, 0 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops

You see the difference? Yep – first of all filer doesn’t ask about the password each time. Of course it has few  other adventages. First of all you can use all the nice features of bash (if you use /bin/bash in you linux/unix). So, for example you can go with:
  
tomek@debian:~$ ssh netapp01 df -Am |  grep -v snap
 Aggregate                total       used      avail capacity
aggr_root                900MB      856MB       43MB      95%
aggr0                  10800MB     1030MB     9769MB      10%



The other advantage of that way of working is that you don’t block the one available ssh session to the filer, especially useful in multi-admin environment.

So.. How to do it? I would go with couple of steps:
  

1. Create an user account in Data ONTAP


If your unix account is, for example, tomek, create same username in Data ONTAP. When making the ssh sessions you will just have to go with ssh filername   instead of ssh username@filername  or ssh -l username filername. To do so:


netapp01> useradmin user add tomek -g Administrators
New password:
Retype new password:
User <tomek> added.

2. Generate ssh keys 

  I won’t go deep into ssh keys. If you don’t understand what is a public key, what is a private key I would strongly recommend google it. But in simple steps you can generate your ssh keys like that:

tomek@debian:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/tomek/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/tomek/.ssh/id_rsa.
Your public key has been saved in /home/tomek/.ssh/id_rsa.pub.
The key fingerprint is:
fa:54:70:bb:61:4c:fd:3e:08:68:74:0f:94:95:51:ba tomek@debian

tomek@debian:~$ cat .ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ1Acxv0WCAA0+NfKoOrFSVQL9jpc2hZrMbw5kDSEK8mZXH8pZAm/0XV9ncBoZjy35+Ih4qYDqS6aJyRvocynLgPQiYAdw9jMJvDsWR5OF5RWGILpHS+UDiw4hVMhGLNj2DmA3v/Jkl9vGfBObd7uhMyEn1CzG2D0xYLLPTb8JjsoBi4dsBXBd1w57kM5++gcPtT8DNu04JQkRZ7yEvgnBCr8YjSZ1wNSaV4HwrDyBzLz4VD87roAxKUemLF7y1L5e6dzikdMxl4xxTT3Gxwrh5XLdt7VR2sxpBT3c16Uiy8WQ+zYrIeH6Ih6BZLZgHhHau8JbYtBxe4MQb9O3jiFx tomek@debian

I recommend to use passphrase for a security reasons. If you won’t use passphrase, other user can copy (if, for example has an access to the root account) your private key and use it to login to filers as you.


3. Send the ssh key to the filer

Ssh public key should go to /etc/sshd/<user_name>/.ssh/authorized_keys. If you just created a new user this path does not exist. To create a path it gets a little bit tricky since mkdir is not a command you can use in normal operation ONTAP mode. To add this file you can always use cifs or nfs share to your /vol/vol0. If you don’t have such you need to use diaguser and go to systemshell. To do so:



– login to the filer and enter the advanced mode

 

tomek@debian:~$ ssh netapp01
tomek@netapp01’s password:

netapp01> priv set advanced

– enable diaguser and set password for diaguser



netapp01*> useradmin diaguser unlock

netapp01*> useradmin diaguser password

Please enter a new password:
Please enter it again:

– go to systemshell using username:diag, password that you set up in previous step

netapp01*> systemshell

Data ONTAP/amd64 (netapp01) (ttyp0)

login: diag
Password:

Warning:  The system shell provides access to low-level
diagnostic tools that can cause irreparable damage to
the system if not used properly.  Use this environment
only when directed to do so by support personnel.

netapp01%

– create the directory and user VI to add a public key to the authorized_keys file



netapp01% mkdir -p /mroot/etc/sshd/tomek/.ssh
netapp01% vi /mroot/etc/sshd/tomek/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ1Acxv0WCAA0+NfKoOrFSVQL9jpc2hZrMbw5kDSEK8mZXH8pZAm/0XV9ncBoZjy35+Ih4qYDqS6aJyRvocynLgPQiYAdw9jMJvDsWR5OF5RWGILpHS+UDiw4hVMhGLNj2DmA3v/Jkl9vGfBObd7uhMyEn1CzG2D0xYLLPTb8JjsoBi4dsBXBd1w57kM5++gcPtT8DNu04JQkRZ7yEvgnBCr8YjSZ1wNSaV4HwrDyBzLz4VD87roAxKUemLF7y1L5e6dzikdMxl4xxTT3Gxwrh5XLdt7VR2sxpBT3c16Uiy8WQ+zYrIeH6Ih6BZLZgHhHau8JbYtBxe4MQb9O3jiFx tomek@debian
~

/mroot/etc/sshd/tomek/.ssh/authorized_keys: 1 lines, 394 characters.

– exit the systemshell and for security reasons lock the diag user, exit the advanced mode


netapp01% exit
logout
netapp01*> useradmin diaguser lock

netapp01*> priv set
netapp01> Connection to netapp01 closed by remote host.
Connection to netapp01 closed.

That’s t! Enjoy your password-less access to the filer! 



tomek@debian:~$ ssh netapp01 df -m
Filesystem               total       used      avail capacity  Mounted on
/vol/vol0/               808MB      187MB      621MB      23%  /vol/vol0/.snapshot       42MB       99MB        0MB     234%  /vol/nfs/                972MB        0MB      972MB       0%  /vol/nfs/.snapshot        51MB        0MB       50MB       1% 

Domain join and ‘Sasl protocol violation’

Recently I had an issue with EMC VNX. I have a cifs_server on Data Mover and when I tried to add the NetBIOS Name to the Domain I was facing the issue:

Brief Description:  DomainJoin::connect:: Unable to connect to the LDAP service on Domain Controller ‘domain_controller.mydomain.net’ (@IP) for compname ‘compname’. Result code is ‘Sasl protocol violation’. Error message is Sasl protocol violation.
Full Description:  DomainJoin::connect:: Unable to connect to the LDAP service on Domain Controller ‘domain_controller.mydomain.nett’ (@IP) for compname ‘compname’. Result code is ‘Sasl protocol violation’. Error message is Sasl protocol violation.
Recommended Action:  Refer to your Customer Service Knowledgebase. Contact your Customer Service.
Message ID:  13157007706 

That was a message shown in EMC Unisphere. My first idea was: go to CLI i try from command line

[nasadmin@VNX ~]$ server_cifs vdm-name -Join compname=vnxname,domain=mydomain.net,admin=useradmin
vdm-name : Enter Password:************

Error 13157007706: vdm-name : DomainJoin::connect:: Unable to connect to the LDAP service on Domain Controller ‘dc.mydomain.net’ (@IP) for compname ‘vnxname’.
Result code is ‘Sasl protocol violation’. Error message is Sasl protocol violation.

OK, let’s try the logs:

[nasadmin@VNX ~]$ server_log vdm-name
2013-11-07 19:26:35: KERBEROS: 4:[vdm-name] WARNING: no response from KDC ip1
2013-11-07 19:26:40: KERBEROS: 4:[vdm-name] WARNING: no response from KDC ip1
2013-11-07 19:26:45: KERBEROS: 4:[vdm-name] WARNING: no response from KDC ip3
2013-11-07 19:26:50: KERBEROS: 4:[vdm-name] WARNING: no response from KDC ip4
2013-11-07 19:26:55: KERBEROS: 4:[vdm-name] WARNING: no response from KDC ip5
2013-11-07 19:26:55: LDAP: 3:[vdm-name] LDAP authentication: GSS initate security context for target: ldap/dc.mydomain.net@mydomain.net – principal: useradmin@mydomain.net failed                    – GSS-API major error: Miscellaneous failure
2013-11-07 19:26:55: LDAP: 3:[vdm-name] LDAP authentication: GSS initate security context for target: ldap/dc.mydomain.net@mydomain.net – principal: useradmin@mydomain.net failed                    – GSS-API minor error: Cannot contact any KDC for requested realm
2013-11-07 19:26:55: LDAP: 3:[vdm-name] LdapClient::connect: error message: Sasl protocol violation, (error code 99)
2013-11-07 19:26:55: SMB: 3:[vdm-name] DomainJoin::connect:: Unable to connect to the LDAP service on Domain Controller ‘dc.mydomain.net’ (@ip1) for compname ‘vnxname’. Result code is ‘Sasl protocol violation’. Error message is Sasl protocol violation.
2013-11-07 19:26:55: SMB: 3:[vdm-name] DomainJoin compname=vnxname domain=mydomain.net DC=dc.mydomain.net IP=ip1 failed
2013-11-07 19:26:55: ADMIN: 3:[vdm-name] Command failed:  :2 domjoin compname=vnxname domain=mydomain.net admin=useradmin password=************************ init

No idea. I tried google it, I checked the timezones, all domain controllers were pingable from the data_mover.. And what was the issue?

The issue was the MTU of the network interface. For some reason MTU=1500 caused the issue, and when I changed it to MTU=900 I was again able to add and/or delete the cifs_server from the domain.

Data protection – NetApp way

When I say data protection I mean the features to back up data and to be able to recover it when needed. Basically you need to back up data for the following reasons:

  • to protect data from accidentally deleted files, application crashers, viruses, data corruption etc.
  • to archive data for future use or for legal purposes
  • to recover from a distaster

NetApp developed many methods of protecting data. To use some of them you need an extra licence, some of them are the standards features of Data ONTAP.

 aggr copy

aggr copy gives up fast block copy of data stored in aggregates. Just a quick remain, all data served by NetApp are located on the aggr. With the aggr copy you can make an exact copy of existing aggregate. It means that all volumes and qtrees that are on the source aggregate will be copied as well.
You can use aggr copy to copy the aggregate within the same filer or to another filer. If the destination is on another filer make sure that rsh authentication is enabled on the source and destination.
The basic example:

filerB> aggr restrict aggr_dest
filerB> aggr copy start filerA:aggr_source filerB:aggr_dest

snapshot copy

NetApp allows you to manually or automatically create and maintain many snapshot copies. Snapshot itself doesn’t copy the data when created, but copies the data that changes between the snapshot and the current state. It means that if you have a snapshot made yesterday at 12:00 you can at any time recover files or even the whole snapshot image to the point of yesterday 12:00.
The basic example:

filerA> snap create volume_01 snapshot_0001

With the snapshot ans SnapRestore (extra license is needed) you can easily recover single file or the whole volume from snapshot.

SnapMirror

With the snapmirror you can replicate the whole volume or the selected qtree to other location (extra license is needed) . You can set SnapMirror in three modes: sync, a-sync and semi-sync. More about SnapMirror you can find in this post.

SnapVault

SnapVault is the backup feature that requires and extra license.  Within the SnapVault you can back up the entire qtree, set up different snapshot schedule on the destination. More about SnapMirror vs SnapVault you can find in this post.

vol copy

With the vol copy you can copy all data from one volume to another, either on the same or different system. Similar to aggr copy, you can initiate a volume copy with the vol copy start command. Teh result is a restricted volume containing the same data as the source volume at the time you initiated the copy opreation.

filerA> vol create vol1 aggr1 50g
filerB> vol create vol1_copy aggr1 50g
filerB> vol restrict vol1_copy
filerB> vol copy start filerA:vol1 filerB:vol1_copy
 […]
filerA> vol status -b 
Volume     Block Size   Vol Size  FS Size 
 ——      ——        ——      ——
 vol1           4096             4346752            4346752
filerB> vol status -b
Volume     Block Size   Vol Size  FS Size 
——      ——        ——      ——
vol1_copy     4096             4346752            4346752 

filerB> vol online vol1_copy

Of course that’s just a simple example.

SyncMirror

Continous mirroring of data to two separate aggregates. This features allows for real-time mirroring of data to matching aggregates physically connected to the same storage system.