How to create SRDF pairs between two EMC symmetrix VMAX storage arrays

How to create SRDF pairs between two EMC symmetrix VMAX storage arrays


This article provides step by step configuration procedure to create SRDF pairs between two EMC symmetrix storage arrays. 


1. Establish the replication link between the production site (source) and dr site (destination). The EMC symmetrix VMAX supports GigE RAs that can be used to configure FCIP between two sites. The other alterbatives are DWDM (FC) or use third party solutions such as Brocade 7800s to establish the connectivity between the sites. Click here to find the procedure for configuring FCIP using Brocade san switches.


2. Create an RDF group between two sites. To create RDF group 'testrdfg' with group id 3 and RDF directors 7G between symmetrix id 089 (prod site) and 099 (dr site), execute the command below


symrdf addgrp -label testrg -rdfg 03 -sid 089 -dir 7g -remote_rdfg 03 -remote_sid 099 -remote_dir 7g -gige 


3. Create a text file on the management server as below with a list of source devices against the destination devices. 


In the example, we will be creating an RDF pair from source device 045C on symmetrix id 089 to destination device 0285 on symmetrix id 099. So the text file will look like this


# cat pair.txt
045C         0285


045C and 0285 should have identical device characteristics. For example, if 045C is a striped meta device with 4 hypers of 60GB, then 0285 must be a striped meta of the same size. Click here for the procedure to create and expand striped meta devices. 


4. Create the pair between the source and target devices using the above text file. The command below will create pair between the devices mentioned in the text file from source array 089 using rdf group 3. rdf group 3 is already configured between Sym ID 089 and 099. So the command will pick the destination device 0285 from Sym ID 099 based on the relationship defined in rdfg 3. The commad will create the pairs in Adaptive copy mode. 



symrdf -file pair.txt -sid 089 -rdfg 3 createpair -type RDF1 -invalidate r2 -rdf_mode acp
_disk 


5. Check the status of the pair. You will see suspended status.


symrdf -file pair.txt  -rdfg 3 query -sid 089


6. Start the synchronization of the pairs. Once the below command is completed, execute command in step 5 to verify the status.


symrdf -file pair.txt  -rdfg 2 esta -sid 089


7. Switch the SRDF mode of operation from adaptive copy to Aysnc or sync, depending on the requirements. It is recommended to wait until the pairs are in sync in adaptive copy mode before switching the mode. There are also other caveats in configuring SRDF/A or SRDF/S which I will explain in another post.


symrdf -file pair.txt -rdfg 2 set mode async (or sync)


8. You could also create a disk group and add all source devices into it and then manage the pairs using the dg name. For example, the below commands create 'pair_dg' and add device 045C into it.


symdg create pair_dg -sid 089 -type RDF1
symld -g pair_dg add dev 045C -sid 089


9. Once the dg is created, use the dg name instead of the file name to query the pair status


symrdf -g pair_dg query -sid 089


Please note that you could have as many devices as you wish in the pair.txt file before issuing create device. You should also take precaution and necessary checks to make sure that the R2 devices on the destination array is not in use.


Enjoy!

How to migrate EMC Symmetrix LUNs non disruptively to another tier using symcli command symmigrate

How to migrate EMC Symmetrix LUNs non disruptively to another tier using symmigrate command

 

EMC Symmetrix Virtual LUN technology enables transparent, non disruptive data migration among storage tiers within the array and between RAID protection schemes without impacting local or remote replication. You can migrate data to unconfigured space or configured space.

Solutions Enabler 7.0 introduced a new symcli command symmigrate. Follwoing example use symmigrate command to non disruptively migrate an EMC LUN to another tier.

 

In this example, EMC symmetrix device 105e is a 30GB concatenated meta device in tier 1. This device needs to be migrated to tier 3. Given below are the step by step tasks to perform this migration

 

Create a concatenated meta device in the target tier (tier 3) of 30GB. Click here to find out the procedure for creating concatenated meta device. For this example we will use 27F7 which is a concatenated meta device in Tier 3 disk group.

 

Create a device file with source and destination devices. In this example I have created ‘migdevice.txt’. The content of the file is listed below. It is not necessary to create a text file, the symmigrate command can be used to migrate all devices in a device group or storage group.

# cat migdevice.txt

105e 27F7

 

Before performing the migration, run symmigrate command in validate mode to make sure that a migration can be completed with the source and target devices mentioned in the configuration file. You can name the migration session so that it is easy to identify and manage later.

#symmigrate -name testmig -sid 8899 -f migdevice.txt validate –v

 

Once the validate operation is completed successfully, the migration session can be established

#symmigrate -name testmig -sid 8899 -f migdevice.txt establish -v

 

Check the progress of the migration using the below command. Wait until the the status changes to “migrated”

#symmigrate -name testmig -sid 8899 query -detail -i 10

 

List the devices and make sure that the storage tier has changed

#symdev show 105E

 

This device will be showing the original disk group of 27F7

#symdev show 27F7

 

This device will be showing the disk group of the original 105E. Basically they would have swapped the disk groups.

 

Once the migration is completed, the session can be removed or terminated. The terminate operation removes the migration session named “testmig”

#symmigrate -name testmig -sid 8899 verify -migrated

#symmigrate -name testmig -sid 8899 terminate

 

All done!

How to replace a failed root disk using Solaris Disk Manager

How to replace a failed root disk using Solaris Disk Manager

 

In this example, c0t0d0 and c0t1d0 are the root disks and they are mirrored using Solaris disk manager. The disk c0t0d0 failed and needs replacing. The failed disk c0t0d0 holds meta mirror d10,d11,d13,d14,d15 and d16.

 

Remove the mirrors from the failed disk

 

metadetach -f d0 d10

metadetach -f d1 d11

metadetach -f d3 d13

metadetach -f d4 d14

metadetach -f d5 d15

metadetach -f d6 d16

 

Clear the meta devices

 

metaclear d10

metaclear d11

metaclear d13

metaclear d14

metaclear d15

metaclear d16

metadb –i

 

Remove the failed disk from the meta db

 

metadb -d c0t0d0s7

metadb –i

 

Remove the disk from OS

 

cfgadm -c unconfigure c0::dsk/c0t0d0

 

Replace the failed disk, possibly by the vendor

 

Configure the new disk in the OS and create new devices

 

cfgadm -c configure c0::dsk/c0t0d0

devfsadm –C

 

Create the partitions same as the existing root disk c0t1d0

 

prtvtoc /dev/rdsk/c0t1d0s2|fmthard -s - /dev/rdsk/c0t0d0s2

 

Install the boot block and make it ready for OS

 

installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t0d0s2

metadb -a -c 3 c0t0d0s7

metadb -i

Create the new meta devices on the partitions of replaced disk

 

metainit -f d10 1 1 c0t0d0s0

metainit -f d11 1 1 c0t0d0s1

metainit -f d13 1 1 c0t0d0s3

metainit -f d14 1 1 c0t0d0s4

metainit -f d15 1 1 c0t0d0s5

metainit -f d16 1 1 c0t0d0s6

 

Re establish the mirror

 

metattach d0 d10

metattach d1 d11

metattach d3 d13

metattach d4 d14

metattach d5 d15

metattach d6 d16

 

How to commission a server and provision storage on an EMC Symmetrix VMAX array with the autoprovisioning groups

How to create commission a server and provision storage on an EMC  Symmetrix VMAX array with the autoprovisioning groups

With EMC Symmetrix VMAX and Enginuity 5874, autoprovisioning replaces Volume Logix as the way to mask storage. This document explains step by step guide to create and provision storage using autoprovisioning groups.

There are four steps involved in provisioning storage.

1.     Create an initiator group – Intiator group contains the world wide port name of the HBA. Initiator group can be cascaded, that is an initiator group can be created by adding two initiator groups, rather than WWPNs. In this example we have WWPNs 10000000caabab1a and 10000000caabab1b

It is a best practice to create the initiator group with “consistent_lun” option

symaccess create –name server1_ig –type initiator –wwn 10000000caabab1a –consistent_lun –sid XXX

Add the second WWPN to the initiator group

symaccess –name server1_ig –type initiator add –wwn 10000000caabab1b –sid XXX

2.       Create a storage group – storage group contains the logical volumes (hypers and metas). Here we will be using 1001 and 1002 as the logical volumes

symaccess create –name server1_sg –type storage

symaccess –name server1_sg –type storage add devs 1001:1002 –sid XXX

3.       Create a port group – Port group contains the front end ports to which the HBAs are zoned in. For the purposes of this excercise, we assume that the HBAs are zoned in to FEs 6E0 and 7E0.

symaccess create –name portgroup_6e0_7e0 –type port –dirport 6E:0, 7E:0 –sid XXX

4.       Create a masking view, binding the initiator group, storage group and port group

       symaccess create view –name server1_mv –ig server1_ig –sg server1_sg –pg portgroup_6e0_7e0

6.       Verify that the storage is provisioned correctly

symaccess show view server1_mv –sid XXX

How to configure a Brocade switch as an Access Gateway – step by step

How to configure  a Brocade switch as an Access Gateway – step by step

Configuring a Brocade switch as an access gateway  is a two step process, first enable the NPIV capabilities on the uplink switch ports by executing steps 1 to 4 and then configure Brocade switch in Access Gateway mode.

In the example below, the we have two switches – “core-sw” which is the core switch where the access gateway will be hooked up and “edge-sw” which is actually being reconfigured as access gateway.

1. Login to the uplink switch where the Access Gateway will be connected.

2. Identify the ports for uplink and verify whether the ports are NPIV capable. “NPIV capability” must be set to “ON” for the identified ports. If not proceed to step 3 otherwise step 4.

core-sw# portcfgshow 

3. Enable NPIV capability for the identified ports. For example port 7 and 15

core-sw#  portcfgnpivport 7  1

core-sw#  portcfgnpivport 15 1

core-sw#  portcfgshow

4. Disable these ports temporarily

core-sw#  portdisable 7

core-sw#  portdisable 15

5. Login to the Brocade switch that need to be configured as AG

6. Disable the switch

edge-sw#  switchdisable

7. Clear any configuration that may exist in the switch

edge-sw#   cfgclear; cfgsave

8. Check the firmware version on the switch. If it is lower than 5.3.1a, upgrade before proceeding

edge-sw#   firmwareshow

9. Enable Access Gateway Mode on the switch. The switch will reboot automatically and come back in access gateway mode.

edge-sw#   ag --modeenable

10. Verify the mode of the switch. This will display the configured switch mode, ie, Access gateway mode enabled or disabled.

c-class-gw#   ag --modeshow

c-class-gw#    switchshow

11. Name the Access Gateway. Follow the standard naming convention. Eg: aga00, agb00

c-class-gw#   switchname aga00

12. List the current F port to NPIV mapping. This is required to delete and remap the ports as per BL standards.

c-class-gw#   ag --mapshow  # list the current mapping

13. Delete the current mapping by executing ag --mapdel port each NPIV port -0 and 17 – 23. Replace the port numbers below using the output from the command above.

c-class-gw#    ag --mapdel 0 “1;3”

c-class-gw#    ag --mapdel 17 “2;4”

c-class-gw#    ag --mapdel 18 “5;6” # repeat this for all NPIV ports

14. Verify the mapping list is empty.

c-class-gw#    ag --mapshow

15. Remap as per BL standards. All odd numbered ports to one NPIV port and the even numbered to another

c-class-gw#    ag --mapset 17 “1;3;5;7;9;11;13;15”

c-class-gw#    ag --mapset 18 “2;4;6;8;10;12;14;16”

16. Verify the mapping

c-class-gw#    ag --mapshow

17. Attach port 17 and 18 to the uplink switch, if not done already
18. Enable the ports on the uplink switch, from step 4

core-sw#   portenable 7

core-sw#   portenable 15

19. Verify the AGs are logged in to the fabric. The port will come up as NPIV if there are AGs have configured devices, otherwise the WWN of the port will be displayed.

core-sw#   switchshow

20. To view the WWN of the server switch, execute portshow on the NPIV devices. The WWN of the server switch will come up in the NS as well.

core-sw#   portshow 7

core-sw#  portshow 15

21. Proceed with the zoning operation as per standards

22. Access Gateway configuration complete

How to configure FCIP tunnel between two Brocade 7800 using CLI

How to configure FCIP tunnel between two Brocade 7800 using CLI

 

In the example below, we assume that the TCP/IP network link between site A and site B is already configured and ready to go. The Brocade 7800 comes with multiple (2 or 6) GigE ports, depending on the configuration you purchase. Also, you could use optical connector or RJ45 copper connector (maximum 2) for connecting the Brocade 7800 to the TCP/IP network. Check the hardware reference manual for more details on physical ports and connectivity details.

 

The end result of this exercise will be a single fabric (merged fabric) across site A and site B consisting of 2 Brocade 7800 fibre switches. If you do not want the fabrics to merge, ie maintain two distinct fabrics, but give selective access to devices in the second fabric, then what you need is FC-FC routing. FC-FC routing will be published in the future.

 

In this example, we will be using port 16 (ge0) as the VE port and the physical port on each side. You will need to get the IP address and the net mask details from your network team, for each site. In this example we use

Site A IP address: 192.168.10.14/255.255.255.0

Site B IP address:192.168.10.67/255.255.255.0

 

1.  Login to the Brocade 7800 switch in site A (SiteA_7800)

 

2.  The first step is to disable the VE ports on the Brocade 7800s. This is to control the merging of the fabric, will enable them when the configuration is completed.

 

SiteA_7800> portdisable 16

 

3.  Assign IP address and subnet mask to port ge0

 

SiteA_7800> portcfg ipif ge0 create 192.168.10.14 255.255.255.0 1500

 

4.  Configure the tunnel to site B from site A with a committed rate of 100Mbit/sec. Note the order of the Ips below (site B first)

 

SiteA_7800>portcfg fciptunnel 16 create 192.168.10.67 192.168.10.14 100000

 

5.  Login to the Brocade 7800 switch at site B (SiteB_7800)

 

6.  Disable the VE port 16 on the Brocade 7800

 

SiteB_7800>portdisable 16

 

7.  Configure the IP address for the ge0 at site B

 

portcfg ipif ge0 create 192.168.10.67 255.255.255.0 1500

 

8.  At this stage, you will be able to ping the IP address at Site A, assuming the network link is configured and network cables are plugged in at both sites.

 

SiteB_7800>portcmd –ping ge0 –s 192.168.10.67 –d 192.168.10.14

 

If you see reply from SiteA, your network link is configured and working well, if not, back to the networks team.

 

9.  At SiteB, configure the tunnel for VE port 16 to connect to the Brocade 7800 switch at siteA

 

portcfg fciptunnel 16 create 192.168.10.14 192.168.10.67 100000

 

10.  Now we are ready to enable the VE ports, the fabrics will merge after executing the below commands. At this point, the rules to connect a new switch to an existing fabric applies. For example, if both switches have defined zone configurations that are different, zone conflict error will occur

 

SiteB>portenable 16

SiteA>portenable 16

 

11.  Confirm that the VE port 16 is online and connected to the switch at siteB

 

SiteA>switchshow|grep VE

 

12.  Verify fabric availability

 

SiteA>fabricshow

 

13.  Check the FCIP configuration parameters

 

SiteA>portshow fcipcircuit all

 

At this point, the FCIP configuration is complete. You may choose to configure additional GigE ports or additional circuits as per the design requirements

 

How to expand striped meta device on EMC symmetrix VMAX using symcli

How to expand striped meta device on EMC symmetrix VMAX using symcli

Expanding a concatenated meta device on an EMC Symmetrix VMAX using symcli is a straight forward procedure. Click here for the procedure. However, the expansion of striped meta is rather complicated. The procedure includes creating a new meta device of equal size and construct (same number of LUNs, hyper size) and using that as a BCV device as a temporary staging area.

 

A RAID-5 BCV device (BCV+R-5) can be used as a staging area. If you are running a really old version of Enginuity version, then RAID-5 BCV may not be supported, in this case use way mirror BCV.

 

Step by step striped meta expansion procedure using symcli

 

The device to expand – 0065 is 180GB and is striped across 3*60GB hypers

 

 1. Create a new striped meta device of the same capacity as the original meta device (180GB) using 3*60GB hypers - 017d,017e and 017f. This will be used as a temporary staging area for expansion

 

symconfigure -sid <xxx> -cmd “form meta from device 017d, config=striped, stripe_size=1 cyl; add dev 017e:017f to meta 017d;” prep

 

Once the prepare command completes successfully, commit the changes

 

symconfigure -sid <xxx> -cmd “form meta from device 017d, config=striped, stripe_size=1 cyl; add dev 017e:017f to meta 017d;” comm

 

 2.  Set the attribute of the new device to BCV RAID-5

 

symconfigure -sid <xxx> -cmd “convert dev 017d to BCV+R-5;” prep

 

Once the prepare command completes without errors, commit changes

 

symconfigure -sid <xxx> -cmd “convert dev 017d to BCV+R-5;” comm

 

 3.  Expand the meta device. The expansion of striped meta requires “n” number of hypers, where “n” is the number of columns in the stripe. For example, if you are expanding a striped meta which has 5 members (striped across 5 hypers), then you will need 5 new hypers. The device we are using in this example, 0065, is currently striped across 3 devices, therefore we need 3 new hypers 0187,0188 and 0189.
 

symconfigure -sid <xxx> -cmd “add dev 0187:0189 to meta 0065,protect_data=TRUE,bcv_meta_head=017d” prep

 

Once the prepare command completes without errors, commit changes

 

symconfigure -sid <xxx> -cmd “add dev 0187:0189 to meta 0065,protect_data=TRUE,bcv_meta_head=017d” comm

 

Wait for the process to complete and that’s it, all done!

 

How to expand a concatenated meta device on EMC Symmetrix VMAX using symcli

A meta device is a Symmetrix mechanism for creating a device larger than the current maximum hyper volume size (262668 cylinders or 240GB on EMC Symmetrix VMAX). There are two types of meta devices – concatenated or striped.

For virtual provisioned environment, concatenated meta volumes are recommended by EMC, striped meta is not recommended.

Given below are the steps to create a concatenated meta device and expand the meta device on an EMC symmetrix VMAX system using symcli

In this example, we are using hypers  0010, 0011 and 0012 for creating the meta and 0013 to expand the already available meta.

 To form the meta for the first time, the meta head (in this case 0010) should not be mapped to any front end ports. If it is mapped, unmap the hyper before proceeding. Also, the meta tails (0011 and 0012) should not be attached to any FAs (unmapped) and the data will be lost.

symconfigure –cmd “form meta from device 0010, config=concatenated; add dev 0011:0012 to meta 0010;” prep

Once the above command completes successfully, commit the changes

symconfigure –cmd “form meta from device 0010, config=concatenated; add dev 0011:0012 to meta 0010;” comm

To add hyper 0013 to already formed meta device 0010 (meta head), execute the following command

symconfigure –cmd “ add dev 0013 to meta 0010;” prep

Once the above command completes successfully, commit the changes

symconfigure –cmd “ add dev 0013 to meta 0010;” comm

Expanding striped meta device is bit more complicated, click here for procedure