Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by zco.. » Sun, 26 Jun 2005 22:24:06



Short Story:
Setup several servers (Enterprise 4500s and SunFire V880s) with Solaris
9 & Veritas Volume Manager 4.0. Have Sun T3B arrays attached in partner
pair configuration.

Shortly (several hours on servers with minimal IO) after reboot of the
Solaris server the T3 array controllers start failing over from one
controller to the other. Controller failovers are occurring several
times a minute and IO performance to the array drops dramatically with
large IO request taking 5 to 10 times longer.

When the Server is shutdown controller failover stops - When the
server is rebooted the problem reoccurs.

Gory Details:

Problem is occurring on all servers with following software
configuration
Clean install of Solaris 9 using 4/04 CDROMs
Recommended and Security patch cluster from April 2005 (Generic
_118558-05)
Installation of Veritas Volume Manager 4.0
Installation of patches for Volume Manager
115209-12
115212-13
115210-12
115217-03
117094-02
117276-01

Problem is seen on different Hardware platforms:
SunFire V880 with 1GB FCAL PCI cards
Sun Enterprise 4500 with 1GB FCAL SBUS cards

Problem Occurs on T3Bs of different firmware versions:
Firmware: 2.01.06
Firmware: 3.1.6

The same hardware functions properly with Solaris 9 with VxVM 3.5.

Using VxVM vxdmpadm I see the following issues.

Quote:>From command: vxdmpadm listapm all

Module name dmpapf for array type "A/PF-T3PLUS" is in
"Not-Active" state.

Quote:>From command: vxdmpadm listenclosure all

Enclosure type "T3" has an associated array type of "A/P-C"

Quote:>From command: "vxdmpadm getattr enclosure T30 iopolicy"

     (T30 is the enclosure name assigned to the first array of type
"T3")
The default and current IO policy is listed as "Round-Robin"

The vxdmpadm commands seem to identify the issue - That the T3 arrays
are not being associated with the proper array type. The incorrect
array type uses an iopolicy of round-robin (rather than Single-Active)
which causes the server to send data alternately down each of the data
paths - with the T3 array recognizes as an indicator to perform a
controller failover.

Have tried setting the IO police to "Single-Active" and loading the
dmpapf module but there was no change in the controller failovers on
the T3. Settings are returned previous values after a "vxdctl
enable" or system reboot.

Have tried building the servers with the T3 attached and adding them
after Volume Manager had been installed.

Have installed Veritas Volume Manager from several different CDROM sets
- including one from a shrink-wrapped box dated May 9th 2005.

Any suggestions or assistance would be appreciated.

 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by tunl » Sun, 26 Jun 2005 23:08:02



> Short Story:
> Setup several servers (Enterprise 4500s and SunFire V880s) with Solaris
> 9 & Veritas Volume Manager 4.0. Have Sun T3B arrays attached in partner
> pair configuration.

> Shortly (several hours on servers with minimal IO) after reboot of the
> Solaris server the T3 array controllers start failing over from one
> controller to the other. Controller failovers are occurring several
> times a minute and IO performance to the array drops dramatically with
> large IO request taking 5 to 10 times longer.

> When the Server is shutdown controller failover stops - When the
> server is rebooted the problem reoccurs.

> Any suggestions or assistance would be appreciated.

 Are you having  a conflict  between  "Traffic Manager"  and DMP ?
 You do not say if you have  a vertias supplied  "array support
library"
 ASL  installed for the T3 - if one exists at all.

 I know that a customer of mine had to install ASL's  when migrating to

 VxVM 4.0 however  they were using EMC Clariions.

 I believe that you should use EITHER Trafficmanager or DMP.
 If useing Traffic manager switch off DMP  from vxdiskadm.
 If useing DMP you probably need the ASL for the T3.

  //Lars

 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by zco.. » Sun, 26 Jun 2005 23:55:45


DMP with the T3 Arrays is supported in the versions of Veritas Volume
Manager shipped by Sun.

I have not enabled traffic manager for any of the multipathed disks or
arrays.

 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by Scott Howar » Mon, 27 Jun 2005 00:05:29



> Setup several servers (Enterprise 4500s and SunFire V880s) with Solaris
> 9 & Veritas Volume Manager 4.0. Have Sun T3B arrays attached in partner
> pair configuration.

> Shortly (several hours on servers with minimal IO) after reboot of the
> Solaris server the T3 array controllers start failing over from one
> controller to the other. Controller failovers are occurring several

What do you have mp_support on the T3 set to?  You can check by logging
into the array and running "sys list".

For VxVM DMP it needs to be set to "rw".

  Scott

 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by zco.. » Mon, 27 Jun 2005 06:46:24


mp_support is set to rw.

These same servers and arrays functioned correctly when Solaris 9 and
Veritas Voulume Manger 3.5 was being used.

Now we are deploying the servers with Solaris 9 and Veritas Volume
Manager 4.0. Am  working with Sun support and have checked most
settings. At this point I am hoping someone has seen and already solved
this problem.

 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by Rodrick Brow » Tue, 28 Jun 2005 00:57:30



Quote:> mp_support is set to rw.

> These same servers and arrays functioned correctly when Solaris 9 and
> Veritas Voulume Manger 3.5 was being used.

> Now we are deploying the servers with Solaris 9 and Veritas Volume
> Manager 4.0. Am  working with Sun support and have checked most
> settings. At this point I am hoping someone has seen and already solved
> this problem.

Disable DMP and just use MPxIO aka Sun Traffic Manager.
 
 
 

Issue with DMP and T3 Arrays (Sol 9 & VxVM 4.0)

Post by zco.. » Tue, 28 Jun 2005 04:06:01


If a solution for DMP is not found soon we will probably switch to
Traffic Manager. But at this point I woulkd perfer DMP so that our
deployment  of Volume Manager and  T3 arrays is consistent across all
servers.
 
 
 

1. Combining A3500 and T3 disk arrays into a single vxvm volume

I'm looking for guidance on combining an A3500 disk array with a T3 disk
array into a
single vxvm volume.

This is part of an Oracle 11i upgrade, and we've been asked to create a
single volume mount
point.

Solaris 8
E5500
The A3500 has 5 trays with a mixture of 9g and 18g drives, approximately
650g usable in 4
LUNs, raid5.  We've made the LUNs approximately the same size, with a hot
spare disk in
each.

The T3 work group array is a sinble raid5 LUN with 477g usable.  The T3(s)
are fiber
attached via a SAN.

The A3500 has 2 controllers, each with 64mb cache; the T3 has 1gb cache.

So the resulting volume will have 5 disks available.  Should we make it
striped, and if so,
how many columns?  Should it be concatenated?

In the very near future, we'll be adding 2 more T3s to this volume.  At that
time, should
we unload and reload so that the data is spread evenly?

Thanks.

2. kbiff docking - nope!

3. vxvm & dmp?

4. Global clipboard for X?

5. t3 array issue

6. nntp-server

7. Veritas VXVM 4.0 installation issues

8. Blame General Electric for BlackOut says FirstEnergy

9. VxVM 2.5: Adding disks: DMP question

10. 6120 (T4) with Solaris 8 and VxVM 3.2: DMP not working

11. Storage Works RA8000 and VxVM DMP

12. T3 partner group delivery - do they come already mirrored + VxVM?

13. Sun T3, Solaris MPxIO and Veritas VxVM