Very Computer

I'm trying to connect StorageEdge T3 to Sun servers using FC switch
and fiber topology.

If I configure FC switch for arbitraded loop topology, everything
works fine on most of the servers.  I can see T3 disks mapped under
standard /dev/dsk/cNtNdNsN devices.  Except on one server that simply
refuses to see the disks -- I tried to replace everything, from cables
and FC card in server to using different port on switch, it simply
don't see T3 disks.

If I configure FC switch for fiber topology than following happens:

On the switch I can see that T3 is connected to an FL port, while Sun
servers are connected to F ports on switch (T3 have old Qlogic 2100
chipset that supports only loop, and Sun FC cards have Qlogic 2200
chipset that supports both loop and fiber, so this is OK).

Now, the problem is that after reboot, Sun servers sometimes see T3
storage, and sometimes after reboot it simple dissapers (and reapears
after some number of reboots).  Also, when they see it, disks are
mapped under strange device names, something of the form cNtWNNdNsN
(where WNN is WNN of T3 FC controller).

I also tried to access T3 from Linux box.  It sees disks under both
arbitrated loop and fiber topology.

Did anybody had this kind of problems with T3 and/or Sun FC cards?
Any advices?

FC switch (well, switches to be more precise, T3 is connected to one,
and servers to another) I use is Compaq FC switch.

BTW, I also have Sun SANbox FC switch (firmware in it supports only
arbitrated loop topology for now) that I tried to connect with Compaq
FC switch -- and it simply was not working.  Any hope for connecting
them?

--

Opinions expressed herein are my own.
Statements included here may be fiction rather than truth.

:%I'm trying to connect StorageEdge T3 to Sun servers using FC switch
:%and fiber topology.

:%If I configure FC switch for arbitraded loop topology, everything
:%works fine on most of the servers.  I can see T3 disks mapped under
:%standard /dev/dsk/cNtNdNsN devices.  Except on one server that simply
:%refuses to see the disks -- I tried to replace everything, from cables
:%and FC card in server to using different port on switch, it simply
:%don't see T3 disks.

:%If I configure FC switch for fiber topology than following happens:

:%On the switch I can see that T3 is connected to an FL port, while Sun
:%servers are connected to F ports on switch (T3 have old Qlogic 2100
:%chipset that supports only loop, and Sun FC cards have Qlogic 2200
:%chipset that supports both loop and fiber, so this is OK).

:%Now, the problem is that after reboot, Sun servers sometimes see T3
:%storage, and sometimes after reboot it simple dissapers (and reapears
:%after some number of reboots).  Also, when they see it, disks are
:%mapped under strange device names, something of the form cNtWNNdNsN
:%(where WNN is WNN of T3 FC controller).

There is a bug similar to this in sunsolve, but it involves HUBs instead of
switches. The problem seemed to have something to do with putting T3 on
consecutive ports in the HUB. You might try putting the T3s on non consecutive
ports in the switch. This is just a WAG, and may not work at all, but then
again it might help :)

Thanks,

Matt
----------
Matthew C. Aycock
Operating Systems Analyst/Developer, Lead
Dept Math/CS
Emory University, Atlanta, GA

In Fibre Channel, World Wide Port address are converted to targets and LUN's
by the driver.  Fabric aware (F_Port or N_Port) port can use different
address at boot.  Normally the address do not change if you do not change
port on the switch.  Brocade addressing can be predic by looking the port
number.  It's the nature of Fabric and works like DHCP even if you are
FL_port on the switch.  FL_port get there port address during LIP (Loop
Initialization Protocol) similar to F_port unless you are using AL_PA
specific.  In your case it a good advise to use AL_PA specific in your T3.
This avoid random address.

If you want to be sure to have the same target address you must have
hardware binding for your T3 WWN or port number to a specific target using
in the Qlogic .conf  file located in /kernel/drv/ .  Sample are within the
file.  To get your WWN or Port Number check in your /var/adm/message file
after boot.

In short:

1) Use AL_PA in your T3 if you have two T3 in pair you must zone your swich
and have two FC HBA.
2) Connect your T3 on the switch and check your WWN using the GUI
3) boot -rv and make sure you see your WWN and port number during the boot
or check the last entries in your message file.
4) modify your qlogic .conf file to use hardware binding
5) boot -r

Another avenue is to use Volume Management Software like Veritas Volume
Manager to avoid all addressing issue.  Volume Management software write
Volume ID on the disk and get recognise anytime even after changing the
server or switch.

I hope this help!
--
Jean Dion Tools & Programs
http://pages.infinit.net/dbtek/

Hm, the thing is that I don't have original Qlogic card (qla2200
driver).  I have Sun OEM card (qlc driver).  qla2200 driver have lot
of configuration options, but qlc driver I'm stuck with is stripped
down version of qla2200 and have only two configurable parameters: the
first one enables and disables hard code loop ID of the card and the
second one is which loop ID I want to give to the card if I enable
hard loop ID.  Since I'm running card in fibre mode, none of them are
of any interest to me.  Seems that Sun decided that all other options
from original qla2200 driver will not be needed by end-users and
removed them.

Also it seems that Sun version of driver is using path_to_inst to keep
WWN -> SCSI-target mappings constant on reboots.

--

Opinions expressed herein are my own.
Statements included here may be fiction rather than truth.

:> If you want to be sure to have the same target address you must have
:> hardware binding for your T3 WWN or port number to a specific target using
:> in the Qlogic .conf  file located in /kernel/drv/ .  Sample are within the
:> file.  To get your WWN or Port Number check in your /var/adm/message file
:> after boot.

: Hm, the thing is that I don't have original Qlogic card (qla2200
: driver).  I have Sun OEM card (qlc driver).  qla2200 driver have lot
: of configuration options, but qlc driver I'm stuck with is stripped
: down version of qla2200 and have only two configurable parameters: the
: first one enables and disables hard code loop ID of the card and the
: second one is which loop ID I want to give to the card if I enable
: hard loop ID.  Since I'm running card in fibre mode, none of them are
: of any interest to me.  Seems that Sun decided that all other options
: from original qla2200 driver will not be needed by end-users and
: removed them.

: Also it seems that Sun version of driver is using path_to_inst to keep
: WWN -> SCSI-target mappings constant on reboots.

Alexander,

Which OS rev are you using? Do you have the latest qlc driver, and
luxadm, and cfgadm patches? Check out http://www.sun.com/storage/san/
and do a search for SUNWsan of sunsolve. For the switch check out patch
110696-03. This has an updated GUI and firmware for the switch. You do need
the earlier version of SUNWsmgr installed for this patch.

Good Luck!

Lyle

why not just upgrade your HBAs to QLogic (branded) QLA2300 boards
& then upgrade the firmware on the SANbox
as a QLogic distributor i know that the SANbox came both ways (loop &
fabric) & that it is just a firmware upgrade difference

  _____                         .     .
 '    \\                  .                .                       |>>
     O//             .                        .                    |
    \_\          .                              .                  |
    | |      .                                    .      .         |
   /  |  .          http://born.as/storage         .   .    .      |

 / .  |        310-544-9439 / 310-544-9309 fax        .       .. . o
----------------------------------------------------------------------
Authorized - DIRECT VAR/VAD/Distributor for new SCSI/FC-AL peripherals
from:  IBM, Seagate, Quantum, NAS / SAN / RAID, QLogic, HDS, ATL, ect.

Yes I know that.  I already talk about it with people from Sun.  The
thing is if I install original Qlogic firmware, we loose support.

Quote:

> Yes I know that.  I already talk about it with people from Sun.  The
> thing is if I install original Qlogic firmware, we loose support.

...And yet another customer who fell in the "Yes, our SAN's are open!" trap...

Rob

OK, I'v installed tons of patches, upgraded firmware in SANbox16 and
HBAs and now I'm in process of playing around with it and seeing how
the things work (doesn't seem that anything changed, but it is too
early to say).

Anyhow, SUNbox now support fabric and F-ports.  But, now I have
problems with E-ports on it (they are called Fabric ports in SANSurfer
GUI? or am I mistakend here?).  If I connect two SANbox switches
everything works fine.  But if I connect it to Compaq switch (I
believe it is repacked Brocade SilkWorm 2400 switch, not sure), it
simply doesn't work -- Compaq switch has no idea which port type to
use, and there is no way for me to force port type to E-port.  Any
help with this?

Some questions on terminology used in Compaq (Brocade?) and Sun
(Qlogic) switches:

Is fabric port type in SANbox same thing as E-port in Compaq?
Is IOT mode on SANbox correct setting if I want to use multiple
switches that are not from the same vendor?
Chassis ID on SANbox is the same thing as Domain ID on Compaq?
Fabric ID is hardcoded to 1 on SANbox when in IOT mode.  Can't find
that parameter in Compaq switch configuration.  Does it have some
other name?

Quote:> Good Luck!

Thanks, I'll need it :-)

Compaq SANswitches use:
E-Port for Interswitch links
F-Port for Fabric adapter connection
L-Port for FC-AL adapter connection

Quote:> Is IOT mode on SANbox correct setting if I want to use multiple
> switches that are not from the same vendor?
> Chassis ID on SANbox is the same thing as Domain ID on Compaq?
> Fabric ID is hardcoded to 1 on SANbox when in IOT mode.  Can't find
> that parameter in Compaq switch configuration.  Does it have some
> other name?

The Compaq SANswitches have a switchDomain id, which is 1 by default. This
id has to be unique in the fabric. You can change it using the configure
command.
"switchShow" and "fabricShow" will tell you the current settings.

Quote:

> > Good Luck!

> Thanks, I'll need it :-)

Regards, Martin

There are interoperability problems between switch vendors... Sun is
using Qlogic switches with different firmware

I have been told that they seem to have been resolved ....  probably
with newer firmware ...

also you need to check the version of the drivers you are using.
especially for the switch. you need to ask the latest firmware from
SunService.

T3 is I remember correctly are using hardware AL_PA. if you 2 targets
have the same AL_PA within the same loop you might get problems. If I
remember correctly hardware AL_PA are the ones that are attributed
last.

Connecting the Compaq to the Sun switch would probably not be
supported.

Greg

OK, after some experimenting (two Compaq switches configuration, one
server node on first switch and two storage nodes (Sun T3 and Compaq
StorageWorks) on second switch), I'm getting 3 different error
messages for exactly same command:

# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Insufficient condition
# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Insufficient condition
# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Insufficient condition
# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Library error: Failed to get fabric device list
# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Library error: Failed to get fabric device list
# cfgadm -c configure 'c3::50001fe1000f8183'
cfgadm: Attachment point not found: failed to create device node: 50001fe1000f8183: I/O error

Above example was with an Compaq StorageWork disk array.  But I'm
getting exactly same errors with T3.

When I do "cfgadm -la", sometimes I'm getting both storage nodes,
sometimes only one, and sometimes none (before installing patches
cfgadm was not working at all).  Also, after installing pathces,
"luxadm probe -p" shows only two internal disks (before it was
sometimes showing external T3 disks connected on fc switches).

Output of "cfgadm -la" looks like this (sometimes lines with WNN of
two disks are missing for no apperent reason):

# cfgadm -la
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t6d0                 CD-ROM       connected    configured   unknown
c1                             fc-private   connected    configured   unknown
c1::21000004cf094d02           disk         connected    configured   unknown
c1::21000004cf094e28           disk         connected    configured   unknown
c2                             scsi-bus     connected    unconfigured unknown
c3                             fc-fabric    connected    unconfigured unknown
c3::50001fe1000f8183           disk         connected    unconfigured unknown
c3::50020f2300008088           disk         connected    unconfigured unknown
c4                             fc           connected    unconfigured unknown

c1 is internal FC controler (it is an Sun Fire280 system), and c3 is
Sun FC card (Qlogic ISP2200A chipset) connected to Compaq FC switch.
c4 is yet another card (same as c3), but it is not connected.

I also tried with only one server and one storage node connected to
the switches.  And I'm getting exactly same error message.

The patches I installed (in order of installation) were:

109189-02: ifp patch
108982-09: fp patch
108983-08: fcip patch
108984-08: qlc patch
111853-01: upgrade for firmware in ISP2200 HBAs
109529-06: luxadm patch
111847-01: SUNWsan patch
110912-01: cfgadm patch
111412-04: something for multipathing patch (required for 111846-01)
111413-04: luxadm patch (required for 111846-01)
111095-05: fp patch (required for 111846-01)
111096-03: fcip patch (required for 111846-01)
111097-04: qlc patch (required for 111846-01)
111846-01: fp plugin for cfgadm patch

Kernel patch on the systems I used for testing was the latest one
(108528-10), and Solaris version is 8 4/01.  So, basicly, I have no
idea what else to upgrade or patch...

As for the problem with SANbox described in one of my earlier posts, I
used firmware that came with 110696-03 to upgrade flash PROM (it
upgraded it from b30351 to 30462).

--

Opinions expressed herein are my own.
Statements included here may be fiction rather than truth.

Yes, I know that.  In SANbox I can set port type to F-port, Fabric,
SL-port, TL-port, F-Ext16, F-Ext24 and F-Ext32 (F-Ext* are grayed out
and cannot be selected in GUI).  The question was if "Fabric port" on
SANbox was "E-port".  If it is E-port, things are supposed to work
when I connect SANbox to Compaq switch.  If it is not E-port (for
example if it is some proprietary thing), than it is not possible to
connect SANbox and Compaq switches and I shouldn't bother trying.

Quote:> > Chassis ID on SANbox is the same thing as Domain ID on Compaq?
> > Fabric ID is hardcoded to 1 on SANbox when in IOT mode.  Can't find
> > that parameter in Compaq switch configuration.  Does it have some
> > other name?
> The Compaq SANswitches have a switchDomain id, which is 1 by default. This
> id has to be unique in the fabric. You can change it using the configure
> command.
> "switchShow" and "fabricShow" will tell you the current settings.

Yup, I also know that.  I have it set something like this:

First Compaq switch: Domain ID 2
Second Compaq switch: Domain ID 3
Third Compaq switch: Domain ID 4
First SANbox switch: Chassis ID 5
Second SANbox switch: Chassis ID 6

They are not all connected together right now, configuration with
which I'm testing have only first Compaq switch (ID 2) and first
SANbox switch (ID 5) connected to second Compaq switch (ID 3).

What I want to know is if Domain ID of Compaq switch and Chassis ID of
SANbox switch is the same thing (with different names).  And what is
this hard coded Fabric ID thing in SANbox (what is the equivalent in
Compaq switch)?

--

Opinions expressed herein are my own.
Statements included here may be fiction rather than truth.

Mixing different switch vendors (ie Ancor and Brocade/Compaq) is not
something you
are going to have much luck with. Interoperability in that regards is
not
currently working, much progress is being made in this regard, but they
are
not there yet. Stick to connecting one type of switch. You could make
multiple fabrics using different vendors, just don't try to connect
them.

Quote:

> Some questions on terminology used in Compaq (Brocade?) and Sun
> (Qlogic) switches:

> Is fabric port type in SANbox same thing as E-port in Compaq?

Fabric Ports on SANBox are for connecting hosts/storage. They are
analogous to
Fports on Brocade/Compaq. SANBox uses Tports in the same way as Brocade
uses Eports.

Quote:> Is IOT mode on SANbox correct setting if I want to use multiple
> switches that are not from the same vendor?

No, do not mix switches at this time. IOT is used when connecting two
Ancor switches.

Quote:> Chassis ID on SANbox is the same thing as Domain ID on Compaq?

I believe so.

Quote:> Fabric ID is hardcoded to 1 on SANbox when in IOT mode.  Can't find
> that parameter in Compaq switch configuration.  Does it have some
> other name?

Not sure on this one.

Quote:

> > Good Luck!

> Thanks, I'll need it :-)

50001fe1000f8183: I/O error

Quote:

> Above example was with an Compaq StorageWork disk array.  But I'm
> getting exactly same errors with T3.

In this configuration, using the Compaq switches, how do you have the HBAs
connected? Via Fports (fabric) or are you using Quickloop?
Currently there is a bug against using QLogic HBAs (which Sun OEMs) in Fport
mode with Brocade switches .If Compaq switches are just rebranded
Brocade switches you will hit the same bug. If you have a valid support
contract you could contact Sun Service or use sunsolve.sun.com to search for
details on this bug. Keywords would be Brocade & Fabric. I am not aware of
how this bug is being tracked by either Qlogic or Brocade.

The workaround is to use Quickloop. Sun has not officially released their
supported configurations using Brocade switches with Qlogic HBAs in Fabric
mode. You must currently use Quickloop. Unfortunately, Quickloop is a
separately licensed feature of the Brocade switch, I'm not sure how Compaq
handles it.

It is supported to use Qlogic HBAs and the Sun switches in Fabric mode. This
support was released 2 weeks ago. In this case, configure the T3 to be on a
TL-port. The TL stands for Translative Loop and is required when connecting
Private Loop devices (which the T3 is..it uses the Qlogic ISP 2100 chip,
which is private loop only) to Public Hosts, which the Qlogic 2200A HBA is
capable of being with the patches you have listed below. The HBA should be
configured as a Fport on the Sun Switch.(do not use FL, it is not supported
at this time)

I am not familar with how the Compaq Storageworks should be configured. If
it is a private loop device, use TL-ports on the Sun switch. If it is a
public loop device use FL-Ports.

I hope this helps

-ryan