Solaris8 x86 PCI Driver with multiple PCI devices

Solaris8 x86 PCI Driver with multiple PCI devices

Post by Steve We » Sat, 20 Oct 2001 02:55:04



I have a PCI adapter card with multiple PCI devices on it. I am
currently trying to find a way to run a PCI bus scan from within my
device driver. The DDK/DKI services seem to only operate on the PCI
configuration that was passed to the driver when it loaded/started. I
would like to program all of the devices on my card from within one
driver if possible. If anyone has any code examples or information on
this I would be very appreciative.

Steve West

 
 
 

Solaris8 x86 PCI Driver with multiple PCI devices

Post by Bruce Adle » Sat, 20 Oct 2001 08:04:55



> I have a PCI adapter card with multiple PCI devices on it. I am

                                 ^^^^^^^^^^^^^^^^^^^^

Quote:> currently trying to find a way to run a PCI bus scan from within my
> device driver.

If your device conforms to the PCI Specs there's no reason to do
anything like that.

Did you really mean to say "multiple PCI devices". If so, then
your card must have a PCI-to-PCI Bridge chip on it. If it does
have a P-P Bridge, then each of your PCI devices shows up on the
secondary side of the P-P bridge and should be listed in the output
of the prtconf command as individual device instances that are child
nodes below the P-P Bridge. Your driver basically completely ignores
the presence of the P-P Bridge (it's handled by the Sun-supplied
"pci_pci" driver) and you just write a normal Solaris device driver
that attaches to one or more device instances. That's the way cards
like Adaptec's AHA-3940 dual PCI-SCSI controller, and various 4-port
Tulip-based PCI-NICs work. Such systems simply have multiple "adp" or
"dnet" instances (for example).

If you're not using a P-P Bridge, then what did you really mean
to say?

Did you instead perhaps mean to say that you've got a "Multifunction
PCI Device"? If so, then each of your device's subfunctions
should also show up in the prtconf output with unique PCI devices
with non-zero values in the "function" field of their PCI
addresses. Each "subfunction" of a Multifunction PCI Device has its
own PCI bus address and its own set of PCI Configuration Registers. So
other than the special form of the device pathname, as far as your
device driver is concerned, the separation between the subfunction
devices is indistinguishable from the separation between totally unique,
unrelated, and unconnected PCI devices in different slots. For example:

three separate cards:




versus a single multifunction card:




That's a Multifunction PCI Device that's inserted in whatever slot
is addressed as "device 8". The ",1" and ",2" parts of the addresses
are the "function" numbers, and indicate the second and third devices
of a Multifunction PCI Device that contains three subfunctions. The
first two subfunctions have identical Vendor and Device ID values
(7000, 1) and would attach to separate instances of a single device
driver. But the third subfunction has a unique Vendor and Device value
(7000, 2) and could either (depending on what bindings you put in the
/etc/driver_aliases file) attach to a completely different
driver, or a third instance of the same driver that attaches to the
first two instances.  See for example the prtconf output for any
"southbridge" chip from any PCI chip vendor, nowadays ,they're all
Multifunction PCI devices and contain multiple USB controllers.

Or, did you perhaps mean to say that you've got a single PCI device
which has multiple sets of device registers and/or memory locations,
mapped through its PCI Base Address Registers, that control multiple
logical devices? That's *not* a "Multiple PCI Device" card because
the logical sub-devices don't have their own unique PCI bus addresses
and unique sets of PCI Configuration Registers. Rather, that's a *single*
PCI device with a complicated implementation. Those "sub-devices" are
not separate PCI devices so there's no way to make them visible in
the device tree as separate PCI device nodes.

Quote:> The DDK/DKI services

You mean "the DDI/DKI", there's no such thing as DDK/DKI services.

Quote:> seem to only operate on the PCI
> configuration that was passed to the driver when it loaded/started.

That's correct. Leaf device drivers should *not* care at all how the
device tree was created nor what its current "shape" is. Even on systems
that support hot-plugging and/or dynamic reconfig, it's not the
job of a leaf device driver to manipulate the device tree. A leaf
device driver simply supports whatever device it attaches to and defers
the job of managing the overall system configuration to the rest of
the kernel and the Sun-supplied nexus drivers. The PCI leaf nodes of
a device tree that come and go due to configuration changes are
auto-magically added and deleted by the kernel as needed.

Quote:> I would like to program all of the devices on my card from within one
> driver if possible.

Exactly what kind of card is that and what kind of devices does it
contain? I suspect that, although your card has a PCI interface on
the host side, that the multiple devices it contains aren't really
"PCI Devices" and therefore there's no need for you to figure out
how to dynamically create and destroy PCI nodes in the device tree.

Even if they really are honest-to-god PCI devices, you shouldn't
be thinking in terms of writing a single monolithic driver and/or
dynamically modifying the device tree. I don't see any advantages to
that model over the current model that requires either attaching
multiple instances of a single driver (if every one of your devices
is identical), or attaching different instances of unique drivers
(if every one of your devices is different)? Writing a single
monolithic driver that grabs everything and touches everything might
seem easier for you but that's not a good general solution and doesn't
fit the architectural model that Solaris implements for PCI hardware
support.

If they really are separate PCI Devices (that share access to the PCI
bus via a common bridge), then why would you want to needlessly
tie them together via a monolithic device driver? That sort of
implementation always leads to unexpected (and almost always undesirable)
side effects and interactions between things like interrupt handlers,
open and close functions, and/or mutex contention. In general,
it's best minimize as much as possible interactions between logically
separate devices, or even logically separate functions within a single
device. For example, even within a NIC driver, the receive and transmit
operations should be as separate as possible to achieve the best
performance (unfortunately no hardware vendor builds a PCI-NIC that
completely and totally separates the receive and transmit hardware
so it's not possible to split a NIC driver into two separate drivers).

Quote:> If anyone has any code examples or information on
> this I would be very appreciative.

Read the driver writing material on Sun's developer site:

    http://soldc.sun.com

 
 
 

Solaris8 x86 PCI Driver with multiple PCI devices

Post by Philip Bro » Mon, 22 Oct 2001 01:17:41



>...
>Did you really mean to say "multiple PCI devices". If so, then
>your card must have a PCI-to-PCI Bridge chip on it.
>...

 Your driver basically completely ignores

Quote:>the presence of the P-P Bridge (it's handled by the Sun-supplied
>"pci_pci" driver) ...

Really nice summary of these issues, Bruce.
It would be nice to see your post archived and put up somewhere like
soldc (after slight reformatting, of course ;-)

--
[Trim the no-bots from my address to reply to me by email!]
[ Do NOT email-CC me on posts. Pick one or the other.]

The word of the day is mispergitude

 
 
 

1. Driver for Multiple PCI Devices

I have written a krenel driver module for a PCI adapter which works
perfectly.  Now I would like to support multiple PCI adapters of the
same type.   I can distinguish between the different adapters by the
PCI configuration data but I don't understand how to use the same
driver for access to multiple adapters.  Presumeably, I will need an
additional set of /dev/... nodes for each additional device but what
else?  What is the standard, "good practice"  way to approach this
problem?

I would appreciate any suggestions or pointers to example code.

Thanks,

Frank  McGirt

2. automatic xdm login

3. x86 PCI Device Driver

4. ls --color in an xterm

5. x86 device drivers and PCI interrupt

6. Screwing with a friends gerp (ksh)

7. PCI Device driver compatibilty between SPARC and x86 platforms

8. I cant get back to win98!

9. pci x86 solaris 2.5 device drivers

10. PCI driver location & Where can I get more PCI drivers?

11. QUERY:- pci driver to mini-pci driver

12. Any ISDN PCI adapters that work on Solaris8/x86?

13. PCI device vs SCSI device driver