Binding "Threads" to processors under Solaris 2.3

Binding "Threads" to processors under Solaris 2.3

Post by lantz moo » Fri, 27 May 1994 03:39:10



Is there any way of binding threads (thread_t) to a processor, without
binding a thread to an lwp (which is in turn bound to a processor)?  Or,
even better, is there any way of restricting an lwp to a set of processors
(>= 1)?

i have a 4 processor SPARCserver 1000.  I want to show how a parallel
application behaves under 1, 2, 3, and 4 processors (is speed up linear,
etc).  So i'd like to be able to restrict the set of processors usable,
without having to go to the trouble of actually turning a processor off.

Any advice will be greatly appreciated.

-lantz

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Bil Lew » Fri, 27 May 1994 08:34:33


: Is there any way of binding threads (thread_t) to a processor, without
: binding a thread to an lwp (which is in turn bound to a processor)?

  No.  Threads are not schedulable entities, only LWPs.

: Or,
: even better, is there any way of restricting an lwp to a set of processors
: (>= 1)?

  I believe so, but it's not recommended & a little ugly.

: i have a 4 processor SPARCserver 1000.  I want to show how a parallel
: application behaves under 1, 2, 3, and 4 processors (is speed up linear,
: etc).  So i'd like to be able to restrict the set of processors usable,
: without having to go to the trouble of actually turning a processor off.

  Probably you'll be much happier turning the processors off.  No possible
interference, no wondering "IF" maybe you made a mistake.  I forget the
command to give, but I know there is one.

-Bil

--
  You must run as fast as to can, just to stay in the same place.  If you
want to get anywhere, you must run twice as fast!



 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Christopher Barb » Sat, 28 May 1994 02:00:29


    LM> is there any way of restricting an lwp to a
    LM> set of processors (>= 1)?

    BL>   I believe so, but it's not recommended & a little ugly.

Supposedly, you can use the system call processor_bind() to bind an LWP to
a specific processor.  However, when I tried this on a 4 processor 670, and
bound all threads to the same processor the program behaved as if the load
was distributed across all the processors.  Since there is no system call
to query whether an LWP is bound and to what, there is no way to tell
exactly what is going on here.  The only way I could force all the LWPs to
run on a single processor was to turn the other processors off (using the
psradm command).

- Chris
--
Christopher Barber

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by James Litchfie » Sat, 28 May 1994 10:05:20


Quote:> Questions about binding threads to processors

You can bind a particular LWP to a processor (see processor_bind(2) for the
programmatic interface). One can create a thread bound to an LWP and then
use processor_bind to bind that thread to a particular processor. It must,
of course, be pointed out that the scheduler does try to keep caches warm
by default and will reschedule an lwp on the same cpu it most recently
ran on (if available) within a certain time quantum.
 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by James Lawt » Thu, 02 Jun 1994 21:37:25



>: i have a 4 processor SPARCserver 1000.  I want to show how a parallel
>: application behaves under 1, 2, 3, and 4 processors (is speed up linear,
>: etc).  So i'd like to be able to restrict the set of processors usable,
>: without having to go to the trouble of actually turning a processor off.
>  Probably you'll be much happier turning the processors off.  No possible
>interference, no wondering "IF" maybe you made a mistake.  I forget the
>command to give, but I know there is one.

The command name is "psradm".

In a related question, is there any way to determine on what processor
your thread/LWP is running?  That is, I have a four processor machine,
and want to run 4 threads concurrently.  How can I find out if they
are all executing, and on which processor they are?

--
James H. Lawton            || Disclaimer: The opinions and views
Rome Laboratory            || expressed here are MINE!  They may
Griffiss AFB, NY 13441     || not reflect those of Rome Laboratory,

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Christopher Barb » Fri, 03 Jun 1994 23:50:27


    JL> In a related question, is there any way to determine on what
    JL> processor your thread/LWP is running?  That is, I have a four
    JL> processor machine, and want to run 4 threads concurrently.  How can
    JL> I find out if they are all executing, and on which processor they
    JL> are?

Not as far as I know.  Of course, even if you did have such a function,
the answer could change as soon as it is delivered since the scheduler is
free to move threads around as it sees fit.  Of course, you could bind the
LWPs to specific processors, but my experiments seem to indicate that the
processor_bind() system call is ignored in any case (any comment from
someone at Sun?).

- Chris

--
Christopher Barber

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Bil Lew » Sat, 04 Jun 1994 08:44:35


Binding threads to LWPs...  Don't.

Yes you CAN, but it's not what you want to do.  99.9% of the people who
think they need that are wrong.  (Yes, I understand that you are that 0.1%,
but that's what all the other guys say too.)

Before you commit this mistake, look VERY VERY hard at the problem, ask a
peer to review it.  Tell 'em some guy at Sun said "don't".  And if you still
want to do it...  tell me how it went.  I'll be interested to know

-Bil
--
  You must run as fast as to can, just to stay in the same place.  If you
want to get anywhere, you must run twice as fast!


 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Casper H.S. D » Sat, 04 Jun 1994 02:10:26



Quote:># No processor bind/ bound threads
>Oops 4 != 3  (lwp_self() != thr_self())
>Oops 6 != 5
>Oops 5 != 4
>26.26u 0.09s 0:26.35 100.0%

Apparently, tcsh's notion of time gets confused by processes
easting more than 100% of the CPU time.

Csh says:
29.0u 0.0s 0:10 289% 0+0k 0+0io 0pf+0w  (3 lwps, unbound)
19.0u 0.0s 0:10 189% 0+0k 0+0io 0pf+0w  (bound thr_self())
10.0u 0.0s 0:09 100% 0+0k 0+0io 0pf+0w  (bound lwp_self())

As it should, 10 seconds elapsed time each go.

Casper

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Christopher Barb » Sun, 05 Jun 1994 01:20:51



    >> Of course, even if you did have such a function, the answer could
    >> change as soon as it is delivered since the scheduler is free to
    >> move threads around as it sees fit.  Of course, you could bind the
    >> LWPs to specific processors, but my experiments seem to indicate
    >> that the processor_bind() system call is ignored in any case (any
    >> comment from someone at Sun?).

    >> Works for me, if I interpret the numbers right.

    >> But you must:

    >>    - create the threads you want to bind with THR_BOUND - call
    >> processor_bind with lwp_self() not thr_self()! ....

Yes, I know.  Actually, there were was a different bug in my test program
(which was a lot more complicated than yours....).  So forget what I said
about processor_bind() not working. :-`)

- Chris
--
Christopher Barber

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Casper H.S. D » Sat, 04 Jun 1994 01:57:13



>Not as far as I know.  Of course, even if you did have such a function,
>the answer could change as soon as it is delivered since the scheduler is
>free to move threads around as it sees fit.  Of course, you could bind the
>LWPs to specific processors, but my experiments seem to indicate that the
>processor_bind() system call is ignored in any case (any comment from
>someone at Sun?).

Works for me, if I interpret the numbers right.

But you must:

        - create the threads you want to bind with THR_BOUND
        - call processor_bind with lwp_self() not thr_self()!

(On a 4CPU Solaris machine, I created 3 threads with a while(1)
loop.  The main thread did a sleep(10).  The threads where THR_BOUND
and pbind was called on lwp_self().  The result was 10second of CPU
time.  If the binding hadn't taken place:

# No processor bind/ bound threads
Oops 4 != 3  (lwp_self() != thr_self())
Oops 6 != 5
Oops 5 != 4
26.26u 0.09s 0:26.35 100.0%

# Processor bind on lwp_self/ bound threads
time ./thr
Oops 5 != 4
Oops 6 != 5
Oops 4 != 3
9.71u 0.06s 0:09.77 100.0%

# Processor bind on thr_self/ bound threads
Oops 6 != 5
Oops 4 != 3
Oops 5 != 4
19.21u 0.08s 0:19.29 100.0%

(Because of the overlap between ids we get two of the threads bound to one
CPU: only 20s CPU time).

When mixing system calls and library calls you must be very carefull.
The processor_bind() system call works on lwps.  The abstraction used
by the thread library is threads.  The lwp ID expected by pbind
cannot be interchanged with a thread id as returned by thr_self,
nor can you perform system calls on threads, unless they are
THR_BOUND threads.

If you with to bind a thread to a processor, want to change the
priority or shceduling calss of a thread or whatever other system
parameter of thread, it *must* be a bound thread and you *must*
operate on the lwp_self() value, not the thr_self() value.

Casper

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Simon E Spe » Sun, 05 Jun 1994 01:37:03



>Binding threads to LWPs...  Don't.

>Yes you CAN, but it's not what you want to do.  99.9% of the people who
>think they need that are wrong.  (Yes, I understand that you are that 0.1%,
>but that's what all the other guys say too.)

Hmmm - I wonder what the 0.1% cases are?

My initial guesses would be:
        extreme real time contraints (bind to a process go to maximum
        RT priority, and busy wait so the processor is always locked and loaded.

        Several threads with moderate sized data requirements such that one
        threads data can fit entirely in per-processor cache. I don't know
        if this would really win that much over libthread's default strategy.

        Real time processes on heavily loaded machines, where it might take too
        long to clear space for the new thread to let the thread meet its
        RT constraints.

Any more?

Simon

--

-------------------------------------------------------------------------------
I have heard the routers pinging, each to each.         | Tel: +1-919-962-9107,
I do not think that they will ping to me-               | Fax: +1-919-962-5604

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by Kevin Sheehan {Consulting Poster Chil » Tue, 07 Jun 1994 04:25:58



>Binding threads to LWPs...  Don't.
>Yes you CAN, but it's not what you want to do.  99.9% of the people who
>think they need that are wrong.  (Yes, I understand that you are that 0.1%,
>but that's what all the other guys say too.)
>Before you commit this mistake, look VERY VERY hard at the problem, ask a
>peer to review it.  Tell 'em some guy at Sun said "don't".  And if you still
>want to do it...  tell me how it went.  I'll be interested to know

I'll agree that 99.9% of the time they are wrong.  We other decimal places
*do* have a use for it.

In applications involving overlapped I/O (aka async, but that is somewhat
overloaded these days) using threads, it is occasionally necessary.

An example (all too sadly from real life :-)

An application does I/O to a driver that is not modifiable, and that does
not sort the requests by offset.  In other words, the requests must be
presented in the proper order.

As thread priority is LWP acquisition priority, *NOT* scheduling priority,
RT threads are required in order to do deterministic scheduling.  Before
request N+1 can be issued, request N must run and block on the I/O request
before the thread generating request N+1 can issue the I/O.

The thread(s) doing I/O therefore run at higher priority than the thread
generating the requests.  Only RT will do this correctly, and to do that
you need THR_BOUND, as the priorities are done with the LWP.

In the 99.9% case, you either have a driver that does the Right Thing
and sorts, so weakly ordered I/O is not a problem, or the order of
the requests does not matter.

Bottom line - if you require deterministic scheduling, and many RT
applications do, you have to use a bound thread.  Now if I could only
*them too :-)

                l & h,
                kev


Uniq Professional Services Pty Ltd  ACN 056 279 335 |    Why Not?
PO Box 70, Paddington, NSW 2021, (Sydney) Australia |
Phone: +61-2-360-7434           Fax: +61-2-331-2572 |

 
 
 

Binding "Threads" to processors under Solaris 2.3

Post by David P. Ke » Tue, 07 Jun 1994 06:42:59




Quote:

>I'll agree that 99.9% of the time they are wrong.  We other decimal places
>*do* have a use for it.

>In applications involving overlapped I/O (aka async, but that is somewhat
>overloaded these days) using threads, it is occasionally necessary.

>An example (all too sadly from real life :-)

>An application does I/O to a driver that is not modifiable, and that does
>not sort the requests by offset.  In other words, the requests must be
>presented in the proper order.

>In the 99.9% case, you either have a driver that does the Right Thing
>and sorts, so weakly ordered I/O is not a problem, or the order of
>the requests does not matter.

>Bottom line - if you require deterministic scheduling, and many RT
>applications do, you have to use a bound thread.  Now if I could only
>gag them too :-)

I must be missing something big here.  It doesn't seem that you need either
RT scheduling or bound threads, if the only requirement is to present
I/O requests in the correct order.

You are synchronizing the threads properly with a mutex (and perhaps
a CV), right?  So what's the scheduling problem?

--

-----------------------------------------------------------------------------
"If it doesn't have an analogy, it isn't Usenet (tm)"

 
 
 

1. Solaris 2.3 "No directory" "No shell"

I'm running SunOS 5.3 on a SPARC 10. We are now getting
"No shell" or "No directory!" errors whenever we log in.
(rlogin, su, login, ...)

We can cd to the directory so we know there's no problem.
We can execute the shell and change to it.

A "truss su - randerso" stops with:
==========================================================
# tail /usr/tmp/truss.lis
poll(0x0003C0B0, 1, 120000)                     = 1
getmsg(4, 0xEFFFE994, 0xEFFFE988, 0xEFFFEA04)   = 0
close(4)                                        = 0
setgroups(4, 0x00031438)                        = 0
setuid(4540)                                    = 0
chdir("/u/randerso")                            Err#13 EACCES
execve("/bin/bash", 0xEFFFFDE0, 0x00023464)     Err#13 EACCES
write(2, " s u :   N o   s h e l l".., 13)      = 13
lseek(0, 0, 1)                                  = 30284
_exit(3)
#
============================================================

That EACCES error appears to be false.
============================================================
# ls -ld /u/randerso /bin/bash
-rwxr-xr-x   1 root     other    1661324 May 15 09:25 /bin/bash
drwxr-xr-x 113 randerso users       5632 Jul 13 17:01 /u/randerso
============================================================

Any help and/or insight will be appreciated.

Lew Randerson

--

--
Lewis E. Randerson
44 Scott Lane                         phone:609/921-2553 FAX:609/921-3487

2. The right combo's work everytime.

3. "sz" and "rz" for solaris 2.3 ??

4. emacs mail

5. "connect" between "bind" and "listen"

6. GUI and Unix

7. GETSERVBYNAME()????????????????????"""""""""""""

8. Picture shaking!

9. Screen doesn't unblank with "screenblank" and "xlock" under 2.3

10. """"""""My SoundBlast 16 pnp isn't up yet""""""""""""

11. HELP - Need The Solaris 2.3 equivalent of Solaris 1.1's "bar"

12. Newbie question about "Listen" and "Bind address"

13. Can "pppd" bind to a pty with "telnet" session over ethernet?