[patch] batch/idle priority scheduling, SCHED_BATCH

[patch] batch/idle priority scheduling, SCHED_BATCH

Post by Vitez Gabo » Fri, 05 Jul 2002 22:10:05




> the load generated by multiple SCHED_BATCH processes does not show up in
> the load average - this is the straightforward solution to not confuse
> load-average-sensitive applications such as sendmail.

I think this will confuse atd too, which is an obvious candidate
for the batch scheduler; it may end up starting all jobs which
sit in it's "batch" queue.

I think a load-average calculation scheme like this would be better:

oldload: is the load average calculated the old way
batchload: is the load average calculated only from the batch scheduler
numcpus: number of cpus...

newload(){
if (oldload > numcpus) return oldload;
if ((oldload+batchload) > numcpus) return numcpus;
return (oldload+batchload)

Quote:}

So the batch processes would show the CPUs maxed out, but would not show
up as overload in the load average. (and you could run
"atd -l <numcpus - 0.3>")

        regards:
                Gabor Vitez
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[patch] batch/idle priority scheduling, SCHED_BATCH

Post by Jakob Oestergaar » Mon, 08 Jul 2002 21:50:07




> > the load generated by multiple SCHED_BATCH processes does not show up in
> > the load average - this is the straightforward solution to not confuse
> > load-average-sensitive applications such as sendmail.

If SCHED_BATCH load does not affect the load average, then it will *not*
confuse sendmail - because the box is not really loaded, SCHED_BATCH
will only run if sendmail wouldn't   :)

I think it's great that SCHED_BATCH doesn't show up in loadavg - it is
really confusing to look at load statistics for boxes which have these
CPU intensive batch jobs running - the load average is perhaps
constantly between 2.1 and 2.2, when the "real" load on the box would
cause it to be between .1 and .2

Idle-time is what you pay for but do not get  ;)   Having CPU hogs
running is a nice way of getting the last penny out of the investment,
but I have many boxes where I don't do it, simply because it would
render the load statistics useless.

Quote:

> I think this will confuse atd too, which is an obvious candidate
> for the batch scheduler; it may end up starting all jobs which
> sit in it's "batch" queue.

True.  I am inclined to saying that any batch system that doesn't keep
track of it's own jobs but only cares about the load average, is flawed.

Load average is "guidance", it is a heuristic, it is not something one
should use as the sole measure when deciding when/where to spawn jobs.
Stuff like that only works in theory...

But atd is in use everywhere, and it does use the load as the only
metric (which is, by the way, why it doesn't start more than one job
per minute, because the loadavg needs time to rise).

Wouldn't it be pretty simple to just make atd have a hard limit on how
many concurrent at jobs (eventually on a per-user basis) it would start?

Unless you really go to extremes (thousands of jobs), the performance of
ten "concurrently" running SCHED_BATCH jobs and the same ten jobs beeing
launched sequentially by atd, should be fairly similar - given the huge
time-slices given by the scheduler to these jobs.

Quote:

> I think a load-average calculation scheme like this would be better:

> oldload: is the load average calculated the old way
> batchload: is the load average calculated only from the batch scheduler
> numcpus: number of cpus...

> newload(){
> if (oldload > numcpus) return oldload;
> if ((oldload+batchload) > numcpus) return numcpus;
> return (oldload+batchload)
> }

> So the batch processes would show the CPUs maxed out, but would not show
> up as overload in the load average. (and you could run
> "atd -l <numcpus - 0.3>")

Hmmm...  Such a hack might work around the shortcomings in the atd
scheduling algorithm.

I don't like it. It adds yet another level of obfuscation to loadavg.

I bet you can do the proper changes to atd in the same amount of code,
keeping the kernel clean and fixing the problem where it really is.

Just my 0.02 Euro,

--
................................................................

:.........................: putrid forms of man                :
:   Jakob ?stergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[patch] batch/idle priority scheduling, SCHED_BATCH

Post by Bernd Eckenfel » Tue, 09 Jul 2002 04:10:10



> Idle-time is what you pay for but do not get  ;)   Having CPU hogs
> running is a nice way of getting the last penny out of the investment,
> but I have many boxes where I don't do it, simply because it would
> render the load statistics useless.

And of course it needs a bit more power and produces much more temperature
(and therefore noise).

BTW: even an batch prio will load the system more than an idle process. The
cache is dirty, the disk has a longer work queue, RSS is used up by that
programs. So for a slow system you have to expect impact.

Greetings
Bernd
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. [patch] batch/idle priority scheduling, SCHED_BATCH

From IEEE 1003.1-2001 / Open Group Base Spec. Issue 6:
"Conforming implementations shall include one scheduling policy
identified as SCHED_OTHER (which may execute identically with either the
FIFO or round robin scheduling policy)."

So, you probably want to add a "#define SCHED_OTHER SCHED_NORMAL" here
in order to prevent future confusion, especially because the user-space
headers have SCHED_OTHER, but no SCHED_NORMAL.

- Nicholas

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. linux problem

3. sched-2.5.24-D3, batch/idle priority scheduling, SCHED_BATCH

4. How to find number of messages buffered in an interface?

5. new kernels

6. Batch scheduling and fixes added to performance patches (-ck)

7. AVM-B1 macht doch keine Faxen

8. Setting Schedule Priority

9. thread scheduling and priority

10. Running consoles with low scheduling priority?

11. batch processing priority

12. Scheduling of low-priority background processes