batch job help

batch job help

Post by Carlo » Thu, 01 Feb 2001 03:43:29



Hello.  I'm new to Unix (AIX 4.3.x) and we are in the process of
porting our application programs from a DG AOS/VS II to AIX.  We submit
jobs in the DG to a batch job queue which may have several streams to
service jobs that are submitted with certain Queue Priority.  One batch
stream can only service one job at a time.  Take for example inventory
receiving post which may be requested by several users simultaneously
is submitted to stream 4.  Stream 4 will execute the posting program
for each user in a sequential order, one post at a time.

Can this be done in AIX?  I tried using the batch which submits the job
alright but executes the job simultaneously - it doesn't line up the
jobs submitted in a sequential manner.

Thanks for your help in this matter.

Carlos

--
Have a nice day.

Sent via Deja.com
http://www.deja.com/

 
 
 

batch job help

Post by smurf » Thu, 01 Feb 2001 04:29:05




Quote:> Hello.  I'm new to Unix (AIX 4.3.x) and we are in the process of
> porting our application programs from a DG AOS/VS II to AIX.  We
submit
> jobs in the DG to a batch job queue which may have several streams to
> service jobs that are submitted with certain Queue Priority.  One
batch
> stream can only service one job at a time.  Take for example inventory
> receiving post which may be requested by several users simultaneously
> is submitted to stream 4.  Stream 4 will execute the posting program
> for each user in a sequential order, one post at a time.

> Can this be done in AIX?  I tried using the batch which submits the
job
> alright but executes the job simultaneously - it doesn't line up the
> jobs submitted in a sequential manner.

> Thanks for your help in this matter.

> Carlos

> --
> Have a nice day.

> Sent via Deja.com
> http://www.deja.com/

You can use the printing subsystem to achieve queueing of the jobs, you
can write your own backend script or maybe even use a shell as the
backend.

Writing you own script is probably the best idea. There is a fair
amount of customisation you can do of course, you could I am sure get
the queue to go to DOWN status if a batch job fails.

Otherwise there are commercial products available to do this.

--
#---------------------------------------------#
# Humble opinions no warranty implied
#---------------------------------------------#

Sent via Deja.com
http://www.deja.com/

 
 
 

batch job help

Post by Carlo » Thu, 01 Feb 2001 04:58:22


Thanks, Smurfy for your reply.

How does a backend created?

Can someone point me to where I can find what are the requirements of a
backend??

Thanks.

Carlos





> > Hello.  I'm new to Unix (AIX 4.3.x) and we are in the process of
> > porting our application programs from a DG AOS/VS II to AIX.  We
> submit
> > jobs in the DG to a batch job queue which may have several streams
to
> > service jobs that are submitted with certain Queue Priority.  One
> batch
> > stream can only service one job at a time.  Take for example
inventory
> > receiving post which may be requested by several users
simultaneously
> > is submitted to stream 4.  Stream 4 will execute the posting program
> > for each user in a sequential order, one post at a time.

> > Can this be done in AIX?  I tried using the batch which submits the
> job
> > alright but executes the job simultaneously - it doesn't line up the
> > jobs submitted in a sequential manner.

> > Thanks for your help in this matter.

> > Carlos

> > --
> > Have a nice day.

> > Sent via Deja.com
> > http://www.deja.com/

> You can use the printing subsystem to achieve queueing of the jobs,
you
> can write your own backend script or maybe even use a shell as the
> backend.

> Writing you own script is probably the best idea. There is a fair
> amount of customisation you can do of course, you could I am sure get
> the queue to go to DOWN status if a batch job fails.

> Otherwise there are commercial products available to do this.

> --
> #---------------------------------------------#
> # Humble opinions no warranty implied
> #---------------------------------------------#

> Sent via Deja.com
> http://www.deja.com/

--
--
Have a nice day.

Sent via Deja.com
http://www.deja.com/

 
 
 

batch job help

Post by Markus B?rtsch » Thu, 01 Feb 2001 06:47:07



Quote:> Thanks, Smurfy for your reply.

> How does a backend created?

> Can someone point me to where I can find what are the requirements of a
> backend??

> Thanks.

> Carlos

A backend is just an ordinary program or shell script. Here an example
of a shell queue. It will execute the submitted scripts sequentially.

----- /etc/qconfig ----
bsh:
??discipline = fcfs
??device = bshdev
bshdev:
??backend = /usr/bin/ksh
-------------------------

To run a shell procedure called myproc using this batch queue, enter:

$ qprt -Pbsh myproc

Here some links into the AIX docs

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/printrgd/prt_
backend_overview.htm

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/qconfi
g.htm

 
 
 

batch job help

Post by Carlo » Thu, 01 Feb 2001 07:46:11


Yes, yes, yes.  This is exactly what I did and I have tested running
jobs sequentially using the queue I created "script" with the backend
= /usr/bin/ksh.

I have found out though that the initial directory that the script will
start running is in /var/spool/lpd/qdir.  Is there a way to configure
the behavior of ksh so that the initial working directory is the
current directory where the "qprt -Pscript myproc" was envoked?

Thanks for all your help.

Carlos




says...
> > Thanks, Smurfy for your reply.

> > How does a backend created?

> > Can someone point me to where I can find what are the requirements
of a
> > backend??

> > Thanks.

> > Carlos

> A backend is just an ordinary program or shell script. Here an
example
> of a shell queue. It will execute the submitted scripts sequentially.

> ----- /etc/qconfig ----
> bsh:
> ??discipline = fcfs
> ??device = bshdev
> bshdev:
> ??backend = /usr/bin/ksh
> -------------------------

> To run a shell procedure called myproc using this batch queue, enter:

> $ qprt -Pbsh myproc

> Here some links into the AIX docs

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/printrgd/prt_
Quote:> backend_overview.htm

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/qconfi

Quote:> g.htm

--
--
Have a nice day.

Sent via Deja.com
http://www.deja.com/

 
 
 

batch job help

Post by Norman Levi » Thu, 01 Feb 2001 11:03:13


How come you are using this mechanism instead of the classic 'batch' facility?


> Yes, yes, yes.  This is exactly what I did and I have tested running
> jobs sequentially using the queue I created "script" with the backend
> = /usr/bin/ksh.

> I have found out though that the initial directory that the script will
> start running is in /var/spool/lpd/qdir.  Is there a way to configure
> the behavior of ksh so that the initial working directory is the
> current directory where the "qprt -Pscript myproc" was envoked?

> Thanks for all your help.

> Carlos




> says...
> > > Thanks, Smurfy for your reply.

> > > How does a backend created?

> > > Can someone point me to where I can find what are the requirements
> of a
> > > backend??

> > > Thanks.

> > > Carlos

> > A backend is just an ordinary program or shell script. Here an
> example
> > of a shell queue. It will execute the submitted scripts sequentially.

> > ----- /etc/qconfig ----
> > bsh:
> >   discipline = fcfs
> >   device = bshdev
> > bshdev:
> >   backend = /usr/bin/ksh
> > -------------------------

> > To run a shell procedure called myproc using this batch queue, enter:

> > $ qprt -Pbsh myproc

> > Here some links into the AIX docs

> http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/printrgd/prt_
> > backend_overview.htm

> http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/qconfi
> > g.htm

> --
> --
> Have a nice day.

> Sent via Deja.com
> http://www.deja.com/

--
Norman Levin

greatest power, the village idiot will come forth to be acclaimed the
leader.'"

 
 
 

batch job help

Post by Dell Colem » Fri, 02 Feb 2001 11:35:32



> I have found out though that the initial directory that the script will
> start running is in /var/spool/lpd/qdir.  Is there a way to configure
> the behavior of ksh so that the initial working directory is the
> current directory where the "qprt -Pscript myproc" was envoked?

There are a couple of ways to do this.
1. At the start of every script you want to execute in this manner
insert a cd command to the proper directory. (This is the way I did
it.)

2. Before issuing the qprt command export a variable (eg. apwd)
containing the directory you want. Then in the backend cd $apwd before
executing the spooled command.

Whatever method you choose remember these jobs execute as the user
that executed qprt.

--
Message sent VIA Followup and E-Mail --

--
Dell Coleman

 
 
 

batch job help

Post by bo » Fri, 02 Feb 2001 16:52:10


blame IBM, if you look on the website for docos on backends you find
a nice batch spooling system for compiling apps...

very wierd

brett


says...

> How come you are using this mechanism instead of the classic 'batch' facility?


> > Yes, yes, yes.  This is exactly what I did and I have tested running
> > jobs sequentially using the queue I created "script" with the backend
> > = /usr/bin/ksh.

> > I have found out though that the initial directory that the script will
> > start running is in /var/spool/lpd/qdir.  Is there a way to configure
> > the behavior of ksh so that the initial working directory is the
> > current directory where the "qprt -Pscript myproc" was envoked?

> > Thanks for all your help.

> > Carlos




> > says...
> > > > Thanks, Smurfy for your reply.

> > > > How does a backend created?

> > > > Can someone point me to where I can find what are the requirements
> > of a
> > > > backend??

> > > > Thanks.

> > > > Carlos

> > > A backend is just an ordinary program or shell script. Here an
> > example
> > > of a shell queue. It will execute the submitted scripts sequentially.

> > > ----- /etc/qconfig ----
> > > bsh:
> > >   discipline = fcfs
> > >   device = bshdev
> > > bshdev:
> > >   backend = /usr/bin/ksh
> > > -------------------------

> > > To run a shell procedure called myproc using this batch queue, enter:

> > > $ qprt -Pbsh myproc

 
 
 

batch job help

Post by Carlo » Sat, 03 Feb 2001 02:16:03


The batch actually uses the "at -q b -m now .......".  I have
experimented on this and I have a script that just sleeps for 40
seconds.  I submitted this script using the batch 5 times and the 5
jobs started simultaneously.  I don't want this to happen in most of
our batch jobs.  I want them to run in a sequential manner, which I can
be accomplished using the queueing mechanism of the printing subsystem.

Plus the "at -l" can only show jobs that are waiting to run in a future
time.  Jobs that are running cannot be listed.  You have to do the ps -
ef command which lists everything, unless of course you pipe it to
grep, etc., etc...



> How come you are using this mechanism instead of the classic 'batch'
facility?


> > Yes, yes, yes.  This is exactly what I did and I have tested running
> > jobs sequentially using the queue I created "script" with the
backend
> > = /usr/bin/ksh.

> > I have found out though that the initial directory that the script
will
> > start running is in /var/spool/lpd/qdir.  Is there a way to
configure
> > the behavior of ksh so that the initial working directory is the
> > current directory where the "qprt -Pscript myproc" was envoked?

> > Thanks for all your help.

> > Carlos




> > says...
> > > > Thanks, Smurfy for your reply.

> > > > How does a backend created?

> > > > Can someone point me to where I can find what are the
requirements
> > of a
> > > > backend??

> > > > Thanks.

> > > > Carlos

> > > A backend is just an ordinary program or shell script. Here an
> > example
> > > of a shell queue. It will execute the submitted scripts
sequentially.

> > > ----- /etc/qconfig ----
> > > bsh:
> > >   discipline = fcfs
> > >   device = bshdev
> > > bshdev:
> > >   backend = /usr/bin/ksh
> > > -------------------------

> > > To run a shell procedure called myproc using this batch queue,
enter:

> > > $ qprt -Pbsh myproc

> > > Here some links into the AIX docs

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/printrgd/prt_
Quote:> > > backend_overview.htm

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/qconfi

- Show quoted text -

> > > g.htm

> > --
> > --
> > Have a nice day.

> > Sent via Deja.com
> > http://www.deja.com/

> --
> Norman Levin


home of
> greatest power, the village idiot will come forth to be acclaimed the
> leader.'"

--
Have a nice day.

Sent via Deja.com
http://www.deja.com/

 
 
 

batch job help

Post by Norman Levi » Mon, 05 Feb 2001 09:28:13



> The batch actually uses the "at -q b -m now .......".  I have
> experimented on this and I have a script that just sleeps for 40
> seconds.  I submitted this script using the batch 5 times and the 5
> jobs started simultaneously.  I don't want this to happen in most of
> our batch jobs.  I want them to run in a sequential manner, which I can
> be accomplished using the queueing mechanism of the printing subsystem.

*** batch is controlled by the "queuedefs" file.  Think it is in /var/adm/cron
Look at the comments.  At the bottom, you will find a single line to control batch
but it is commented out.  Even when uncommented, it will let 100 jobs run at a time.
Change the "j" field to 1 to only allow 1 job at a time.

> Plus the "at -l" can only show jobs that are waiting to run in a future
> time.  Jobs that are running cannot be listed.  You have to do the ps -
> ef command which lists everything, unless of course you pipe it to
> grep, etc., etc...



> > How come you are using this mechanism instead of the classic 'batch'
> facility?


> > > Yes, yes, yes.  This is exactly what I did and I have tested running
> > > jobs sequentially using the queue I created "script" with the
> backend
> > > = /usr/bin/ksh.

> > > I have found out though that the initial directory that the script
> will
> > > start running is in /var/spool/lpd/qdir.  Is there a way to
> configure
> > > the behavior of ksh so that the initial working directory is the
> > > current directory where the "qprt -Pscript myproc" was envoked?

> > > Thanks for all your help.

> > > Carlos




> > > says...
> > > > > Thanks, Smurfy for your reply.

> > > > > How does a backend created?

> > > > > Can someone point me to where I can find what are the
> requirements
> > > of a
> > > > > backend??

> > > > > Thanks.

> > > > > Carlos

> > > > A backend is just an ordinary program or shell script. Here an
> > > example
> > > > of a shell queue. It will execute the submitted scripts
> sequentially.

> > > > ----- /etc/qconfig ----
> > > > bsh:
> > > >   discipline = fcfs
> > > >   device = bshdev
> > > > bshdev:
> > > >   backend = /usr/bin/ksh
> > > > -------------------------

> > > > To run a shell procedure called myproc using this batch queue,
> enter:

> > > > $ qprt -Pbsh myproc

> > > > Here some links into the AIX docs

> http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/printrgd/prt_
> > > > backend_overview.htm

> http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/qconfi

--
Norman Levin

greatest power, the village idiot will come forth to be acclaimed the
leader.'"

 
 
 

batch job help

Post by Jose Pina Coelh » Fri, 09 Feb 2001 06:43:14



> blame IBM, if you look on the website for docos on backends you find
> a nice batch spooling system for compiling apps...

> very wierd

Not at all.  It was made so that you could run a development
environment on a machine with 8M of RAM.  That way, the compiles
become sequential and the machine won't trash.

--
Doing AIX support was the most monty-pythonesque
activity available at the time.

 
 
 

batch job help

Post by Lizzi » Wed, 14 Feb 2001 21:34:56


Can always take the queue down, stuff a bunch of scripts in there in the
correct order, then use cron to fire up the queue at a specific time. They
will run one at a time, and you can even fire up another loaded queue from
a script in queue 1, like if you don't want to start running reports until
a update finishes. Put the reports in their own queue and start the queue
from tail end of the script you want finished before they run.
If you need much more than this I'd probably recommend looking at real
scheduling software, but for smallish stuff this works fine.

Liz



Quote:> The batch actually uses the "at -q b -m now .......".  I have
> experimented on this and I have a script that just sleeps for 40
> seconds.  I submitted this script using the batch 5 times and the 5
> jobs started simultaneously.  I don't want this to happen in most of
> our batch jobs.  I want them to run in a sequential manner, which I can
> be accomplished using the queueing mechanism of the printing subsystem.