concurrency in sh

concurrency in sh

Post by Dave Verne » Tue, 04 Oct 1994 22:43:08



OK.

I have a program which starts writing information to a file.
Multiple instances of this program must be able to run,
and it must be ensured that no two instances write to the same
file.

To accomplish this, I have the program write to a lock file
before it enters the "critical section". Unfortunately, it
seems if the two identical instances are started within about
.25 - 75 second apart, both will enter the critical section,
and both will write to the same file. Since one goes slower,
it overwrites what the first instance wrote, thus creating
havoc.

Is file creation an atomic operation?? Any ideas how to make
this scheme fool-proof?? And still be implemented in sh?

Thanks

Dave

--

           DEFEND THE EAST COAST FISHERY FROM FOREIGN OVERFISHING!
             /* DISCLAIMER: MyViews != ThoseViewsOf(*UW,&COGNOS) */
This article dedicated to Canada's greatest folk artist, Stan Rogers, R.I.P.'83
    VP, Early-Starters-Yet-All-Nighters-Yet-Creamed-By-CS-Assignments Society

        "Data integrity is like virtue -- it is lost only once."
                        --      Mark Terribile from "Practical C++"

 
 
 

concurrency in sh

Post by DFRusse » Wed, 05 Oct 1994 21:40:18


|> OK.
|>
|> I have a program which starts writing information to a file.
|> Multiple instances of this program must be able to run,
|> and it must be ensured that no two instances write to the same
|> file.
|>
|> To accomplish this, I have the program write to a lock file
|> before it enters the "critical section". Unfortunately, it
|> seems if the two identical instances are started within about
|> .25 - 75 second apart, both will enter the critical section,
|> and both will write to the same file. Since one goes slower,
|> it overwrites what the first instance wrote, thus creating
|> havoc.
|>
|> Is file creation an atomic operation?? Any ideas how to make

Usually [always?], no.

"link", if I remember correctly, is.

|> this scheme fool-proof?? And still be implemented in sh?
|>
|> Thanks
|>
|> Dave
|>
|>
|>
|>
|> --

|>            DEFEND THE EAST COAST FISHERY FROM FOREIGN OVERFISHING!
|>              /* DISCLAIMER: MyViews != ThoseViewsOf(*UW,&COGNOS) */
|> This article dedicated to Canada's greatest folk artist, Stan Rogers, R.I.P.'83
|>     VP, Early-Starters-Yet-All-Nighters-Yet-Creamed-By-CS-Assignments Society
|>
|>   "Data integrity is like virtue -- it is lost only once."
|>                   --      Mark Terribile from "Practical C++"

--
Disclaimer: I don't speak for Martin Marietta or the EPA.
----------------------------------------------------------

P.O. Box 14365, MD-4501-1B, Research Triangle Park, NC 27709  

 
 
 

concurrency in sh

Post by Dan Merc » Thu, 06 Oct 1994 06:23:49


: OK.

: I have a program which starts writing information to a file.
: Multiple instances of this program must be able to run,
: and it must be ensured that no two instances write to the same
: file.

: To accomplish this, I have the program write to a lock file
: before it enters the "critical section". Unfortunately, it
: seems if the two identical instances are started within about
: .25 - 75 second apart, both will enter the critical section,
: and both will write to the same file. Since one goes slower,
: it overwrites what the first instance wrote, thus creating
: havoc.

: Is file creation an atomic operation?? Any ideas how to make
: this scheme fool-proof?? And still be implemented in sh?
 mktemp(1)                                                         mktemp(1)

 NAME
      mktemp - make a name for a temporary file

 SYNOPSIS
      mktemp [-c] [-d directory_name] [-p prefix]

 DESCRIPTION
      mktemp makes a name that is suitable for use as the pathname of a
      temporary file, and writes that name to the standard output.  The name
      is chosen such that it does not duplicate the name of an existing
      file.  If the -c option is specified, a zero-length file is created
      with the generated name.

      The name generated by mktemp is the concatenation of a directory name,
      a slash (/), the value of the LOGNAME environment variable truncated
      to {NAME_MAX} - 6 characters, and the process ID of the invoking
      process.

      The directory name is chosen as follows:

           1.   If the -d option is specified, directory_name is used.

           2.   Otherwise, if the TMPDIR environment variable is set and a
                string that would yield a unique name can be obtained by
                using the value of that variable as a directory name, this
                value is used.

           3.   Otherwise, if a string that would yield a unique name can be
                obtained using /tmp as the directory, /tmp is used.

           4.   Otherwise, . (current directory) is used.

      If the -p option is specified, prefix is used instead of the value of
      the LOGNAME environment variable for name generation.

 RETURN VALUE
      mktemp returns zero on successful completion and non-zero if syntax,
      file access, or file creation errors were encountered or a unique
      pathname could not be generated.

 SEE ALSO
      mktemp(3C), umask(1).

 Hewlett-Packard Company            - 1 -     HP-UX Release 9.0: August 1992

: Thanks

: Dave

: --

:            DEFEND THE EAST COAST FISHERY FROM FOREIGN OVERFISHING!
:              /* DISCLAIMER: MyViews != ThoseViewsOf(*UW,&COGNOS) */
: This article dedicated to Canada's greatest folk artist, Stan Rogers, R.I.P.'83
:     VP, Early-Starters-Yet-All-Nighters-Yet-Creamed-By-CS-Assignments Society

:       "Data integrity is like virtue -- it is lost only once."
:                       --      Mark Terribile from "Practical C++"

--
Dan Mercer                                            ClearSystems, Inc

======================================================================
All opinions expressed are my own and do not reflect the opinions of
my employer or my employer's clients,  in particular 3M Company.
All advice or software offered or presented is provided As Is with no
warranty either expressed or implied.  Follow at your own risk.
Objects in the mirror are closer than they appear.

 
 
 

concurrency in sh

Post by Vladimir Stavits » Fri, 07 Oct 1994 06:00:07



|> |> OK.
|> |>
|> |> I have a program which starts writing information to a file.
|> |> Multiple instances of this program must be able to run,
|> |> and it must be ensured that no two instances write to the same
|> |> file.
|> |>
|> |> To accomplish this, I have the program write to a lock file
|> |> before it enters the "critical section". Unfortunately, it
|> |> seems if the two identical instances are started within about
|> |> .25 - 75 second apart, both will enter the critical section,
|> |> and both will write to the same file. Since one goes slower,
|> |> it overwrites what the first instance wrote, thus creating
|> |> havoc.
|> |>
|> |> Is file creation an atomic operation?? Any ideas how to make
|>
|> Usually [always?], no.
|>
|> "link", if I remember correctly, is.
|>
|> |> this scheme fool-proof?? And still be implemented in sh?
|> |>
|> |> Thanks
|> |>
|> |> Dave
|> |>
|> |>
|> |>
|> |>
|> |> --

|> |>            DEFEND THE EAST COAST FISHERY FROM FOREIGN OVERFISHING!
|> |>              /* DISCLAIMER: MyViews != ThoseViewsOf(*UW,&COGNOS) */
|> |> This article dedicated to Canada's greatest folk artist, Stan Rogers, R.I.P.'83
|> |>     VP, Early-Starters-Yet-All-Nighters-Yet-Creamed-By-CS-Assignments Society
|> |>
|> |>     "Data integrity is like virtue -- it is lost only once."
|> |>                     --      Mark Terribile from "Practical C++"
|>
|> --
|> Disclaimer: I don't speak for Martin Marietta or the EPA.
|> ----------------------------------------------------------

|> P.O. Box 14365, MD-4501-1B, Research Triangle Park, NC 27709  

link is considered to be 'atomic', but it is not nfs-safe to the best of
my knowledge; a while ago i implemented some short C code which uses nfs-safe
lockf() to achieve the functionality you need; the result is a simple executable
i use in my shell scripts to get synchronous access to whatever resources. If
you want me to send the source to you - let me know.

Hope this helps
--
Vladimir Stavitsky

 
 
 

concurrency in sh

Post by Peter Fu » Sat, 08 Oct 1994 16:34:11



Quote:> I have a program which starts writing information to a file.
> Multiple instances of this program must be able to run,
> and it must be ensured that no two instances write to the same
> file.
> To accomplish this, I have the program write to a lock file
> before it enters the "critical section". Unfortunately, it

[...]


Quote:>  mktemp(1)                                                         mktemp(1)
>  NAME
>       mktemp - make a name for a temporary file

I don't think, that this would solve the original posters problem.

In the past I allowed '/etc/link' to be run by normal users (chmod 775).
And than I used the following '/bin/sh'-code.  All I can say is, that it
seems to work (for me and our customers) :

    # mutual exclusion in /bin/sh :
    LOCKDIR=/tmp
    LOCKFILE=${LOCKDIR}/LCK..MyUniqueName
    TMPLOCKFILE=${LOCKDIR}/LCK..$$
    test -x /etc/link || {
         echo "Please enter   chmod 775 /etc/link   as root" ; exit 1
    }
    if test -f $LOCKFILE
    then if kill -0 `cat $LOCKFILE`
         then : there is already another prcoess in critical section :
              exit 0
         else : zombie LCK-file of a vanished process, ignore it :
              rm -f $LOCKFILE
         fi
    fi
    echo $$ > $TMPLOCKFILE
    if /etc/link $TMPLOCKFILE $LOCKFILE
    then rm -f $TMPLOCKFILE
         # -- insert your code (critical section) here --
         rm $LOCKFILE
    else : Another process has entered the critical section at the same time:
         rm -f $TMPLOCKFILE
         exit 0
    fi

I would love to see a more elegant or cleaner solution for this task.
I seeking for clean way to solve this in '/bin/sh' without having
to write an additional new tool in C.
Plese note, that 'chmod 775 /etc/link' might introduce a serious
security hole.  But this was not an issue in our applications, since
the machines are not connected to the net and unwanted access is denied
through "traditional non-software methods" (closed shop).

Regards, Peter
-=-=-
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany
office: +49 421 3499934 (ArtCom GmbH, Schwachhauser Heerstr.78, D-28209 Bremen)

 
 
 

concurrency in sh

Post by Michael Haard » Sun, 09 Oct 1994 05:37:57



> I would love to see a more elegant or cleaner solution for this task.

Basically, this is the only portable way to do it.  As I was told, there
is a flaw when doing it on NFS mounted file systems.  Instead of
checking if ln was successful, it should be checked if the file has a
link count of two after to be really safe.

Quote:> I seeking for clean way to solve this in '/bin/sh' without having
> to write an additional new tool in C.

But if you are looking for a new tool in C :) lockfile from the procmail
package should do the job.  Using procmail for mail delivery is
recommended anyway, so it would not really be a new tool.

Quote:> Plese note, that 'chmod 775 /etc/link' might introduce a serious
> security hole.

I don't know exactly what /etc/link does.  If it is a link command which
lets root make links to directories as well, then I don't see how a
mortal user could abuse it.  Now if it was 4755, that would be something
else.

Michael
--
Twiggs and root are a wonderful tree (tm) Twiggs & root 1992 :-)

 
 
 

1. Execute a sh script under perl and sh: sh script; perl script?

I can execute the following perl scripts under either shell or Perl
====

        if 0;
[perl scripts ...]
====
i.e.
sh script
perl script

How do I execute a sh script under either shell or Perl
simililar to what shown above?

Thanks.
--
Michael Wang

http://www.mindspring.com/~mwang    

2. Screen print to system printer. Possible??

3. root's shell, /sbin/sh, /bin/sh and /usr/ksh

4. PPP-Where do i post to get help...

5. #!/bin/sh #!/usr/bin/sh can I do both for 2 diff machines

6. DECUS SPELL for Unix?

7. ASCII 128-255 in sh script: sh strips bit 7!!!

8. Of Minitel, Linux and IBM

9. sh var -> awk var -> sh var

10. sh Xinstall.sh -check error

11. Any difference between /sbin/sh and /bin/sh?

12. Making a cd with makecdfs.sh burncd.sh

13. Hiding non-sh features from sh