Read-only root filesystem during boot

Read-only root filesystem during boot

Post by Tore Lind » Fri, 31 May 2002 20:21:43



After a reboot a week ago the server would no longer boot normally. The
error messages indicated it was no longer possible to write to /var/...
It turned out / suddenly was read-only. We used a workaround (mount -o
rewrite,rw /) and were able to get the server up and running again, but this
is obviously not an acceptable long term solution. Any ideas?
The server is running Solaris 7.

Tore Linde

 
 
 

Read-only root filesystem during boot

Post by Tore Lind » Fri, 31 May 2002 22:03:18


vfstab entry for root:
/dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1
 no      -

any related /var/adm/messages messages:
It did not mount the /var file system. I therefore don't have the exact
messages, but the first error message said something like "unable to write
to /var/spool/lp"

We did run fsck and some minor errors were corrected.

You're right about the typo - it's supposed to be remount.

Tore



> #   After a reboot a week ago the server would no longer boot normally.
The
> #   error messages indicated it was no longer possible to write to
/var/...
> #   It turned out / suddenly was read-only. We used a workaround (mount -o
> #   rewrite,rw /) and were able to get the server up and running again,
but this
> #   is obviously not an acceptable long term solution. Any ideas?
> #   The server is running Solaris 7.

> Show us your vfstab entry for root.

> Show us any related /var/adm/messages messages (boot without the fix).

> Have you run fsck? If not, run it while / is RO.

>     Don't you mean, "mount -o remount,rw /"?


 
 
 

Read-only root filesystem during boot

Post by Tore Lind » Sat, 01 Jun 2002 22:14:28


(thanks so far - we're probably in very different time zones)

the whole vfstab:

#device         device          mount           FS      fsck    mount
mount
#to mount       to fsck         point           type    pass    at boot
options
#
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
/dev/dsk/c0t0d0s3       -       -       swap    -       no      -
/dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1
 no      -
/dev/dsk/c0t0d0s1       /dev/rdsk/c0t0d0s1      /var    ufs     1
 no      -
swap    -       /tmp    tmpfs   -       yes     -
#
/dev/dsk/c0t0d0s4       /dev/rdsk/c0t0d0s4      /export/ledig0  ufs     2
yes     logging
#
# Raid Lun 0:
/dev/dsk/c2t5d0s0       /dev/rdsk/c2t5d0s0      /export/stud20  ufs     2
yes     rq,logging
/dev/dsk/c2t5d0s1       /dev/rdsk/c2t5d0s1      /usr/local      ufs     2
yes     logging
/dev/dsk/c2t5d0s3       /dev/rdsk/c2t5d0s3      /usr/openv      ufs     2
yes     logging
/dev/dsk/c2t5d0s4       /dev/rdsk/c2t5d0s4      /export/stud21  ufs     2
yes     rq,logging
/dev/dsk/c2t5d0s5       /dev/rdsk/c2t5d0s5      /public         ufs     2
yes     logging
#
# Raid Lun 1:
/dev/dsk/c2t5d1s0       /dev/rdsk/c2t5d1s0      /export/stab10  ufs     2
yes     logging
/dev/dsk/c2t5d1s1       /dev/rdsk/c2t5d1s1      /export/stab11  ufs     2
yes     logging
/dev/dsk/c2t5d1s3       /dev/rdsk/c2t5d1s3      /export/stab12  ufs     2
yes     logging
/dev/dsk/c2t5d1s4       /dev/rdsk/c2t5d1s4      /export/stab13  ufs     2
yes     logging
/dev/dsk/c2t5d1s5       /dev/rdsk/c2t5d1s5      /export/stab14  ufs     2
yes     logging
#
# Raid Lun 2:
/dev/dsk/c2t5d2s1       /dev/rdsk/c2t5d2s1      /sybase/backup  ufs     2
yes     logging
/dev/dsk/c2t5d2s0       /dev/rdsk/c2t5d2s0      /sybase         ufs     2
yes     logging
/dev/dsk/c2t5d2s3       /dev/rdsk/c2t5d2s3      /export/stud22  ufs     2
yes     logging
/dev/dsk/c2t5d2s4       /dev/rdsk/c2t5d2s4      /drift  ufs     2       yes
logging
/dev/dsk/c2t5d2s5       /dev/rdsk/c2t5d2s5      /export/stud23  ufs     2
yes     logging
/dev/dsk/c2t5d2s6       /dev/rdsk/c2t5d2s6      /export/stud24  ufs     2
yes     logging
#
# Raid Lun 3:
#
elg:/opt        -       /opt            nfs     -       yes     bg
elg:/opt/STD    -       /STD            nfs     -       yes     bg
elg:/opt/frame  -       /opt/frame      nfs     -       yes     bg
mink:/var/mail  -       /var/mail       nfs     -       yes     bg
grevling:/pub   -       /pub            nfs     -       yes     bg

The way it works now is as follows:
At first there are a few error messages. Then I get the single user prompt
(and this is new, it usually boot to run level3). I log in as root and
execute some commands:

mount -o remount,rw /
umountall
umount /var
mount /var
umont /proc
mount /proc
umount /dev/fd
mount /dev/fd

I then edit /etc/mnttab and leave these entries:

  /
  /var
  /proc
  /dev/fd

Then I press ctrl-D and choose run level 3.

Where do I find out where the remount is stuck (I probably should know this
but I don't - Sorry!)?



> #   vfstab entry for root:
> #   /dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1
> #    no      -
> #
> #   any related /var/adm/messages messages:
> #   It did not mount the /var file system. I therefore don't have the
exact
> #   messages, but the first error message said something like "unable to
write
> #   to /var/spool/lp"
> #
> #   We did run fsck and some minor errors were corrected.

> Well, you've really broken up the root partition into a bunch of
> mounts - I just use /.

> Sorry, can I see the whole vfstab?

> Also, where is your remount stuck in the boot sequence,
> and can you add a little code before it (touch/echo success/failure)
> to say whether it is now mounting RW by itself.

> Watch the console during boot for error msgs.



- Show quoted text -


> #   vfstab entry for root:
> #   /dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1
> #    no      -
> #
> #   any related /var/adm/messages messages:
> #   It did not mount the /var file system. I therefore don't have the
exact
> #   messages, but the first error message said something like "unable to
write
> #   to /var/spool/lp"
> #
> #   We did run fsck and some minor errors were corrected.

> Well, you've really broken up the root partition into a bunch
> of mounts - I just use /.

> Sorry, can I see the whole vfstab?

> Also, where is your remount stuck in the boot sequence,
> and can you add a little code before it (touch/echo success/failure)
> to say whether it is now mounting RW by itself.

> Watch the console during boot for error msgs.

 
 
 

Read-only root filesystem during boot

Post by Kjetil Torgrim Homm » Wed, 05 Jun 2002 00:54:04



Quote:

>   >  /dev/dsk/c0t0d0s1    /dev/rdsk/c0t0d0s1   /var  ufs   1 no   -

>   I'm going to guess that the /var parameter for "mount at boot"
>   should be "yes", not "no."

no, it shouldn't.  /var and /usr are explicitly mounted.  the yes/no
is used by mount -a which comes later.

Quote:>   A general suggestion is to change the last "-" for /, /var to
>   "logging".  See 'man mount_ufs.' That should eliminate the need
>   for 'fsck'.

not really, but it should speed up the fsck.

--
Kjetil T.               your computer can help researchers understand how
                        proteins work.    http://folding.stanford.edu

 
 
 

Read-only root filesystem during boot

Post by Kjetil Torgrim Homm » Wed, 05 Jun 2002 01:03:02


[Tore Linde]:

Quote:

>   vfstab entry for root:
>   /dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1
>    no      -

I hope the line wrapping is caused by your news client.  (get a better
one, please.)

Quote:>   any related /var/adm/messages messages: It did not mount the /var
>   file system. I therefore don't have the exact messages, but the
>   first error message said something like "unable to write to
>   /var/spool/lp"

>   We did run fsck and some minor errors were corrected.

the relevant scripts are /etc/rcS.d/S30rootusr.sh and
/etc/rcS.d/S40standardmounts.sh.  run them manually (after it has
failed) and see what happens.  you may want to use sh -x to get more
diagnostic output.

--
Kjetil T.               your computer can help researchers understand how
                        proteins work.    http://folding.stanford.edu

 
 
 

Read-only root filesystem during boot

Post by Thomas Schu » Wed, 05 Jun 2002 02:36:05






>#  >
>#  >   >  /dev/dsk/c0t0d0s1    /dev/rdsk/c0t0d0s1   /var  ufs   1 no   -
>#  >  
>#  >   I'm going to guess that the /var parameter for "mount at boot"
>#  >   should be "yes", not "no."
>#  
>#   no, it shouldn't.  /var and /usr are explicitly mounted.  the yes/no
>#   is used by mount -a which comes later.
>#   the relevant scripts are /etc/rcS.d/S30rootusr.sh and
>#   /etc/rcS.d/S40standardmounts.sh.

>Sorry, I saw no "explicit mount" for /var anywhere in there.

/var is done in /etc/rcS.d/S70buildmnttab.sh (in Solaris 8 at least).
/etc/rcS.d/S40standardmounts.sh does mount /usr among other things.
In any case, /var should be 'no' for mount at boot.

Quote:>#  
>#  >   A general suggestion is to change the last "-" for /, /var to
>#  >   "logging".  See 'man mount_ufs.' That should eliminate the need
>#  >   for 'fsck'.
>#  
>#   not really, but it should speed up the fsck.

>Man fsck:

>    logging

>    This prevents file systems from becoming inconsistent,
>    therefore eliminating the need to run fsck.

--
Tom Schulz

 
 
 

Read-only root filesystem during boot

Post by Kjetil Torgrim Homm » Wed, 05 Jun 2002 07:38:51




>   >   not really, but it should speed up the fsck.

>   Man fsck:

>       logging

>       This prevents file systems from becoming inconsistent,
>       therefore eliminating the need to run fsck.

fsck needs to run nonetheless, to replay the journal.

--
Kjetil T.               your computer can help researchers understand how
                        proteins work.    http://folding.stanford.edu

 
 
 

Read-only root filesystem during boot

Post by Andrew Gabri » Wed, 05 Jun 2002 19:10:42




Quote:

> It sounds like other code does the flushing.

fsck momentarily mounts it and gets the ufs code in the kernel
to flush the log.

--
Andrew Gabriel
Consultant Software Engineer

 
 
 

1. '/dev/null: read-only filesystem' during boot

hi,

i'm running a pretty vanilla ( i think ) caldera 2.2 with kernel 2.2.14.  i
rebooted yesterday and things started hanging durring fsck.  booting into
single user gives:

        Loading kernel......
        /dev/null on /proc type proc (rw)
        Mounted /proc...
        Loading default modules...
        /etc/rd.c/rc.modules: /dev/null: Read-only file system
        Unmounted /proc...
        /etc/rc.d/rc: /var/log/rc-log: Read-Only file system
                above line repeated several times

when i then log in as root, mount gives:
        /dev/sda1 on / type ext2 (rw)
        /proc on /proc type proc (rw)

but when i try to touch anything, it tells me things are read only.  after
mount -n -o remount,rw /, mount's output is the same but now the filesystem is
normal (i can change files, etc).

by touch'ing /fastboot, i can get the system to startup sort of normally, but
loading the modules still runs into trouble, so a lot of things don't work.

any help would be much appreciated.

brady richards

2. SUID BITS

3. how to make root filesystem read-write after boot net ?

4. UDP Servers

5. How to boot in single user mode, the root filesystem mounted in read/WRITE mode

6. Creative Labs CD 2X Problems...

7. Wont boot after VFS: mounting root (ext2 filesystem) as read only

8. INFORMIX-SE download

9. Root filesystem read-only, NOT fixed by section 10.11 in the FAQ

10. Read-Only root Filesystem Problem

11. root filesystem read-only

12. Mounting root filesystem read-only

13. Boot: bad file descriptor, read-only filesystem?