Very Computer

Hi,

I don't know if anyone can help with this weird problem but, as of last
Sunday, my
inetd won't run.
It gets started OK by one of the xxinetsvc scripts in the /etc/rc.xx
directories, then
disappears into a black hole.
Any attempt to start it by hand brings complaints about
'bind: Address already in use'
for almost every entry in /etc/services, and results in a loss of all
network functions.
This is a Solaris 2.7 system, and has not (apparently) been tinkered
with, although
it went down ungracefully, due to a power outage on Saturday.

If anyone has any clues, could you please copy to my email address, as
my ISP's
support of this newsgroup can be quite random.

--
Best regards,
Mark
 --------------------------------------------------------------------
 Mark Sitkowski C.Eng M.I.E.E           ~{D|?To?~}!
 Design Simulation Systems Ltd  ABN: 55-057-527-401
 14 Loddon Street
 Box Hill North, Victoria 3129
 --------------------------------------------------------------------
 Phone: (613-9) 897-1375                ~{?????~}!

 Mobile:     0413-890-777
 Web http://www.angelfire.com/de/designsim
     http://designsim.freeyellow.com
 --------------------------------------------------------------------

Try invoking via truss(1), with output to a log file, to see where it's
failing.

Quote:>Hi,

>I don't know if anyone can help with this weird problem but, as of last
>Sunday, my
>inetd won't run.
>It gets started OK by one of the xxinetsvc scripts in the /etc/rc.xx
>directories, then
>disappears into a black hole.
>Any attempt to start it by hand brings complaints about
>'bind: Address already in use'
>for almost every entry in /etc/services, and results in a loss of all
>network functions.

That usually means that inetd is already running, as the first inetd has
the ports bound.  So are you sure that inetd is dying?

Do you have "lsof" installed on the system?  If so, try using it to see
what process has those ports bound.

--

Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

Inetd is definitely dead, but all net services appear to run normally. When I run
lsof, it says:
csh#[2] lsof
csh#[3]
(maybe it's broken, too).

I tried truss -f, since checking the return value of the original inetd invocation
was
zero, and found that the child process ate something it didn't like, and fell
asleep (file enclosed). Each time it woke up, it complained about the ports again.

I've been looking after Suns for 12 years now, and I've never seen this one....
Has anyone else?

Mark

--
Best regards,
Mark
 --------------------------------------------------------------------
 Mark Sitkowski C.Eng M.I.E.E           ~{D|?To?~}!
 Design Simulation Systems Ltd  ABN: 55-057-527-401
 14 Loddon Street
 Box Hill North, Victoria 3129
 --------------------------------------------------------------------
 Phone: (613-9) 897-1375                ~{?????~}!
 Fax:   (613-9) 897-1376   E-mail: xma...@comcen.com.au
 Mobile:     0413-890-777
 Web http://www.angelfire.com/de/designsim
     http://designsim.freeyellow.com
 --------------------------------------------------------------------

[ truss_out 57K ]
530:    fork()          (returning as child ...)        = 529
530:    close(0)                                        = 0
530:    close(1)                                        = 0
530:    close(2)                                        = 0
530:    open("/", O_RDONLY)                             = 0
530:    fcntl(0, F_DUP2FD, 0x00000001)                  = 1
530:    fcntl(0, F_DUP2FD, 0x00000002)                  = 2
530:    setsid()                                        = 530
530:    brk(0x00028D38)                                 = 0
530:    brk(0x0002AD38)                                 = 0
530:    fstat(-1, 0xEFFFF278)                           Err#9 EBADF
530:    sigfillset(0xEF67A5E0)                          = 0
530:    sigaction(SIGALRM, 0xEFFFF220, 0xEFFFF2C0)      = 0
530:    sigprocmask(SIG_UNBLOCK, 0xEFFFF2F0, 0xEFFFF2E0) = 0
530:    auditsys(BSM_AUDITCTL, 0x00000004)              Err#22 EINVAL
530:    sigprocmask(0, 0x00000000, 0xEFFFF290)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF290, 0xEFFFF280)  = 0
530:    open("/etc/inetd.conf", O_RDONLY)               = 3
530:    fstat64(3, 0xEFFFECE0)                          = 0
530:    brk(0x0002AD38)                                 = 0
530:    brk(0x0002CD38)                                 = 0
530:    ioctl(3, TCGETA, 0xEFFFEC6C)                    Err#25 ENOTTY
530:    read(3, " #\n # i d e n t\t " @ (".., 8192)     = 4282
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF230, 0xEFFFF220)  = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_SETMASK, 0xEFFFF230, 0xEFFFF220) = 0
530:    open("/etc/netconfig", O_RDONLY)                = 4
530:    fstat64(4, 0xEFFFEBF8)                          = 0
530:    brk(0x0002CD38)                                 = 0
530:    brk(0x0002ED38)                                 = 0
530:    ioctl(4, TCGETA, 0xEFFFEB84)                    Err#25 ENOTTY
530:    read(4, " #\n #   T h e   " N e t".., 8192)     = 1064
530:    read(4, 0x0002BE54, 8192)                       = 0
530:    llseek(4, 0, SEEK_CUR)                          = 1064
530:    llseek(4, 0, SEEK_SET)                          = 0
530:    read(4, " #\n #   T h e   " N e t".., 8192)     = 1064
530:    brk(0x0002ED38)                                 = 0
530:    brk(0x00030D38)                                 = 0
530:    read(4, 0x0002BE54, 8192)                       = 0
530:    llseek(4, 0, SEEK_CUR)                          = 1064

530:    close(4)                                        = 0
530:    open("/etc/nsswitch.conf", O_RDONLY)            = 4
530:    fstat64(4, 0xEFFFEE70)                          = 0
530:    brk(0x00030D38)                                 = 0
530:    brk(0x00032D38)                                 = 0
530:    ioctl(4, TCGETA, 0xEFFFEDFC)                    Err#25 ENOTTY
530:    read(4, " #\n #   / e t c / n s s".., 8192)     = 690
530:    read(4, 0x0002F024, 8192)                       = 0
530:    llseek(4, 0, SEEK_CUR)                          = 690
530:    close(4)                                        = 0
530:    open("/usr/lib/nss_files.so.1", O_RDONLY)       = 4
530:    fstat(4, 0xEFFFEDDC)                            = 0
530:    mmap(0x00000000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE, 4, 0) = 0xEF770000
530:    mmap(0x00000000, 81920, PROT_READ|PROT_EXEC, MAP_PRIVATE, 4, 0) = 0xEF580000
530:    mmap(0xEF593000, 3372, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 4, 12288) = 0xEF593000
530:    munmap(0xEF584000, 61440)                       = 0
530:    memcntl(0xEF580000, 5584, MC_ADVISE, 0x0003, 0, 0) = 0
530:    close(4)                                        = 0
530:    open("/dev/zero", O_RDONLY)                     = 4
530:    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 4, 0) = 0xEF570000
530:    close(4)                                        = 0
530:    munmap(0xEF770000, 4096)                        = 0
530:    open("/etc/services", O_RDONLY)                 = 4
530:    fstat64(4, 0xEFFFEEE0)                          = 0
530:    ioctl(4, TCGETA, 0xEFFFEE6C)                    Err#25 ENOTTY
530:    read(4, " # i d e n t\t " @ ( # )".., 8192)     = 3563
530:    llseek(4, 0xFFFFFFFFFFFFF36F, SEEK_CUR)         = 346
530:    close(4)                                        = 0
530:    so_socket(2, 2, 0, "", 1)                       = 4
530:    setsockopt(4, 65535, 4, 0xEFFFF29C, 4, 1)       = 0
530:    bind(4, 0x0002BA00, 16, 3)                      Err#125 EADDRINUSE
530:    fstat(-1, 0xEFFFE578)                           Err#9 EBADF
530:    open("/dev/conslog", O_WRONLY)                  = 5
530:    fcntl(5, F_SETFD, 0x00000001)                   = 0
530:    fstat(5, 0xEFFFE578)                            = 0
530:    fstat(5, 0xEFFFEFD8)                            = 0
530:    time()                                          = 1027940514
530:    open("/usr/share/lib/zoneinfo/Australia/Tasmania", O_RDONLY) = 6
530:    read(6, "\0\0\0\0\0\0\0\0\0\0\0\0".., 8192)     = 825
530:    close(6)                                        = 0
530:    getpid()                                        = 530 [529]
530:    putmsg(5, 0xEFFFE690, 0xEFFFE684, 0)            = 0
530:    open("/etc/.syslog_door", O_RDONLY)             = 6
530:    door_info(6, 0xEFFFE5C8)                        = 0
530:    getpid()                                        = 530 [529]
530:    door_call(6, 0xEFFFE5B0)                        = 0
530:    close(6)                                        = 0
530:    close(4)                                        = 0
530:    alarm(600)                                      = 0
529:    llseek(0, 0, SEEK_CUR)                          = 8163
529:    _exit(0)
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF230, 0xEFFFF220)  = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_SETMASK, 0xEFFFF230, 0xEFFFF220) = 0
530:    open("/etc/services", O_RDONLY)                 = 4
530:    fstat64(4, 0xEFFFEEE0)                          = 0
530:    ioctl(4, TCGETA, 0xEFFFEE6C)                    Err#25 ENOTTY
530:    read(4, " # i d e n t\t " @ ( # )".., 8192)     = 3563
530:    llseek(4, 0xFFFFFFFFFFFFF37E, SEEK_CUR)         = 361
530:    close(4)                                        = 0
530:    so_socket(2, 2, 0, "", 1)                       = 4
530:    setsockopt(4, 65535, 4, 0xEFFFF29C, 4, 1)       = 0
530:    bind(4, 0x0002C6D0, 16, 3)                      Err#125 EADDRINUSE
530:    fstat(5, 0xEFFFEFD8)                            = 0
530:    time()                                          = 1027940514
530:    getpid()                                        = 530 [1]
530:    putmsg(5, 0xEFFFE690, 0xEFFFE684, 0)            = 0
530:    open("/etc/.syslog_door", O_RDONLY)             = 6
530:    door_info(6, 0xEFFFE5C8)                        = 0
530:    getpid()                                        = 530 [1]
530:    door_call(6, 0xEFFFE5B0)                        = 0
530:    close(6)                                        = 0
530:    close(4)                                        = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF230, 0xEFFFF220)  = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_SETMASK, 0xEFFFF230, 0xEFFFF220) = 0
530:    open("/etc/services", O_RDONLY)                 = 4
530:    fstat64(4, 0xEFFFEEE0)                          = 0
530:    ioctl(4, TCGETA, 0xEFFFEE6C)                    Err#25 ENOTTY
530:    read(4, " # i d e n t\t " @ ( # )".., 8192)     = 3563
530:    llseek(4, 0xFFFFFFFFFFFFF3DA, SEEK_CUR)         = 453
530:    close(4)                                        = 0
530:    so_socket(2, 1, 0, "", 1)                       = 4
530:    setsockopt(4, 65535, 4, 0xEFFFF29C, 4, 1)       = 0
530:    bind(4, 0x0002C758, 16, 3)                      = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF230, 0xEFFFF220)  = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_SETMASK, 0xEFFFF230, 0xEFFFF220) = 0
530:    open("/etc/services", O_RDONLY)                 = 6
530:    fstat64(6, 0xEFFFEEE0)                          = 0
530:    ioctl(6, TCGETA, 0xEFFFEE6C)                    Err#25 ENOTTY
530:    read(6, " # i d e n t\t " @ ( # )".., 8192)     = 3563
530:    llseek(6, 0xFFFFFFFFFFFFF92D, SEEK_CUR)         = 1816
530:    close(6)                                        = 0
530:    so_socket(2, 2, 0, "", 1)                       = 6
530:    setsockopt(6, 65535, 4, 0xEFFFF29C, 4, 1)       = 0
530:    bind(6, 0x0002C7E0, 16, 3)                      Err#125 EADDRINUSE
530:    fstat(5, 0xEFFFEFD8)                            = 0
530:    time()                                          = 1027940514
530:    getpid()                                        = 530 [1]
530:    putmsg(5, 0xEFFFE690, 0xEFFFE684, 0)            = 0
530:    open("/etc/.syslog_door", O_RDONLY)             = 7
530:    door_info(7, 0xEFFFE5C8)                        = 0
530:    getpid()                                        = 530 [1]
530:    door_call(7, 0xEFFFE5B0)                        = 0
530:    close(7)                                        = 0
530:    close(6)                                        = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_BLOCK, 0xEFFFF230, 0xEFFFF220)  = 0
530:    sigprocmask(0, 0x00000000, 0xEFFFF230)          = 0
530:    sigprocmask(SIG_SETMASK, 0xEFFFF230, 0xEFFFF220) = 0
530:    open("/etc/services", O_RDONLY)                 = 6
530:    fstat64(6, 0xEFFFEEE0)                          = 0
530:    ioctl(6, TCGETA, 0xEFFFEE6C)                    Err#25 ENOTTY
530:    read(6, " # i d e n t\t " @ ( # )".., 8192)     = 3563
530:    llseek(6, 0xFFFFFFFFFFFFF904, SEEK_CUR)         = 1775
530:  
...

read more »

Quote:>Inetd is definitely dead, but all net services appear to run normally. When I run

If all net services run normally, inetd must be alive, since it's involved
in many of them.  If it's not showing up in ps, the problem is probably
with ps, not inetd.

Quote:>lsof, it says:
>csh#[2] lsof
>csh#[3]
>(maybe it's broken, too).

Both ps and inetd have to peek at lots of process data structures.  If
they're both having problems, I suspect that this interface is where the
problem is.  They probably make heavy use of the /proc filesystem, so maybe
this is broken.

IIRC, you said this started after you had a crash or power failure, so
probably something got corrupted as a result.  You may need to reinstall
the OS.

--

Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.

One of the users runs the old calendar manager (cm) under fvwm. cm issues a
warning on login that inetd may not be running, and ps shows nothing.
I compiled and installed the latest version of lsof (which works) but makes no
mention
of inetd.
I've discovered that, in fact, neither telnet nor ftp work (into the machine), at
login
time, but work out of the machine - as you'd expect.
If an attempt is made to start inetd by hand, telnet and ftp then work in both
directions, but there is still no sign of inetd in the process table.

csh#[9] inetd -s -t&
[1] 1629
csh#[10]
[1]    Done                 inetd -s -t
csh#[10]
csh#[10] ps -edaf|grep inetd
csh#[11]
At this time, there are no complaints about ports being in use.
The next attempt to start it, gives this:
# Aug  1 21:29:59 powerstar inetd[1649]: ftp/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: telnet/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: shell/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: login/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: exec/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: uucp/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: systat/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: netstat/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: time/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: echo/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: discard/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: daytime/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: chargen/tcp: bind: Address already in use
Aug  1 21:29:59 powerstar inetd[1649]: fs/tcp: bind: Address already in use

-- still no inetd.

Quote:

> Both ps and inetd have to peek at lots of process data structures.  If
> they're both having problems, I suspect that this interface is where the
> problem is.  They probably make heavy use of the /proc filesystem, so maybe
> this is broken.

As far as I can make out, this gets repopulated with each reboot, and the contents
seem to match those on another SS20.

Quote:

> IIRC, you said this started after you had a crash or power failure, so
> probably something got corrupted as a result.  You may need to reinstall
> the OS.

It's a point of honour not to do that. Only BillyWare needs the OS reinstalled to fix

bugs. However, if desperation sets in, who knows..?

--
Best regards,
Mark
 --------------------------------------------------------------------
 Mark Sitkowski C.Eng M.I.E.E           ~{D|?To?~}!
 Design Simulation Systems Ltd  ABN: 55-057-527-401
 14 Loddon Street
 Box Hill North, Victoria 3129
 --------------------------------------------------------------------
 Phone: (613-9) 897-1375                ~{?????~}!

 Mobile:     0413-890-777
 Web http://www.angelfire.com/de/designsim
     http://designsim.freeyellow.com
 --------------------------------------------------------------------

What is the lsof HASSECURITY setting?  Check it in `lsof -h` output.
If `lsof -h` reports "Only root can list all files;..." then lsof
probably won't find inetd.

Since inetd complains that the telnet listening port is in use when
you try to start it -- i.e., you report:

Quote:> ...
>csh#[9] inetd -s -t&
>[1] 1629
>csh#[10]
>[1]    Done                 inetd -s -t
>csh#[10]
>...
>The next attempt to start it, gives this:
>Aug  1 21:29:59 powerstar inetd[1649]: ftp/tcp: bind: Address already in use

Try using an lsof that will list all open files (e.g., run it as
root) to see what process has the ftp TCP service in use with:

        # lsof -i TCP:ftp

Vic Abell, lsof author

You could try using the SunOS/BSD compatible version of ps in /usr/ucb.
You'd invoke with something like:

        /usr/ucb/ps auxww | grep inet

If you have a recent Solaris version, there's also prstat.

I'd start wondering if either something was corrupt on the disk, or maybe
you've been hacked and someone installed a modified /usr/bin/ps.

In any case, I hope you have backups...
--
Jeff Wieland (who sits across the hall from Vic)

Fact is, it confirms that inetd is dead:

csh%[10] /usr/ucb/ps auxww | grep inet
marks     3481  0.2  0.6  928  656 pts/7    S 22:51:56  0:00 grep inet

I've since discovered that the machine was attacked by exploiting the security

hole in Sun's libnsl.so (see CERT Advisory CA-2002-25 & Sun Alert 46122).
The side effects of the rubbish that was planted somewhere (I think a
modload'ed
driver or something) are that neither patchadd nor pkgadd will work, so I
can't
get rid of it.
I'm reminded strongly of a similar weakness in sendmail, resulting in the
infamous
'Internet Worm' incident.

Any ideas?

--
Best regards,
Mark
 --------------------------------------------------------------------
 Mark Sitkowski C.Eng M.I.E.E           ~{D|?To?~}!
 Design Simulation Systems Ltd  ABN: 55-057-527-401
 14 Loddon Street
 Box Hill North, Victoria 3129
 --------------------------------------------------------------------
 Phone: (613-9) 897-1375                ~{?????~}!

 Mobile:     0413-890-777
 Web http://www.angelfire.com/de/designsim
     http://designsim.freeyellow.com
 --------------------------------------------------------------------