Postgres server output logfile

Postgres server output logfile

Post by Mintoo La » Fri, 31 Jan 2003 13:36:23



--0-1463806419-1043901353=:39633
Content-Type: text/plain; charset=us-ascii

Hi Everybody,

I am new to postgresql. I started my postgres using the command

postmaster -D /usr/local/pgsql/data > logfile 2>&1 &

Now after running it for some time I noticed that the size of this logfile has become very large. Is this logfile used to store any important information used by the database for recovery in case of crash? If no,is there any way I can specify a different log file withouting stopping the server ?

Any help is appreciated.

Thanks,

Tarun

---------------------------------
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now
--0-1463806419-1043901353=:39633
Content-Type: text/html; charset=us-ascii

<P>Hi Everybody,</P>
<P>I am new to postgresql. I started my postgres using the command</P>
<P>postmaster -D /usr/local/pgsql/data > <STRONG>logfile</STRONG> 2>&1 &</P>
<P>Now after running it for some time I noticed that the size of this logfile has become very large. Is this logfile used to store any important information used by the database for recovery in case of crash? If no,is there any way I can specify a different log file withouting stopping the server ?</P>
<P>Any help is appreciated.</P>
<P>Thanks,</P>
<P>Tarun</P>
<P>?</P><p><br><hr size=1>Do you Yahoo!?<br>
<a href="http://rd.yahoo.com/mail/mailsig/*http://mailplus.yahoo.com">Yahoo! Mail Plus</a> - Powerful. Affordable. <a href="http://rd.yahoo.com/mail/mailsig/*http://mailplus.yahoo.com">Sign up now</a>
--0-1463806419-1043901353=:39633--

 
 
 

Postgres server output logfile

Post by Tom La » Fri, 31 Jan 2003 14:00:56



> I am new to postgresql. I started my postgres using the command
> postmaster -D /usr/local/pgsql/data > logfile 2>&1 &
> Now after running it for some time I noticed that the size of this
> logfile has become very large. Is this logfile used to store any
> important information used by the database for recovery in case of
> crash?

No, it's only messages for human consumption.

Quote:> If no,is there any way I can specify a different log file
> withouting stopping the server ?

Unfortunately not.  The recommended procedure for production servers is
not to send the postmaster's stdout/stderr directly to a disk file, but
to pipe it into some script that rotates the output.  There's a usable
script in the Apache distribution, or you can roll your own with little
effort.  See the PG admin guide for more discussion (in the routine-
maintenance chapter).

                        regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

 
 
 

Postgres server output logfile

Post by Giles Le » Mon, 03 Feb 2003 15:51:12


------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"


> Unfortunately not.  The recommended procedure for production servers is
> not to send the postmaster's stdout/stderr directly to a disk file, but
> to pipe it into some script that rotates the output.  There's a usable
> script in the Apache distribution, or you can roll your own with little
> effort. ...

There was some interest in such logging programs a while back, and I
wrote this one:

    ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

I've attached a message I sent out when I packaged that up.  I had no
time back then (2001, yikes) and didn't receive any comments or bug
reports.

If there is interest now, then I'll update the documentation, format
the code per the PostgreSQL coding standards and offer it again.

A patch to the postmaster so that it starts the log program and can
re-start it if someone kills it would improve robustness even further,
but 'kill -9' on the postmaster is bad, and so is 'kill -9' on the log
program. :-)

Regards,

Giles

------- =_aaaaaaaaaa0
Content-Type: message/rfc822



Subject: Re: [HACKERS] Log rotation?


Date: Sat, 08 Sep 2001 07:21:19 +1000


Hi Peter,

Quote:> I've been playing with a little program I wrote whose sole purpose is to
> write its stdin to a file and close and reopen that file when it receives
> a signal.  I figured this could work well when integrated transparently
> into pg_ctl.

> So, is log rotation a concern?  Is this a reasonable solution?  Other
> ideas?

There was a discussion of this over a year ago.  After I contributed
to the discussion Tom Lane suggested I write something, and in the
tradition of software development I did so but never quite finished it
... I got to the point that it works for me, then got distracted
before completing a test suite.

You may well prefer your own code, but this one supports rotation on
size and/or time basis as well as on receipt of SIGHUP, and places a
timestamp on each line written. It's also pretty careful about errors,
which is one of the things that was disliked about the Apache program
last time it was discussed.

I am happy to contribute the code using the standard PostgreSQL
license if it's wanted.  (If anyone wants it under a BSD license or
GPL for another purpose that's fine too.)

I use the code on HP-UX, and developed it on NetBSD.  There shouldn't
be too many portability problems lurking, other than the usual hassles
of what % escape to use in printf() for off_t.  I doubt anyone wants
log files larger than a couple of GB anyway? :-)

ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

Quote:> (No Grand Unified Logging Solutions please.  And no, "use syslog" doesn't
> count.)

<grin>

One improvement I suggest is that the postmaster be taught to start
(and restart if necessary) the log program.  This avoids fragile
startup scripts and also avoids taking down PostgreSQL if someone
sends the wrong signal to the log program.

Cheers,

Giles

------- =_aaaaaaaaaa0
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

------- =_aaaaaaaaaa0--

 
 
 

Postgres server output logfile

Post by Giles Le » Mon, 03 Feb 2003 18:24:33


------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"

[ possible duplicate -- sorry folks if so! ]


> Unfortunately not.  The recommended procedure for production servers is
> not to send the postmaster's stdout/stderr directly to a disk file, but
> to pipe it into some script that rotates the output.  There's a usable
> script in the Apache distribution, or you can roll your own with little
> effort. ...

There was some interest in such logging programs a while back, and I
wrote this one:

    ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

I've attached a message I sent out when I packaged that up.  I had no
time back then (2001, yikes) and didn't receive any comments or bug
reports.

If there is interest now, then I'll update the documentation, format
the code per the PostgreSQL coding standards and offer it again.

A patch to the postmaster so that it starts the log program and can
re-start it if someone kills it would improve robustness even further,
but 'kill -9' on the postmaster is bad, and so is 'kill -9' on the log
program. :-)

Regards,

Giles

------- =_aaaaaaaaaa0
Content-Type: message/rfc822



Subject: Re: [HACKERS] Log rotation?


Date: Sat, 08 Sep 2001 07:21:19 +1000


Hi Peter,

Quote:> I've been playing with a little program I wrote whose sole purpose is to
> write its stdin to a file and close and reopen that file when it receives
> a signal.  I figured this could work well when integrated transparently
> into pg_ctl.

> So, is log rotation a concern?  Is this a reasonable solution?  Other
> ideas?

There was a discussion of this over a year ago.  After I contributed
to the discussion Tom Lane suggested I write something, and in the
tradition of software development I did so but never quite finished it
... I got to the point that it works for me, then got distracted
before completing a test suite.

You may well prefer your own code, but this one supports rotation on
size and/or time basis as well as on receipt of SIGHUP, and places a
timestamp on each line written. It's also pretty careful about errors,
which is one of the things that was disliked about the Apache program
last time it was discussed.

I am happy to contribute the code using the standard PostgreSQL
license if it's wanted.  (If anyone wants it under a BSD license or
GPL for another purpose that's fine too.)

I use the code on HP-UX, and developed it on NetBSD.  There shouldn't
be too many portability problems lurking, other than the usual hassles
of what % escape to use in printf() for off_t.  I doubt anyone wants
log files larger than a couple of GB anyway? :-)

ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

Quote:> (No Grand Unified Logging Solutions please.  And no, "use syslog" doesn't
> count.)

<grin>

One improvement I suggest is that the postmaster be taught to start
(and restart if necessary) the log program.  This avoids fragile
startup scripts and also avoids taking down PostgreSQL if someone
sends the wrong signal to the log program.

Cheers,

Giles

------- =_aaaaaaaaaa0--

---------------------------(end of broadcast)---------------------------

 
 
 

Postgres server output logfile

Post by Jean-Luc Lachan » Wed, 05 Feb 2003 01:21:27


He is my very simple solution:

8<---------------------------
/*
 * log2file - send stdin to file opening it and closing it regularly
 */

#include <stdio.h>

FILE *  output;
char    buffer[ BUFSIZ];

int
main( int argc, char * argv[ ])
    {
    if ( argc != 2)
        {
        fprintf( stderr, "Usage: %s <output file>\n", argv[ 0]);
        exit( 1);
        }

    output = fopen(( const char *) argv[ 1], "a+");
    fclose( output);
    if ( output == NULL)
        {
        fprintf( stderr, "%s: cannot write to file: %s", argv[ 0], argv[
1]);
        exit( 2);
        }

    fclose( stderr);
    fclose( stdout);

    while ( ! feof( stdin))
        {
        fgets( buffer, sizeof( buffer), stdin);
        output = fopen(( const char *) argv[ 1], "a+");
        fputs( buffer, output);
        fclose( output);
        }
    exit( 0);
    }
8<------------------------------

In /etc/rc.d/init.d/postgresl

[...]
su -l postgres -s /bin/sh -c "/usr/bin/pg_ctl  -D $PGDATA -p
/usr/bin/postmaster start 2>&1 < /dev/null" |\
      /usr/local/bin/log2file /var/log/postgres &
[...]



> > Unfortunately not.  The recommended procedure for production servers is
> > not to send the postmaster's stdout/stderr directly to a disk file, but
> > to pipe it into some script that rotates the output.  There's a usable
> > script in the Apache distribution, or you can roll your own with little
> > effort. ...

> There was some interest in such logging programs a while back, and I
> wrote this one:

>     ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

> I've attached a message I sent out when I packaged that up.  I had no
> time back then (2001, yikes) and didn't receive any comments or bug
> reports.

> If there is interest now, then I'll update the documentation, format
> the code per the PostgreSQL coding standards and offer it again.

> A patch to the postmaster so that it starts the log program and can
> re-start it if someone kills it would improve robustness even further,
> but 'kill -9' on the postmaster is bad, and so is 'kill -9' on the log
> program. :-)

> Regards,

> Giles

>   ------------------------------------------------------------------------

> Subject: Re: [HACKERS] Log rotation?
> Date: Sat, 08 Sep 2001 07:21:19 +1000



> Hi Peter,

> > I've been playing with a little program I wrote whose sole purpose is to
> > write its stdin to a file and close and reopen that file when it receives
> > a signal.  I figured this could work well when integrated transparently
> > into pg_ctl.

> > So, is log rotation a concern?  Is this a reasonable solution?  Other
> > ideas?

> There was a discussion of this over a year ago.  After I contributed
> to the discussion Tom Lane suggested I write something, and in the
> tradition of software development I did so but never quite finished it
> ... I got to the point that it works for me, then got distracted
> before completing a test suite.

> You may well prefer your own code, but this one supports rotation on
> size and/or time basis as well as on receipt of SIGHUP, and places a
> timestamp on each line written. It's also pretty careful about errors,
> which is one of the things that was disliked about the Apache program
> last time it was discussed.

> I am happy to contribute the code using the standard PostgreSQL
> license if it's wanted.  (If anyone wants it under a BSD license or
> GPL for another purpose that's fine too.)

> I use the code on HP-UX, and developed it on NetBSD.  There shouldn't
> be too many portability problems lurking, other than the usual hassles
> of what % escape to use in printf() for off_t.  I doubt anyone wants
> log files larger than a couple of GB anyway? :-)

> ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz

> > (No Grand Unified Logging Solutions please.  And no, "use syslog" doesn't
> > count.)

> <grin>

> One improvement I suggest is that the postmaster be taught to start
> (and restart if necessary) the log program.  This avoids fragile
> startup scripts and also avoids taking down PostgreSQL if someone
> sends the wrong signal to the log program.

> Cheers,

> Giles

>   ------------------------------------------------------------------------

> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

 
 
 

Postgres server output logfile

Post by Giles Le » Wed, 05 Feb 2003 06:41:10



> He is my very simple solution:

One of the constraints on the code that I wrote was that it try to be
robust.  Having the database go down because the log writing program
exited due to a transiently full filesystem was thought to be
undesirable -- see Tom Lane's comments in the archives.

Quote:>     if ( output == NULL)
>         {
>         fprintf( stderr, "%s: cannot write to file: %s", argv[ 0], argv[
> 1]);
>         exit( 2);
>         }

The Apache rotatelogs program is similar, or was when I looked at it
last.  Perhaps they've worked on it recently.  (Perhaps I should offer
some improvements ... in my copious spare time. :-)

Regards,

Giles

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

 
 
 

Postgres server output logfile

Post by Bruno Wolff I » Wed, 05 Feb 2003 21:50:28


On Tue, Feb 04, 2003 at 08:39:03 +1100,


> > He is my very simple solution:

> One of the constraints on the code that I wrote was that it try to be
> robust.  Having the database go down because the log writing program
> exited due to a transiently full filesystem was thought to be
> undesirable -- see Tom Lane's comments in the archives.

multilog will block instead of exit when a file system fills up. That
may or may not be better for you.

---------------------------(end of broadcast)---------------------------

 
 
 

1. Setting logfile location with pg_ctl and postgres.conf

I am using beta3 and am setting my runtime options via
data/postgresql.conf, which works just fine:

log_connections = on
log_pid = true
log_timestamp = true
fsync = off
max_connections = 256
sort_mem = 8192
shared_buffers = 2048
tcpip_socket = on
unix_socket_directory = '/usr/db/pgsql/sockdir'

When starting with

/usr/db/pgsql/bin/pg_ctl -D /usr/db/pgsql/data

I don't see how you can set the log file to something like

/usr/db/pgsql/log/postgresql_db.log

From the docs you can see how to use the syslog facility via
postgresql.conf but it's not really obvious how you use both
the config file and pg_ctl with a logfile location like the above.

That I've only managed to do like:

/usr/db/pgsql/bin/postmaster -i -d 2 -B 2048 -N 256 -o '-F -S 8192'\

Is there a 'proper' way to accomplish this or do I simply create my own
startscript?

Regards, Frank

BTW: How do you see runtime options passed to the postmaster via the
config file? The command

pg_ctl status

only shows those options that were passed on
the command line.

2. When will a view ( joined tables! ) be updateable?

3. Postgres server output log

4. BEST C++ BOOK YOU MUST READ

5. enable pl/pgsql in postgres from postgres-server-7.0.3-2 RPM

6. D3/NT Memory Leakage

7. truncated output from sql server 7 using cold fusion to output

8. Affected Rows

9. SQL Server 7 backups / logfiles

10. Proxy Server Logfiles and DB compaction

11. SQL SERVER 7.0 DB logfile size growth....problem..(for WEB)

12. Web Application Server 3.0 Logfiles

13. Can't backup the logfile when the mdf file is corrupt in SQL Server 2000