LISTEN/NOTIFY benchmarks?

LISTEN/NOTIFY benchmarks?

Post by prasha.. » Thu, 01 May 2003 09:18:15





> > If the listening backend had registered tens of thousands of LISTENs,
> > it would be sent an equivalent number of signals during a single run
> > of AtCommit_Notify().

> Not unless the notifier had notified all tens of thousands of condition
> names in a single transaction.

Unfortunately, that is a possibility in our application.  We are now
working around this non-scalability.

Regardless, it would seem redundant to send more than one SIG_USR2 to the
recipient backend in that loop.

-- prashanth

---------------------------(end of broadcast)---------------------------

 
 
 

LISTEN/NOTIFY benchmarks?

Post by Sean Chittend » Thu, 01 May 2003 09:31:45


Quote:> I'm not an expert on signals, not even a novice, so I might be
> totally off base, but it seems like the Async Notification
> implementation does not scale.  If it does not, does anyone have a
> solution for the problem of signalling a each event in a possibly
> very large set of events to a large number of clients?

<brainfart_for_the_archives> Hrm.... I should see about porting
kqueue/kevent as a messaging buss for the listen/notify bits to
postgresql... that does scale and it scales well to tens of thousands
of connections a second (easily over 60K, likely closer to 1M is the
limit)....  </brainfart_for_the_archives>

--
Sean Chittenden

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

 
 
 

LISTEN/NOTIFY benchmarks?

Post by Gavin Sher » Thu, 01 May 2003 10:11:19



> > I'm not an expert on signals, not even a novice, so I might be
> > totally off base, but it seems like the Async Notification
> > implementation does not scale.  If it does not, does anyone have a
> > solution for the problem of signalling a each event in a possibly
> > very large set of events to a large number of clients?

> <brainfart_for_the_archives> Hrm.... I should see about porting
> kqueue/kevent as a messaging buss for the listen/notify bits to
> postgresql... that does scale and it scales well to tens of thousands
> of connections a second (easily over 60K, likely closer to 1M is the
> limit)....  </brainfart_for_the_archives>

Except that it is FreeBSD specific -- being system calls and all -- if I
remember correctly. If you're going to move to a system like that, which
is a good idea, best move to a portable system.

Thanks,

Gavin

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command

 
 
 

LISTEN/NOTIFY benchmarks?

Post by Sean Chittend » Thu, 01 May 2003 10:26:56


Quote:> > > I'm not an expert on signals, not even a novice, so I might be
> > > totally off base, but it seems like the Async Notification
> > > implementation does not scale.  If it does not, does anyone have
> > > a solution for the problem of signalling a each event in a
> > > possibly very large set of events to a large number of clients?

> > <brainfart_for_the_archives> Hrm.... I should see about porting
> > kqueue/kevent as a messaging buss for the listen/notify bits to
> > postgresql... that does scale and it scales well to tens of
> > thousands of connections a second (easily over 60K, likely closer
> > to 1M is the limit)....  </brainfart_for_the_archives>

> Except that it is FreeBSD specific -- being system calls and all --
> if I remember correctly. If you're going to move to a system like
> that, which is a good idea, best move to a portable system.

You can #ifdef abstract things so that select() and poll() work if
available.  Though now that I think about it, a queue that existed
completely in userland would be better... an shm implementation that's
abstracted would be ideal, but shm is a precious resource and can't
scale all that big.  A shared mmap() region, however, is much less
scarce and can scale much higher.  mmap() + semaphore as a gate to a
queue would be ideal, IMHO.

I shouldn't be posti^H^H^H^H^Hrambling though, haven't slept in 72hrs.
:-/  *stops reading email*  -sc

--
Sean Chittenden

---------------------------(end of broadcast)---------------------------

 
 
 

1. LISTEN/NOTIFY benchmarks?

Hi,

I'm looking for information on the scalabality of the LISTEN/NOTIFY
mechanism.  How well does it scale with respect to:

- hundreds of clients registered for LISTENs

    I guess this translates to hundreds of the corresponding backend
    processes receiving SIG_USR2 signals.  The efficiency of this is
    probably OS-dependent.  Would anyone be in a position to give me
    signal delivery benchmarks for FreeBSD on Unix?

- each client registered for thousands of LISTENs

    From a look at backend/commands/async.c, it would seem that each
    listening backend would get a signal for *every* LISTEN it
    registered for, resulting in thousands of signals to the same
    listening backend, instead of only one.  Would it help if this was
    optimized so that a signal was sent only once?  Again, info on
    relevant signal delivery benchmarks would be useful.  

I'm not an expert on signals, not even a novice, so I might be totally
off base, but it seems like the Async Notification implementation does
not scale.  If it does not, does anyone have a solution for the
problem of signalling a each event in a possibly very large set of
events to a large number of clients?

Thanks,

--prashanth

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

2. MD-Rockville-259462--ORACLE-WINSTAR- Sr. Oracle Developer

3. NOTIFY/LISTEN Using Embedded SQL (ecpg)

4. TEXT Columns

5. listen/notify argument (old topic revisited)

6. run-time error

7. Opening a PASSWORD'd .MDB from VB ?

8. problem with notify/listen

9. LISTEN/NOTIFY

10. Synchronous LISTEN/NOTIFY?

11. LISTEN/NOTIFY

12. DBD::Pg and NOTIFY (was Re: [GENERAL] Synchronous LISTEN/NOTIFY?)