Notifiaction service flawed in its design ?

Notifiaction service flawed in its design ?

Post by Paul Campbel » Wed, 30 Jul 2003 20:47:50



Having used the notification service intensively for 18 months now on my
current project I'm comming to the conclusion that the design is fundamentally
flawed.

The principal flaw is that it uses object references as the sole means for
deterministic consumer, producer and channel identity, and the identity
of all channel child objects such as admin and proxy objects. Granted
these objects have ID's but these cannot be specified at creation time
meaning that object references for all the above objects have to be
remembered "forever" by both producers and consumers. And further
more those objects references have to remain valid forever too such
that consumers and producers need to remain in a stateful lock-step
with the state of the object tree in the notification server (again
forever).

I cannot for example easily combine network mobile clients (e.g. web start
deployed clients) with persistent QoS based on a "well known" client ID of
some form. I would intead have to resort a very complex and admin intensive
setup involving an implementation respository to track clients. So I am
forced to with go the whole hog with persistent references for everything and
implementation repositories all over the place (which need updating everytime
something moves or goes away). Or if I make everything transient and
dynamically resolved (and thus zero-admin/self healing), I am forced to use transient
event channels and thus have to roll my own event reliability protocol between
producers and consumers.

If I use persistent event channels I cannot do a clean up of a consumer that
I know has "gone for good"  - I have to rely on vendor specific parameters
to cause the clean up using usually some time based criteria, which has the
effect of making the channel no longer "persistent" in effect so I cant win.

What I really want is a system where all persistent state related to events is
handled by the notification service and consumers and producers need only
remember thier espective unique IDs and a topic ID  in order to continue thier
"conversation" where  they left off. I want a system where events get retained
forever and yet where I have the facility to manually or programmatically do my
own clean up and not rely on simplistic proprietary retry counts and timeouts
or whatever.

Paul C.

 
 
 

Notifiaction service flawed in its design ?

Post by Michi Hennin » Thu, 31 Jul 2003 08:46:17


On Tue, 29 Jul 2003, Paul Campbell wrote:
> The principal flaw is that it uses object references as the sole means for
> deterministic consumer, producer and channel identity, and the identity
> of all channel child objects such as admin and proxy objects.

Yes, it's less than convenient. The Notification Service was lumped with
a backward compatibility requirement with the Event Service. The
Event Service does not allow you to identify its objects other than via
their IORs.

> Granted
> these objects have ID's but these cannot be specified at creation time
> meaning that object references for all the above objects have to be
> remembered "forever" by both producers and consumers.

Right. The argument at the time was that they should be put into the
Naming Service. This creates its own share of problems, of course, such
as how to keep the Event Service and the Naming Service in sync, and
what to do if the Naming Service isn't available at the time an Event
Service object is created or destroyed, and making clients responsible
for keeping track of the IORs (instead of the service itself).

This is common to all the early services. For example, if you ever lose
the IOR to a naming context, there is no standard way to get that IOR back
if the context isn't bound into some other context for which you have an IOR.
So, there is a real chance that disconnected garbage contexts accumulate
in the service over time. (Some vendors offer proprietary means to reconnect
lost contexts.) The same is true for the trading service. If you lose
an offer ID, you are hosed. Later specifications have tended to add collection
manager interfaces, so you can at least list everything and get it back
instead of being stuck completely, but that's of little help if you need
to use a service that doesn't do that. To be fair, the Notification
Service *does* provide collection manager interfaces, but it's still
clumsy because, as you say, you cannot control the ID of objects.

> And further
> more those objects references have to remain valid forever too such
> that consumers and producers need to remain in a stateful lock-step
> with the state of the object tree in the notification server (again
> forever).

Yes. Not nice. If the notion of what's in the service ever goes out of
sync with the clients' or servers' notion, (or with what is in the
naming service), you have a problem.

> What I really want is a system where all persistent state related to events is
> handled by the notification service and consumers and producers need only
> remember thier espective unique IDs and a topic ID  in order to continue thier
> "conversation" where  they left off. I want a system where events get retained
> forever and yet where I have the facility to manually or programmatically do my
> own clean up and not rely on simplistic proprietary retry counts and timeouts
> or whatever.

In many ways, the root of the problem is related to the opaqueness of
IORs: a client isn't allowed to manufacture an IOR out of thin air, and
the object identity inside an IOR is hidden from the client. This
immediately means that we need things such as the Naming Service in
order to keep track of IORs. It also means that the standard factory
pattern is not very useful:

    interface SomeObject { /* ... */ };

    interface Factory {
        SomeObject create(/* params */);
    };

The problem is that, once I have created the IOR for the object, I know
nothing about the object without asking it further questions. In
particular, I don't know the object's identity. If I lose the IOR, I
can't ever get back to the object.

To some extent, the problem can be alleviated by something like the following:

    exception IdExistsException {};

    interface Factory {
        SomeObject create(in IdentityType id /* other params */)
                        raises(IdExistsException);
    };

At least I can now control the object identity and, therefore, use it
again later to get the object back, given its identity:

    interface Collection {
        SomeObject find(in IdentityType id);
    };

But now I get a chicken-and-egg problem because I don't know the
identity of the collection manager so, if I lose that, I'm just as
stuck. (Of course, this leads to an infinite regression of collection
managers.)

Consider what happens if you do *not* make IORs opaque and allow clients
to manufacture IORs from a logical name (such as the combination of
adapter name and object identity). Suddenly, things get a lot easier
because there is no longer a need for a naming service and, if I lose an
IOR, I can always reconstruct it as long as I know the adapter name and
the object identity. (The architectural consequences of non-opaque IORs
go much further than this, actually -- many APIs get a lot simpler with
such a design and identity comparison can be performed as a local
operation instead requiring a remote invocation.)

Unfortunately, IORs are opaque, so that's not an option for CORBA.

If you are interested, the object identity discussion has a long history
in the OMG. Here are a few relevant references:

- A Collection of Electronic Mail Messages on the Subject
  of "Object Identity Crisis in CORBA"
  ftp://ftp.omg.org/pub/docs/om/96-10-05.pdf

- Green Paper on Object Identity
  ftp://ftp.omg.org/pub/docs/om/97-04-01.pdf

- Identity, Equality and Location
  ftp://ftp.omg.org/pub/docs/om/97-03-10.pdf

- The Importance of Using Object References as Identifiers
  of Objects -- Comparison of CORBA Object References
  ftp://ftp.omg.org/pub/docs/1994/94-06-12.pdf

- Objects, References, Identifiers & Equality White Paper
  ftp://ftp.omg.org/pub/docs/1993/93-07-05.pdf

Personally, I think it would have been better to not make IORs opaque
and to support a strong notion of object identity (instead of the weak
notion that CORBA uses). Many things would be a lot simpler that way.

Another problem I see is that the Event Service (and hence, the
Notification Service) is channel-based, not topic-based. To decide which
events I want to receive, I connect to a channel, that is, logically,
the consumer does *not* say "Give me the events I am interested in" but
says "I know that the events I want come over this and that channel."
The problem with this is that, to get the events the consumer wants, it
must be aware to some extent of the physical channel structure of the
service -- if the channel structure has to change (and there are good
reasons for changing it, for example, to improve performance), suppliers
and consumers may have to change to reflect the change. (It's a bit like
the difference between a UUCP-style e-mail adress and modern domain name
e-mail address: one exposes the network topology whereas the other one hides
it.) A topic-based approach is far nicer for clients of the service
because they can simply state what the event is about, instead of having
to figure out where to get it from.

As far as QoS parameters are concerned, there are many ambiguities
around these, especially around how different parameter settings should
interact. We struggled with this when we implemented the Notification
Service and ended up (like any other vendor) making best guesses at
reasonable behavior. (Coincidentally, the same is true for some of the
trader link and search policies.)

Historically, the birth of the Notification Service was a very difficult
one. There was a fairly large group of cosubmitters, including IBM, DSTC,
BEA, IONA, ICL, HP, NEC, Oracle, Nortel, Visigenic, Fujitsu, Expersoft,
and GMD Fokus. It was very difficult to reach agreement among all these
parties (with the usual result that the specification ended up bigger and
more complex than necessary). I vividly remember insistence by IBM and
Fujitsu to make everything in the service transactional (and that
was prior to the updated OTS spec), which would have made the service
useless for one of its main application domains (telecom), whereas
others wanted something far more light-weight, and yet others were keen
on being able to do sophisticated manipulations of subscription
expressions to limit event propagation to places where events might
actually be consumed. Right toward the end, a rift developed for which
no compromise could be found and, as a result, HP split off and did
its own submission with Telefonica (which wasn't accepted in the end).

Anyway, what you see today is the result of compromise. Quite possibly,
the attempt to make the service so general is a problem. Something more
tighly focussed would probably be more convenient to use.

Cheers,

Michi.

--
Michi Henning                Ph: +61 4 1118-2700
ZeroC, Inc.                  http://www.zeroc.com

 
 
 

1. Mustek design flaw?

I've used a Mustek 600 III EP plus for the past 6 months and have just
returned it to the store who fortunately returned my money.  The problem
was that the flat cable attached to the scanning head was making contact
with the underside of the glass.  This happened when the head was at
approximately the half-way point and beyond and was scratching the
coating of the glass leaving visible lines.

I wonder if it was only my scanner that had this problem or is it a
design flaw.  I also wonder if other Mustek models have this problem.
I'd be interested to hear from other Mustek owners.

The kindly store was Wal-mart.  I'm about to buy a Umax 1220s instead.

cheers

2. Epson color stylus

3. Flaw in Kensington Orbit Design???

4. FS SunOS CDROMs

5. PGP design flaw (?) - Doubtful, I know

6. re-running the module configuration script

7. Serial port has glitch right at mid-night --- Seems to be a Palm OS design flaw

8. Naming Service flaw

9. Design plug-in with Acrobat plug-ins SDK

10. INS service of omniORB

11. INS (International Network Services)

12. Robotic CAD design services