Strictly Enforcing Query Plan Stability

Strictly Enforcing Query Plan Stability

Post by Dave » Sun, 31 Dec 1899 09:00:00



It is.  Control access to the database via passwords.  Application passwords
should not be given out to anyone, just the application.

--
Dave A


Quote:

> I have a wishlist feature I would love Oracle to support. In fact I'm
> surprised Oracle doesn't support it already. Perhaps there's a trick to
> accomplish what I want and I just don't know it?

> What I would like is to be able to set privileges such that my application
> cannot run _any_ query that doesn't have an outline already stored for it.
> Essentially what I want is to be able to guarantee that no code could
possibly
> go live without _every_ SQL query being analyzed and the plan approved by
a
> DBA. Any unapproved query should immediately get an error, not be run with
> some ad hoc query plan that could very well bring the whole application
down.

> It seems to me that anything less is simply inadequate for a production
> mission critical system. Given the types of applications that run on
Oracle
> I'm surprised this isn't a fundamental feature of the system since day 1.

> --
> greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Tue, 10 Oct 2000 10:36:39


I have a wishlist feature I would love Oracle to support. In fact I'm
surprised Oracle doesn't support it already. Perhaps there's a trick to
accomplish what I want and I just don't know it?

What I would like is to be able to set privileges such that my application
cannot run _any_ query that doesn't have an outline already stored for it.
Essentially what I want is to be able to guarantee that no code could possibly
go live without _every_ SQL query being analyzed and the plan approved by a
DBA. Any unapproved query should immediately get an error, not be run with
some ad hoc query plan that could very well bring the whole application down.

It seems to me that anything less is simply inadequate for a production
mission critical system. Given the types of applications that run on Oracle
I'm surprised this isn't a fundamental feature of the system since day 1.

--
greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by Connor McDonal » Sun, 31 Dec 1899 09:00:00



> I have a wishlist feature I would love Oracle to support. In fact I'm
> surprised Oracle doesn't support it already. Perhaps there's a trick to
> accomplish what I want and I just don't know it?

> What I would like is to be able to set privileges such that my application
> cannot run _any_ query that doesn't have an outline already stored for it.
> Essentially what I want is to be able to guarantee that no code could possibly
> go live without _every_ SQL query being analyzed and the plan approved by a
> DBA. Any unapproved query should immediately get an error, not be run with
> some ad hoc query plan that could very well bring the whole application down.

> It seems to me that anything less is simply inadequate for a production
> mission critical system. Given the types of applications that run on Oracle
> I'm surprised this isn't a fundamental feature of the system since day 1.

> --
> greg

Easy enough to do...

In its simplest form, users have 'create session' and a role which lets
them do whats needed in the applications.  The role is only enabled when
they launch the app (as protected by a role password).

Then outside the app, the only thing that they can do is connect.  All
you need do then to ensure that nothing is 'grant select to public' and
you're protected..

HTH
--
===========================================
Connor McDonald
http://www.oracledba.co.uk
(faster/mirrored at http://www.oradba.freeserve.co.uk)

Its not the voices in my head that bother me...
  its the voices in yours.

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Sun, 31 Dec 1899 09:00:00


Uhm, both of you have completely failed to understand the issue. The issue is
that anyone with create session ability can run arbitrary SQL on any table
they have access to. Moreover any developer can introduce queries into the
application and they can slip by the QA process without being analyzed.

All I want is for Oracle to not allow arbitrary SQL from the user the
application connects as. It should *only* allow SQL to be executed if it has a
stored outline. If there's no outline the optimizer should immediately throw
an error.

This seems like a minimum requirement for a stable production system. Anything
less means that the operations people can never guarantee that an error in
judgement by developers releasing new code won't take downt the entire
production system by trying to run a full table scan a thousand times a second
on a million record table.

Without this type of feature from Oracle everything depends on the developers
diligently running explain plan and testing every query in the application. If
they miss one it should be exactly like if they accidentally update a table
they're not supposed to update (permission denied) or delete a file they're
not supposed to delete etc. Oracle should not just make up new query plans on
the fly on the production database. Frankly that's insane.


> It is.  Control access to the database via passwords.  Application passwords
> should not be given out to anyone, just the application.

> --
> Dave A



> > I have a wishlist feature I would love Oracle to support. In fact I'm
> > surprised Oracle doesn't support it already. Perhaps there's a trick to
> > accomplish what I want and I just don't know it?

> > What I would like is to be able to set privileges such that my application
> > cannot run _any_ query that doesn't have an outline already stored for it.
> > Essentially what I want is to be able to guarantee that no code could
> possibly
> > go live without _every_ SQL query being analyzed and the plan approved by
> a
> > DBA. Any unapproved query should immediately get an error, not be run with
> > some ad hoc query plan that could very well bring the whole application
> down.

> > It seems to me that anything less is simply inadequate for a production
> > mission critical system. Given the types of applications that run on
> Oracle
> > I'm surprised this isn't a fundamental feature of the system since day 1.

> > --
> > greg

--
greg
 
 
 

Strictly Enforcing Query Plan Stability

Post by fran » Sun, 31 Dec 1899 09:00:00


1st of all - it is exactly what the optimizer can do - rewrite queries.
2nd - you cannot be serious in developing applications like that:
  data changes over time, making assumptions, explain plans and whatever
you have tested useless.

Yuo are a sales company: how would you optimize the customer vs products
queries (e.g. orders, order lines)?

The simple answer is: you cannot, unless you *know* your data.
If I'm on the internet, I might have millions of customers, with only 30 products
(example? Think Pokemon!).
If I were "traditional", I could have thousands of products, and only a few
customers
(example? Think producing raw material; input for other processes).

In both examples, I would like the othe table to be the driving table... So how to

resolve that? You cannot. The optimizer can (well, should...) because it knows
data.

BTW, The above is exactly why most apps, inclusing Oracle apps, perform badly.

my 2.5C


> Uhm, both of you have completely failed to understand the issue. The issue is
> that anyone with create session ability can run arbitrary SQL on any table
> they have access to. Moreover any developer can introduce queries into the
> application and they can slip by the QA process without being analyzed.

> All I want is for Oracle to not allow arbitrary SQL from the user the
> application connects as. It should *only* allow SQL to be executed if it has a
> stored outline. If there's no outline the optimizer should immediately throw
> an error.

> This seems like a minimum requirement for a stable production system. Anything
> less means that the operations people can never guarantee that an error in
> judgement by developers releasing new code won't take downt the entire
> production system by trying to run a full table scan a thousand times a second
> on a million record table.

> Without this type of feature from Oracle everything depends on the developers
> diligently running explain plan and testing every query in the application. If
> they miss one it should be exactly like if they accidentally update a table
> they're not supposed to update (permission denied) or delete a file they're
> not supposed to delete etc. Oracle should not just make up new query plans on
> the fly on the production database. Frankly that's insane.


> > It is.  Control access to the database via passwords.  Application passwords
> > should not be given out to anyone, just the application.

> > --
> > Dave A



> > > I have a wishlist feature I would love Oracle to support. In fact I'm
> > > surprised Oracle doesn't support it already. Perhaps there's a trick to
> > > accomplish what I want and I just don't know it?

> > > What I would like is to be able to set privileges such that my application
> > > cannot run _any_ query that doesn't have an outline already stored for it.
> > > Essentially what I want is to be able to guarantee that no code could
> > possibly
> > > go live without _every_ SQL query being analyzed and the plan approved by
> > a
> > > DBA. Any unapproved query should immediately get an error, not be run with
> > > some ad hoc query plan that could very well bring the whole application
> > down.

> > > It seems to me that anything less is simply inadequate for a production
> > > mission critical system. Given the types of applications that run on
> > Oracle
> > > I'm surprised this isn't a fundamental feature of the system since day 1.

> > > --
> > > greg

> --
> greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Sun, 31 Dec 1899 09:00:00



> 1st of all - it is exactly what the optimizer can do - rewrite queries.
> 2nd - you cannot be serious in developing applications like that:
>   data changes over time, making assumptions, explain plans and whatever
> you have tested useless.

Not useless, perhaps not optimal but at least predictable. What's useless is
an application that might due to a single bad query suddenly stop working
completely.

I'm very serious. This seems like a major flaw in Oracle that makes it
fundamentally unsuitable for mission critical OLTP applications. It means the
operations people cannot guarantee that they know what Oracle is doing at all
time and cannot guarantee that the application will continue to work as the
data changes.

Quote:> Yuo are a sales company: how would you optimize the customer vs products
> queries (e.g. orders, order lines)?

You make a fundamental assumption that the optimizer is infallible. Have you
actually tried to use the damned thing? It's very helpful for initially
finding good query plans in development but it's not appropriate to be using
it at run-time. And in any the optimizer isn't going to do anything
intelligent in situations where you really need to build a new index or just
rethink your query. The most important quality for a production system is
predictability, every query should have a known deterministic behaviour.

That need for deterministic behaviour is why Oracle now includes plan
stability as a new feature in 8i. However it's inadequate since the default is
still to use the optimizer for queries without stored outlines. That means the
system as a whole is still unpredictable and it may run any plan at any time
without warning. That's fine for development, and probably even acceptable for
a data warehouse, but it's completely unacceptable in a stable OLTP
application.

--
greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by fran » Sun, 31 Dec 1899 09:00:00




> > 1st of all - it is exactly what the optimizer can do - rewrite queries.
> > 2nd - you cannot be serious in developing applications like that:
> >   data changes over time, making assumptions, explain plans and whatever
> > you have tested useless.

> Not useless, perhaps not optimal but at least predictable. What's useless is
> an application that might due to a single bad query suddenly stop working
> completely.

I have never ever seen that happen, or -probably more important- heard about
such an event.

Quote:

> I'm very serious. This seems like a major flaw in Oracle that makes it
> fundamentally unsuitable for mission critical OLTP applications. It means the
> operations people cannot guarantee that they know what Oracle is doing at all
> time and cannot guarantee that the application will continue to work as the
> data changes.

Well, who come there are some many of them around? All of these designers
and users must have less high standard than you have.

Quote:

> > Yuo are a sales company: how would you optimize the customer vs products
> > queries (e.g. orders, order lines)?

> You make a fundamental assumption that the optimizer is infallible. Have you
> actually tried to use the damned thing?

No, I'm not. That's why hints were invented. And yes, I have used it.

Quote:> It's very helpful for initially
> finding good query plans in development but it's not appropriate to be using
> it at run-time.

Why would that be?

Quote:> And in any the optimizer isn't going to do anything
> intelligent in situations where you really need to build a new index or just
> rethink your query.

Agree wholehartedly here - perfomance starts with design, understanding, and
developers that know Oracle, and don't regard indexes as panacee for lack of
performance due to writing bad code.

Quote:> The most important quality for a production system is
> predictability, every query should have a known deterministic behaviour.

I predict that every database will show degrading performance over time.
And I make a living with it. Tuning is an ongoing job - thank heavens!

Quote:

> That need for deterministic behaviour is why Oracle now includes plan
> stability as a new feature in 8i. However it's inadequate since the default is
> still to use the optimizer for queries without stored outlines. That means the
> system as a whole is still unpredictable and it may run any plan at any time
> without warning. That's fine for development, and probably even acceptable for
> a data warehouse, but it's completely unacceptable in a stable OLTP
> application.

> --
> greg

Point here is:
- this dicussion is quite academic: if computers finally would get a
decent O/S and UI, life would be easier. Yeah, if databases would be predictable,
ditto. If databases would be smart, you and I would be out of a job.
- I don't believe in a perfect query plan. What may look perfect in test,
(e.g. index use) may be better off doing FTS on production. Or, just a slight
addition
to the query restrictions. The optimizer can, and in most cases will detect that,
and
go for another plan - with better response.
- People are unpredictable. People generate data. So, data will be unpredictable.
And datadistribution, too.

I do agree to a certain degree, but I have seen too many implementations where
assumptions were not the actual truth, making designs, well, err,
less-than-optimal.
Still happy with the optimizer, which is getting better with every new release.

Frank

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Sun, 31 Dec 1899 09:00:00



> I have never ever seen that happen, or -probably more important- heard about
> such an event.

> > I'm very serious. This seems like a major flaw in Oracle that makes it
> > fundamentally unsuitable for mission critical OLTP applications. It means the
> > operations people cannot guarantee that they know what Oracle is doing at all
> > time and cannot guarantee that the application will continue to work as the
> > data changes.

> Well, who come there are some many of them around? All of these designers
> and users must have less high standard than you have.

This sounds like a typical scenario where Oracle is being used for purposes
that it hasn't in the past. Perhaps you've been working exclusively on
"traditional" database applications like data warehouses and environments with
very tightly controlled code and very rigorous testing regimen. However the
world has changed and Oracle is scrambling to catch up.

Quote:> > You make a fundamental assumption that the optimizer is infallible. Have you
> > actually tried to use the damned thing?

> No, I'm not. That's why hints were invented. And yes, I have used it.

> > It's very helpful for initially finding good query plans in development
> > but it's not appropriate to be using it at run-time.

> Why would that be?

So you don't think the plan stability features introduced in 8i are useful for
anything? If the optimizer changes behaviour it's infallibly because the new
plan is better than the old one? If a new query is introduced and it's checked
with one data set then it's undoubtedly going produce the same plan or better
on the production system?

Perhaps some concrete examples would be clearer:

I write a web site involving basic queries involving some joins etc. I test it
out on a sample database, I test it out on a QA system, including checking the
plans and disk i/o etc. It goes into production.

It gradually slows down as more users sign on, that's expected, but it's
basically satisfactory. I'm monitoring the response time planning to do some
tuning well before the response time becomes too slow. All of the sudden
everything grinds to a complete halt, Oracle has decided to start using full
table scans because the tables have passed some arbitrary size that the
optimizer decided was worth doing full table scans. It turns out Oracle was
overeager in switching to full table scans and they were actually slightly
slower. Had it stuck with the existing plans at least I could have continued
to monitor the response time and react in plenty of time to deal with the
performance degradation.

That scenario is handled by the plan stability features in 8i. However the
next scenario is what I'm actually complaining about:

Now I have one of the developers implement a new web page. I check the code
and test it on the QA system. We establish stored outlines for every query
with an acceptable execution plan with acceptable response times. However
either due to a fault in my testing procedure or perhaps due to some obscure
corner case that wasn't anticipated at all, there's a query in the new code
that I don't verify. It turns out that this query does a full table scan on a
large table. When this code is placed into production and receives hundreds of
hits per second that full table scan completely hammers Oracle saturating the
disk i/o and blocking every process doing the same query. The entire
application is killed by an execution plan that Oracle devised ad hoc without
having been approved.

This is the scenario I would like to guarantee won't occur. The development
people can do their job with testing but they can't guarantee that bugs will
never occur. But from an operations point of view we should be able to
guarantee that a bug causes an immediate error rather than having Oracle just
try to do its best to come up with an ad hoc execution plan and possibly kill
the entire application.

--
greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by fran » Sun, 31 Dec 1899 09:00:00


As you say: bugs can occur. The optimizer will do a fts when and if that is more
efficient to do than an index scan, followed by ta by rowid.
If that is not the case, then that's a bug in the optimizer.
And: index reads, followed by table access by rowid is *not* always the
best way to get your results!
I keep demonstrating that to developers, and the result is always the same:
astonishment and shock.


> > I have never ever seen that happen, or -probably more important- heard about
> > such an event.

> > > I'm very serious. This seems like a major flaw in Oracle that makes it
> > > fundamentally unsuitable for mission critical OLTP applications. It means the
> > > operations people cannot guarantee that they know what Oracle is doing at all
> > > time and cannot guarantee that the application will continue to work as the
> > > data changes.

> > Well, who come there are some many of them around? All of these designers
> > and users must have less high standard than you have.

> This sounds like a typical scenario where Oracle is being used for purposes
> that it hasn't in the past. Perhaps you've been working exclusively on
> "traditional" database applications like data warehouses and environments with
> very tightly controlled code and very rigorous testing regimen. However the
> world has changed and Oracle is scrambling to catch up.

> > > You make a fundamental assumption that the optimizer is infallible. Have you
> > > actually tried to use the damned thing?

> > No, I'm not. That's why hints were invented. And yes, I have used it.

> > > It's very helpful for initially finding good query plans in development
> > > but it's not appropriate to be using it at run-time.

> > Why would that be?

> So you don't think the plan stability features introduced in 8i are useful for
> anything? If the optimizer changes behaviour it's infallibly because the new
> plan is better than the old one? If a new query is introduced and it's checked
> with one data set then it's undoubtedly going produce the same plan or better
> on the production system?

> Perhaps some concrete examples would be clearer:

> I write a web site involving basic queries involving some joins etc. I test it
> out on a sample database, I test it out on a QA system, including checking the
> plans and disk i/o etc. It goes into production.

> It gradually slows down as more users sign on, that's expected, but it's
> basically satisfactory. I'm monitoring the response time planning to do some
> tuning well before the response time becomes too slow. All of the sudden
> everything grinds to a complete halt, Oracle has decided to start using full
> table scans because the tables have passed some arbitrary size that the
> optimizer decided was worth doing full table scans. It turns out Oracle was
> overeager in switching to full table scans and they were actually slightly
> slower. Had it stuck with the existing plans at least I could have continued
> to monitor the response time and react in plenty of time to deal with the
> performance degradation.

> That scenario is handled by the plan stability features in 8i. However the
> next scenario is what I'm actually complaining about:

> Now I have one of the developers implement a new web page. I check the code
> and test it on the QA system. We establish stored outlines for every query
> with an acceptable execution plan with acceptable response times. However
> either due to a fault in my testing procedure or perhaps due to some obscure
> corner case that wasn't anticipated at all, there's a query in the new code
> that I don't verify. It turns out that this query does a full table scan on a
> large table. When this code is placed into production and receives hundreds of
> hits per second that full table scan completely hammers Oracle saturating the
> disk i/o and blocking every process doing the same query. The entire
> application is killed by an execution plan that Oracle devised ad hoc without
> having been approved.

> This is the scenario I would like to guarantee won't occur. The development
> people can do their job with testing but they can't guarantee that bugs will
> never occur. But from an operations point of view we should be able to
> guarantee that a bug causes an immediate error rather than having Oracle just
> try to do its best to come up with an ad hoc execution plan and possibly kill
> the entire application.

> --
> greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by Mark D Powel » Sun, 31 Dec 1899 09:00:00


The thread is on Greg's desire for the optimizer to reject all queries
that do not have plan stability defined.

I would like to point out that request is not reasonable or well
considered because of something called adhoc!  If the end-user has
access to end-user reporting tools there is no way to antiscipate all
possible queries nor should you have to.  The CBO was invented to
handle adhoc queries.  For production code you should lock the path in
using hints or the new query plan statbility feature, but even here we
have found the CBO is usually good enough and we only tune what we have
to.




> > I have a wishlist feature I would love Oracle to support. In fact
I'm
> > surprised Oracle doesn't support it already. Perhaps there's a
trick to
> > accomplish what I want and I just don't know it?

> > What I would like is to be able to set privileges such that my
application
> > cannot run _any_ query that doesn't have an outline already stored
for it.
> > Essentially what I want is to be able to guarantee that no code
could possibly
> > go live without _every_ SQL query being analyzed and the plan
approved by a
> > DBA. Any unapproved query should immediately get an error, not be
run with
> > some ad hoc query plan that could very well bring the whole
application down.

> > It seems to me that anything less is simply inadequate for a
production
> > mission critical system. Given the types of applications that run
on Oracle
> > I'm surprised this isn't a fundamental feature of the system since
day 1.

> > --
> > greg

> Easy enough to do...

> In its simplest form, users have 'create session' and a role which
lets
> them do whats needed in the applications.  The role is only enabled
when
> they launch the app (as protected by a role password).

> Then outside the app, the only thing that they can do is connect.  All
> you need do then to ensure that nothing is 'grant select to public'
and
> you're protected..

> HTH
> --
> ===========================================
> Connor McDonald
> http://www.oracledba.co.uk
> (faster/mirrored at http://www.oradba.freeserve.co.uk)

--
Mark D. Powell  -- The only advice that counts is the advice that
 you follow so follow your own advice --

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

Strictly Enforcing Query Plan Stability

Post by Jonathan Lewi » Sun, 31 Dec 1899 09:00:00


But Greg points out that ad-hoc is particularly the thing he wants
to stop - any ad-hoc query which therefore does not have a pre-
computed plan should fail at parse time.

If this were my aim, I think I would go for a 3-tier architecture,
using the 'trusted proxy user', so that end users never knew
their own Oracle passwords, and certainly never learnt the
password of the proxy (8.1 / OCI only feature though).

You could try creating a view on the outln$ table that
forced a once-per-execute call to a function that failed
if no rows were returned.  Then the recursive SQL to
parse the query would crash if no plan existed.

--

Jonathan Lewis
Yet another Oracle-related web site:  http://www.jlcomp.demon.co.uk


>The thread is on Greg's desire for the optimizer to reject all queries
>that do not have plan stability defined.

>I would like to point out that request is not reasonable or well
>considered because of something called adhoc!  If the end-user has
>access to end-user reporting tools there is no way to antiscipate all
>possible queries nor should you have to.  The CBO was invented to
>handle adhoc queries.  For production code you should lock the path in
>using hints or the new query plan statbility feature, but even here we
>have found the CBO is usually good enough and we only tune what we have
>to.




>> > I have a wishlist feature I would love Oracle to support. In fact
>I'm
>> > surprised Oracle doesn't support it already. Perhaps there's a
>trick to
>> > accomplish what I want and I just don't know it?

>> > What I would like is to be able to set privileges such that my
>application
>> > cannot run _any_ query that doesn't have an outline already stored
>for it.
>> > Essentially what I want is to be able to guarantee that no code
>could possibly
>> > go live without _every_ SQL query being analyzed and the plan
>approved by a
>> > DBA. Any unapproved query should immediately get an error, not be
>run with
>> > some ad hoc query plan that could very well bring the whole
>application down.

>> > It seems to me that anything less is simply inadequate for a
>production
>> > mission critical system. Given the types of applications that run
>on Oracle
>> > I'm surprised this isn't a fundamental feature of the system since
>day 1.

>> > --
>> > greg

>> Easy enough to do...

>> In its simplest form, users have 'create session' and a role which
>lets
>> them do whats needed in the applications.  The role is only enabled
>when
>> they launch the app (as protected by a role password).

>> Then outside the app, the only thing that they can do is connect.  All
>> you need do then to ensure that nothing is 'grant select to public'
>and
>> you're protected..

>> HTH
>> --
>> ===========================================
>> Connor McDonald
>> http://www.oracledba.co.uk
>> (faster/mirrored at http://www.oradba.freeserve.co.uk)

>--
>Mark D. Powell  -- The only advice that counts is the advice that
> you follow so follow your own advice --

>Sent via Deja.com http://www.deja.com/
>Before you buy.

 
 
 

Strictly Enforcing Query Plan Stability

Post by Ghos » Sun, 31 Dec 1899 09:00:00


  From a developer's viewpoint, I've watched this whole discussion in
both shock and amu*t.  If the developers aren't skilled enough to
write decent queries, either train them or fire them.  If you have to
go through every inch of their code, either they are incompetent or
you are overbearing.  If you don't have a little (just a little) trust
in the Oracle parser, then you're fighting yourself.

  Again, from a developer's viewpoint, I would recommend the
following:

1. Let the Oracle parser do it's job.  It's fairly good, and can be
overridden with hints.
2. Ask (don't demand) the developers to learn and help to train them
in SQL tuning and testing practices.  I'd recommend the book "Oracle
SQL High Performance Tuning" by Guy Harrison - It's written for
developers.  Teach them to run traces and tkprof it.  Give them plug
and play explain plan scripts.  If you'd like the scripts I use then
send an Email.  Basically, give them the work and they'll eventually
thank you for it.
3. If the developers don't have a Development database that matches
Production in both size and data, they can't tune.  Period, end of
story.  If you want to do their job, then fire them and start coding.
4. No database is 100%.  No code is 100%.  Whether it's a patch, data
entry, or untested little piece of code, nothing is perfect.  Of
course, striding for 100% is still the goal as long as it doesn't have
a severe impact on getting things into production to begin with.  If
you don't allow programs to get into Production, how long do you think
you'll have a job?
5. Start hammering Oracle with requests for a plan stability option
(although that goes against the Cost based optimizer they are
pushing).  I've seen several  situations where plan stability would
have caused Production to fail, so be careful on what you wish for.

--
---------------------------------------------------------------------
Chad Thompson,
Programmer Analyst: VB, VC++, PLSQL, Oracle HRMS, Security Apps

Thou shalt not tick off the dragon.... for thou art crunchy and
taste good with ketchup.
---------------------------------------------------------------------


> As you say: bugs can occur. The optimizer will do a fts when and if that is more
> efficient to do than an index scan, followed by ta by rowid.
> If that is not the case, then that's a bug in the optimizer.
> And: index reads, followed by table access by rowid is *not* always the
> best way to get your results!
> I keep demonstrating that to developers, and the result is always the same:
> astonishment and shock.



> > > I have never ever seen that happen, or -probably more important- heard about
> > > such an event.

> > > > I'm very serious. This seems like a major flaw in Oracle that makes it
> > > > fundamentally unsuitable for mission critical OLTP applications. It means the
> > > > operations people cannot guarantee that they know what Oracle is doing at all
> > > > time and cannot guarantee that the application will continue to work as the
> > > > data changes.

> > > Well, who come there are some many of them around? All of these designers
> > > and users must have less high standard than you have.

> > This sounds like a typical scenario where Oracle is being used for purposes
> > that it hasn't in the past. Perhaps you've been working exclusively on
> > "traditional" database applications like data warehouses and environments with
> > very tightly controlled code and very rigorous testing regimen. However the
> > world has changed and Oracle is scrambling to catch up.

> > > > You make a fundamental assumption that the optimizer is infallible. Have you
> > > > actually tried to use the damned thing?

> > > No, I'm not. That's why hints were invented. And yes, I have used it.

> > > > It's very helpful for initially finding good query plans in development
> > > > but it's not appropriate to be using it at run-time.

> > > Why would that be?

> > So you don't think the plan stability features introduced in 8i are useful for
> > anything? If the optimizer changes behaviour it's infallibly because the new
> > plan is better than the old one? If a new query is introduced and it's checked
> > with one data set then it's undoubtedly going produce the same plan or better
> > on the production system?

> > Perhaps some concrete examples would be clearer:

> > I write a web site involving basic queries involving some joins etc. I test it
> > out on a sample database, I test it out on a QA system, including checking the
> > plans and disk i/o etc. It goes into production.

> > It gradually slows down as more users sign on, that's expected, but it's
> > basically satisfactory. I'm monitoring the response time planning to do some
> > tuning well before the response time becomes too slow. All of the sudden
> > everything grinds to a complete halt, Oracle has decided to start using full
> > table scans because the tables have passed some arbitrary size that the
> > optimizer decided was worth doing full table scans. It turns out Oracle was
> > overeager in switching to full table scans and they were actually slightly
> > slower. Had it stuck with the existing plans at least I could have continued
> > to monitor the response time and react in plenty of time to deal with the
> > performance degradation.

> > That scenario is handled by the plan stability features in 8i. However the
> > next scenario is what I'm actually complaining about:

> > Now I have one of the developers implement a new web page. I check the code
> > and test it on the QA system. We establish stored outlines for every query
> > with an acceptable execution plan with acceptable response times. However
> > either due to a fault in my testing procedure or perhaps due to some obscure
> > corner case that wasn't anticipated at all, there's a query in the new code
> > that I don't verify. It turns out that this query does a full table scan on a
> > large table. When this code is placed into production and receives hundreds of
> > hits per second that full table scan completely hammers Oracle saturating the
> > disk i/o and blocking every process doing the same query. The entire
> > application is killed by an execution plan that Oracle devised ad hoc without
> > having been approved.

> > This is the scenario I would like to guarantee won't occur. The development
> > people can do their job with testing but they can't guarantee that bugs will
> > never occur. But from an operations point of view we should be able to
> > guarantee that a bug causes an immediate error rather than having Oracle just
> > try to do its best to come up with an ad hoc execution plan and possibly kill
> > the entire application.

> > --
> > greg

--
---------------------------------------------------------------------
Chad Thompson,
Programmer Analyst: VB, VC++, PLSQL, Oracle HRMS, Security Apps

Thou shalt not tick off the dragon.... for thou art crunchy and
taste good with ketchup.
---------------------------------------------------------------------

  thomp901.vcf
< 1K Download
 
 
 

Strictly Enforcing Query Plan Stability

Post by Mark D Powel » Sun, 31 Dec 1899 09:00:00


Yes, I had lost track of the fact the poster had mentioned adhoc while
reading the responses.  Still I think the request is unreasonable. By
its very nature adhoc is unknown.  It is an unnecessary hinderance to
the users to make them predefine and clear their data retrieval needs
before they know them.  The database exists for the needs of the end-
user, not the DBA.

The IO pigs can be monitored and query plans added where missing or
necessary.

Still you posted an interesting solution.



> But Greg points out that ad-hoc is particularly the thing he wants
> to stop - any ad-hoc query which therefore does not have a pre-
> computed plan should fail at parse time.

> If this were my aim, I think I would go for a 3-tier architecture,
> using the 'trusted proxy user', so that end users never knew
> their own Oracle passwords, and certainly never learnt the
> password of the proxy (8.1 / OCI only feature though).

> You could try creating a view on the outln$ table that
> forced a once-per-execute call to a function that failed
> if no rows were returned.  Then the recursive SQL to
> parse the query would crash if no plan existed.

> --

> Jonathan Lewis
> Yet another Oracle-related web site:  http://www.jlcomp.demon.co.uk


> >The thread is on Greg's desire for the optimizer to reject all
queries
> >that do not have plan stability defined.

> >I would like to point out that request is not reasonable or well
> >considered because of something called adhoc!  If the end-user has
> >access to end-user reporting tools there is no way to antiscipate all
> >possible queries nor should you have to.  The CBO was invented to
> >handle adhoc queries.  For production code you should lock the path
in
> >using hints or the new query plan statbility feature, but even here
we
> >have found the CBO is usually good enough and we only tune what we
have
> >to.




> >> > I have a wishlist feature I would love Oracle to support. In fact
> >I'm
> >> > surprised Oracle doesn't support it already. Perhaps there's a
> >trick to
> >> > accomplish what I want and I just don't know it?

> >> > What I would like is to be able to set privileges such that my
> >application
> >> > cannot run _any_ query that doesn't have an outline already
stored
> >for it.
> >> > Essentially what I want is to be able to guarantee that no code
> >could possibly
> >> > go live without _every_ SQL query being analyzed and the plan
> >approved by a
> >> > DBA. Any unapproved query should immediately get an error, not be
> >run with
> >> > some ad hoc query plan that could very well bring the whole
> >application down.

> >> > It seems to me that anything less is simply inadequate for a
> >production
> >> > mission critical system. Given the types of applications that run
> >on Oracle
> >> > I'm surprised this isn't a fundamental feature of the system
since
> >day 1.

> >> > --
> >> > greg

> >> Easy enough to do...

> >> In its simplest form, users have 'create session' and a role which
> >lets
> >> them do whats needed in the applications.  The role is only enabled
> >when
> >> they launch the app (as protected by a role password).

> >> Then outside the app, the only thing that they can do is connect.
All
> >> you need do then to ensure that nothing is 'grant select to public'
> >and
> >> you're protected..

> >> HTH
> >> --
> >> ===========================================
> >> Connor McDonald
> >> http://www.oracledba.co.uk
> >> (faster/mirrored at http://www.oradba.freeserve.co.uk)

> >--
> >Mark D. Powell  -- The only advice that counts is the advice that
> > you follow so follow your own advice --

--
Mark D. Powell  -- The only advice that counts is the advice that
 you follow so follow your own advice --

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Mon, 30 Oct 2000 17:13:19



Quote:> Yes, I had lost track of the fact the poster had mentioned adhoc while
> reading the responses.  Still I think the request is unreasonable. By
> its very nature adhoc is unknown.  It is an unnecessary hinderance to
> the users to make them predefine and clear their data retrieval needs
> before they know them.  The database exists for the needs of the end-
> user, not the DBA.

Obviously you wouldn't apply such a feature to a database where ad hoc queries
were part of its intended use. But such a database would be a data warehouse
type database or a development environment. In a mission critical OLTP
database ad hoc queries are anathema.

In such a database it's inappropriate to have some cowboy (usually me, I
admit:) to be running ad hoc queries in the same database as the application
performing 100s of updates per second. The batch queries should be run on a
separate database where a sudden increase in i/o won't cripple the production
application.

Jonathan's solution is fairly interesting though. It's possible to do this at
the application layer even without a proxy. All queries are executed through a
driver interface, if that driver interface checks every query against a
database of approved queries then it could alert admins to new queries. That
database could even be stored in the Oracle database, but probably it would be
more reasonable to store them in an in-memory hash. Hmm.

--
greg

 
 
 

Strictly Enforcing Query Plan Stability

Post by Greg Star » Tue, 31 Oct 2000 00:56:00


So now I have an idea. What happens if you turn on CREATE_STORE_OUTLINES and
USE_STORED_OUTLINES but you put a triggger on OUTLN.OL$ that checks if the
current user is the user you want to restrict and throws an error?

--
greg

 
 
 

1. Wishlist for 7.4: Plan stability

Is someone keeping a "hopeful" todo list?

- Brandon

----------------------------------------------------------------------------
 c: 917-697-8665                                            h: 201-798-4983

---------------------------(end of broadcast)---------------------------

2. Migrating database (7.1.3 - 7.2.1)

3. ACCESS '97 / VB 5.0 : SERIOUS SECURITY PROBLEM or is it just me?

4. Why is Stored Proc plan slower than query plan

5. Network Down On Informix V.7.1

6. Execution plan,Query plan?

7. ADO Bug ?

8. Locking Plan part of Query Plan?

9. Stored Procedure Dynamic Queries and Query Plan

10. Query processor could not produce a query plan...

11. Error 8623 - query processor could not producde a query plan

12. Stored Procedure dynamic queries and query plan