Article: Code Verification

Article: Code Verification

Post by Lidor Wyssock » Sat, 12 Jul 2003 03:24:26



Hi,

I would like to refer you to an article dealing with the process of Code
Verification. The article presents the goals of code verification as well as
some guidelines for an effective code verification.

I would be happy to exchange some opinions on this subject....

The article is available at:
http://www.qualityprogramming.org/Implementation/CodeVerification/Cod...
cation.htm

                                           Thanks,
                                                   Lidor Wyssocky

____________________________
Lidor Wyssocky

http://www.QualityProgramming.org
Quality Oriented Programming

 
 
 

Article: Code Verification

Post by JXSter » Sat, 12 Jul 2003 03:06:33


On Thu, 10 Jul 2003 20:24:26 +0200, "Lidor Wyssocky"
...

Quote:>http://www.qualityprogramming.org/Implementation/CodeVerification/Cod...

"The source code is the only medium future developers will be able to
use to interact with the system when extending it or fixing it."

No, there can be all sorts of UML diagrams, requirements narratives,
etc.

"Having such an important rule in the development of a software
product, the code has to be somehow verified. The most common approach
in regard to verifying source code is testing its functionality."

No, it doesn't have to be verified, and often isn't.

Testing is not verification of code, only of results.  Famously,
verifying any finite sequence is not a guaranty of the next entry in
the sequence. 1,2,3, ... may have been produced by the equation:

y = x + (x - 1) * (x - 2) * (x - 3)

--

Didn't read any further.

J.

 
 
 

Article: Code Verification

Post by Lidor Wyssock » Sun, 13 Jul 2003 01:24:03



> On Thu, 10 Jul 2003 20:24:26 +0200, "Lidor Wyssocky"

> ...

>http://www.qualityprogramming.org/Implementation/CodeVerification/Cod...
ication.htm

> "The source code is the only medium future developers will be able to
> use to interact with the system when extending it or fixing it."

> No, there can be all sorts of UML diagrams, requirements narratives,
> etc.

Assuming you keep updating them as the project advances and changes.... (do
you?)

Quote:

> "Having such an important rule in the development of a software
> product, the code has to be somehow verified. The most common approach
> in regard to verifying source code is testing its functionality."

> No, it doesn't have to be verified, and often isn't.

> Testing is not verification of code, only of results.  Famously,
> verifying any finite sequence is not a guaranty of the next entry in
> the sequence. 1,2,3, ... may have been produced by the equation:

> y = x + (x - 1) * (x - 2) * (x - 3)

> --

> Didn't read any further.

If you had read any further you would notices that the article indeed says
testing is veryfing only the functionality of the binary product iteself and
not the code. Thus, the article suggest formal code reviews. As for the
necessity of code verification, I understand you believe a quality product
is judget merely by its functional behavior. I claim this is not true. If in
a year no one in your company can maintain the code, easily locate problems
or make adjustments the customer might want - your product is of poor
quality. This kind of stuff can't be verified with functional testsing.

                    Lidor Wyssocky

--
____________________________
Lidor Wyssocky

http://www.QualityProgramming.org
Quality Oriented Programming

- Show quoted text -

Quote:

> J.

 
 
 

Article: Code Verification

Post by JXSter » Sun, 13 Jul 2003 03:14:51


On Fri, 11 Jul 2003 18:24:03 +0200, "Lidor Wyssocky"


>> "The source code is the only medium future developers will be able to
>> use to interact with the system when extending it or fixing it."

>> No, there can be all sorts of UML diagrams, requirements narratives,
>> etc.

>Assuming you keep updating them as the project advances and changes.... (do
>you?)

Well me, yes, I do, the narratives above all.  But even when I or
others do not, old and obsolete docs are often very useful in figuring
out why code was written the way it is.

Quote:>> No, it doesn't have to be verified, and often isn't.

>> Testing is not verification of code, only of results.  Famously,
>> verifying any finite sequence is not a guaranty of the next entry in
>> the sequence. 1,2,3, ... may have been produced by the equation:

>> y = x + (x - 1) * (x - 2) * (x - 3)
>If you had read any further you would notices that the article indeed says
>testing is veryfing only the functionality of the binary product iteself and
>not the code.

But the worst (?) case is that there wasn't any verification by
testing, either.  Then it is even more important to verify, validate,
and armor-plate code, one way or another -- or all ways possible.

Quote:> Thus, the article suggest formal code reviews.

Seems like a good idea but in twenty years I've never seen it done.

Quote:>As for the
>necessity of code verification, I understand you believe a quality product
>is judget merely by its functional behavior.

Well, you're not so far wrong on that.  The ultimate judgement *is*
its external behavior, how could it not be?

Quote:> I claim this is not true.

<cough>

Quote:>If in
>a year no one in your company can maintain the code, easily locate problems
>or make adjustments the customer might want - your product is of poor
>quality. This kind of stuff can't be verified with functional testsing.

If in a day it produces no results, I don't care what happens in a
year.  Many negative results CAN be verified with functional testing.
Normal operation is a form of functional testing.

My favorite approaches to code verification include assertions,
invariants, and extensive error handling.  And coding conventions and
design rules.  Only somewhere south of that do we get to formal code
reviews, and somewhere south of THAT do we find formal proofs of
correctness.

J.

 
 
 

Article: Code Verification

Post by Lidor Wyssock » Sun, 13 Jul 2003 16:01:11



> On Fri, 11 Jul 2003 18:24:03 +0200, "Lidor Wyssocky"

> >> "The source code is the only medium future developers will be able to
> >> use to interact with the system when extending it or fixing it."

> >> No, there can be all sorts of UML diagrams, requirements narratives,
> >> etc.

> >Assuming you keep updating them as the project advances and changes....
(do
> >you?)

> Well me, yes, I do, the narratives above all.  But even when I or
> others do not, old and obsolete docs are often very useful in figuring
> out why code was written the way it is.

> >> No, it doesn't have to be verified, and often isn't.

> >> Testing is not verification of code, only of results.  Famously,
> >> verifying any finite sequence is not a guaranty of the next entry in
> >> the sequence. 1,2,3, ... may have been produced by the equation:

> >> y = x + (x - 1) * (x - 2) * (x - 3)

> >If you had read any further you would notices that the article indeed
says
> >testing is veryfing only the functionality of the binary product iteself
and
> >not the code.

> But the worst (?) case is that there wasn't any verification by
> testing, either.  Then it is even more important to verify, validate,
> and armor-plate code, one way or another -- or all ways possible.

> > Thus, the article suggest formal code reviews.

> Seems like a good idea but in twenty years I've never seen it done.

The fact that it isn't common, doesn't suggest it is not necessary....

- Show quoted text -

Quote:

> >As for the
> >necessity of code verification, I understand you believe a quality
product
> >is judget merely by its functional behavior.

> Well, you're not so far wrong on that.  The ultimate judgement *is*
> its external behavior, how could it not be?

> > I claim this is not true.

> <cough>

> >If in
> >a year no one in your company can maintain the code, easily locate
problems
> >or make adjustments the customer might want - your product is of poor
> >quality. This kind of stuff can't be verified with functional testsing.

> If in a day it produces no results, I don't care what happens in a
> year.  Many negative results CAN be verified with functional testing.
> Normal operation is a form of functional testing.

> My favorite approaches to code verification include assertions,
> invariants, and extensive error handling.  And coding conventions and
> design rules.  Only somewhere south of that do we get to formal code
> reviews, and somewhere south of THAT do we find formal proofs of
> correctness.

I never claimed functionality is not the first thing that should be
verified. I merely suggest it is not enough.
When you work alone it might be good enough. But the code you write today
will have to be maintained by others, don't you want to be sure those other
won't have a hard time dealing with? Who gets to ensure that every developer
involved in the product indeed used the coding standards? Who gets to verify
that the correct design rules were applied? Functional tests can't do that!

Didn't you ever had to maintain a piece of code you didn't write and just
knew that no one apart from the person who wrote it ever checked how this
code was written (and even this person himself won't easily understand this
code today).

Now, some people will say: if you are a good developer there is no code you
can't eventually follow and understand. That might be true, but why making
life harder than it is? Why take a chance of missing something in the code?
Wouldn't it be easier to verify that the code is clean, clear,
understandable and well designed *before* releasing the product? (In
addition to functional tests of course...)

Lidor

--
____________________________
Lidor Wyssocky

http://www.QualityProgramming.org
Quality Oriented Programming

- Show quoted text -

Quote:> J.

 
 
 

Article: Code Verification

Post by JXSter » Mon, 14 Jul 2003 00:13:45


On Sat, 12 Jul 2003 09:01:11 +0200, "Lidor Wyssocky"


>> > Thus, the article suggest formal code reviews.

>> Seems like a good idea but in twenty years I've never seen it done.

>The fact that it isn't common, doesn't suggest it is not necessary....

It _does_ suggest that it is not _necessary_, although it may well be
a good idea.  I was just about to start such a procedure, on a project
I was on a year or so ago, because too many people were turning in
crap, and I wanted a social mechanism to focus their attention on
quality.  But that's code review as an instructional and social trick,
not technical. Plus, it never happened -- project was, um, reorganized
preparatory to cancellation.

Quote:>> My favorite approaches to code verification include assertions,
>> invariants, and extensive error handling.  And coding conventions and
>> design rules.  Only somewhere south of that do we get to formal code
>> reviews, and somewhere south of THAT do we find formal proofs of
>> correctness.

>I never claimed functionality is not the first thing that should be
>verified. I merely suggest it is not enough.

I agree that more can and should be done, as I say above.

Quote:>When you work alone it might be good enough. But the code you write today
>will have to be maintained by others, don't you want to be sure those other
>won't have a hard time dealing with? Who gets to ensure that every developer
>involved in the product indeed used the coding standards? Who gets to verify
>that the correct design rules were applied? Functional tests can't do that!

If it passes the functional tests, I'm willing to let the next guy in
to do maintenance, note of the code was written to spec.  That, and a
few informal inspections by the project lead, are enough to turn up
problems before they go very far.

Quote:>Didn't you ever had to maintain a piece of code you didn't write and just
>knew that no one apart from the person who wrote it ever checked how this
>code was written (and even this person himself won't easily understand this
>code today).

I've seen it all.

Quote:>Now, some people will say: if you are a good developer there is no code you
>can't eventually follow and understand. That might be true, but why making
>life harder than it is? Why take a chance of missing something in the code?
>Wouldn't it be easier to verify that the code is clean, clear,
>understandable and well designed *before* releasing the product? (In
>addition to functional tests of course...)

If all a code review does is check adherence to standards, it is not
worth the time to get an entire group together.  If it has to actually
review the logic in detail, it is far too slow a process. 90%+ of code
is just cloning a skeleton and filling in details, there shouldn't
really be anything to review.  The other 10% tends to be one-time
code, and the purpose of the review should be to look for gross
errors, not fine points.  And review is very expensive.  Yeah, errors
are even more expensive, I know, but the temptation to skip the
process and get lucky, is obviously very strong.

--

Hey, I just about convinced myself of something here, sorry if it's in
the opposite direction you were trying to go.

Joshua Stern

 
 
 

Article: Code Verification

Post by Lidor Wyssock » Mon, 14 Jul 2003 05:59:28



> On Sat, 12 Jul 2003 09:01:11 +0200, "Lidor Wyssocky"

> >> > Thus, the article suggest formal code reviews.

> >> Seems like a good idea but in twenty years I've never seen it done.

> >The fact that it isn't common, doesn't suggest it is not necessary....

> It _does_ suggest that it is not _necessary_, although it may well be
> a good idea.  I was just about to start such a procedure, on a project
> I was on a year or so ago, because too many people were turning in
> crap, and I wanted a social mechanism to focus their attention on
> quality.  But that's code review as an instructional and social trick,
> not technical. Plus, it never happened -- project was, um, reorganized
> preparatory to cancellation.

What you are claiming is that most of the products our industry produces is
of good quality. I think many (developers and customers) disagree with that.
So the fact that something is not being done (e.g. code verification) does
not suggest it is not necessary. It isn't necessary only if most code would
be good to begin with.

Quote:

> >> My favorite approaches to code verification include assertions,
> >> invariants, and extensive error handling.  And coding conventions and
> >> design rules.  Only somewhere south of that do we get to formal code
> >> reviews, and somewhere south of THAT do we find formal proofs of
> >> correctness.

> >I never claimed functionality is not the first thing that should be
> >verified. I merely suggest it is not enough.

> I agree that more can and should be done, as I say above.

> >When you work alone it might be good enough. But the code you write today
> >will have to be maintained by others, don't you want to be sure those
other
> >won't have a hard time dealing with? Who gets to ensure that every
developer
> >involved in the product indeed used the coding standards? Who gets to
verify
> >that the correct design rules were applied? Functional tests can't do
that!

> If it passes the functional tests, I'm willing to let the next guy in
> to do maintenance, note of the code was written to spec.  That, and a
> few informal inspections by the project lead, are enough to turn up
> problems before they go very far.

The question is will a few informal inspection do the trick when everyone is
on a tight schedule? If this process is a formal one (part of teh dev
process) it can't skipped. I do agree that on the long run this process also
has some educational goals.

Quote:

> >Didn't you ever had to maintain a piece of code you didn't write and just
> >knew that no one apart from the person who wrote it ever checked how this
> >code was written (and even this person himself won't easily understand
this
> >code today).

> I've seen it all.

And are you happy with what you saw...?

- Show quoted text -

Quote:

> >Now, some people will say: if you are a good developer there is no code
you
> >can't eventually follow and understand. That might be true, but why
making
> >life harder than it is? Why take a chance of missing something in the
code?
> >Wouldn't it be easier to verify that the code is clean, clear,
> >understandable and well designed *before* releasing the product? (In
> >addition to functional tests of course...)

> If all a code review does is check adherence to standards, it is not
> worth the time to get an entire group together.  If it has to actually
> review the logic in detail, it is far too slow a process. 90%+ of code
> is just cloning a skeleton and filling in details, there shouldn't
> really be anything to review.  The other 10% tends to be one-time
> code, and the purpose of the review should be to look for gross
> errors, not fine points.  And review is very expensive.  Yeah, errors
> are even more expensive, I know, but the temptation to skip the
> process and get lucky, is obviously very strong.

If you has read the rest of the article you would have noticed that I don't
suggest code review to be done by the entire team (nor be any one specific
from the team).

Reviewing compliance to code standards could be done automatically. But
verifying the clearness of the code, and verfying some design and
implementation decisions done by the developer could not be done
automatically. Letting one person (not from the team) review for a couple of
hours a code written during a week does not have such a great cost. The cost
of having to deal such design mistakes or merely bad code a couple of month
in the future is far more great.

Quote:> --

> Hey, I just about convinced myself of something here, sorry if it's in
> the opposite direction you were trying to go.

No need to be sorry. There's nothing like a good discussion...

Lidor

--
____________________________
Lidor Wyssocky

http://www.QualityProgramming.org
Quality Oriented Programming

- Show quoted text -

Quote:

> Joshua Stern

 
 
 

Article: Code Verification

Post by JXSter » Mon, 14 Jul 2003 05:10:30


On Sat, 12 Jul 2003 22:59:28 +0200, "Lidor Wyssocky"


>> >The fact that it isn't common, doesn't suggest it is not necessary....

>> It _does_ suggest that it is not _necessary_, although it may well be
>> a good idea.  I was just about to start such a procedure, on a project
>> I was on a year or so ago, because too many people were turning in
>> crap, and I wanted a social mechanism to focus their attention on
>> quality.  But that's code review as an instructional and social trick,
>> not technical. Plus, it never happened -- project was, um, reorganized
>> preparatory to cancellation.

>What you are claiming is that most of the products our industry produces is
>of good quality. I think many (developers and customers) disagree with that.
>So the fact that something is not being done (e.g. code verification) does
>not suggest it is not necessary. It isn't necessary only if most code would
>be good to begin with.

Au contraire, you are claiming that good quality is necesary.  I never
said that!  Obviously, good quality is NOT necesary.  Too dang bad for
all of us, but there it is.  

I'm sure code reviews would help, but they're low on my list, and very
expensive, that's all I've said.

Quote:>The question is will a few informal inspection do the trick when everyone is
>on a tight schedule? If this process is a formal one (part of teh dev
>process) it can't skipped. I do agree that on the long run this process also
>has some educational goals.

When on a tight schedule, many regular things get skipped, and I
daresay code inspections would be the first to go.  And this would be
the right thing to do, if my other suggested steps were still present.

Quote:>> I've seen it all.

>And are you happy with what you saw...?

Heaven's no.

Quote:>If you has read the rest of the article you would have noticed that I don't
>suggest code review to be done by the entire team (nor be any one specific
>from the team).

Well you've got me there.

Quote:>Reviewing compliance to code standards could be done automatically. But
>verifying the clearness of the code, and verfying some design and
>implementation decisions done by the developer could not be done
>automatically. Letting one person (not from the team) review for a couple of
>hours a code written during a week does not have such a great cost. The cost
>of having to deal such design mistakes or merely bad code a couple of month
>in the future is far more great.

I dunno ... I guess I will have to read the rest of your article
before responding on this point.

J.

 
 
 

Article: Code Verification

Post by Tokoma » Mon, 14 Jul 2003 06:57:53



> Au contraire, you are claiming that good quality is necesary.  I never
> said that!  Obviously, good quality is NOT necesary.  Too dang bad for
> all of us, but there it is.  

Good quality is a perfectly flat floor.

Quote:> I'm sure code reviews would help, but they're low on my list, and very
> expensive, that's all I've said.

They are less expensive if everyone takes them at their own pace.

Quote:> When on a tight schedule, many regular things get skipped, and I
> daresay code inspections would be the first to go.  And this would be
> the right thing to do, if my other suggested steps were still present.

Then make simple things more regular.

I read and write prose every day to colleagues.

--
Tokomar

 
 
 

Article: Code Verification

Post by taas.. » Tue, 15 Jul 2003 22:43:49


[snip]
Quote:> If all a code review does is check adherence to standards, it is not
> worth the time to get an entire group together.  If it has to actually
> review the logic in detail, it is far too slow a process. 90%+ of code
> is just cloning a skeleton and filling in details, there shouldn't
> really be anything to review.  The other 10% tends to be one-time
> code, and the purpose of the review should be to look for gross
> errors, not fine points.  And review is very expensive.  Yeah, errors
> are even more expensive, I know, but the temptation to skip the
> process and get lucky, is obviously very strong.

[snip]

I'm curious.  Are you serious about this statement: "90%+ of code is
just cloning a skeleton and filling in details..."?

If so, you should really read "The Pragmatic Programmer: From Journeyman
to Master" by Andrew Hunt and David Thomas (ISBN: 020161622X).
Especially the part concerning the DRY (Don't Repeat Yourself)
principle.  They explain using meta data to fill in the details.

Also, what kind of software do you develop where 90%+ is what sounds to
be highly repetitive?

Tad

 
 
 

Article: Code Verification

Post by JXSter » Wed, 16 Jul 2003 01:13:17



Quote:>> If all a code review does is check adherence to standards, it is not
>> worth the time to get an entire group together.  If it has to actually
>> review the logic in detail, it is far too slow a process. 90%+ of code
>> is just cloning a skeleton and filling in details, there shouldn't
>> really be anything to review.  The other 10% tends to be one-time
>> code, and the purpose of the review should be to look for gross
>> errors, not fine points.  And review is very expensive.  Yeah, errors
>> are even more expensive, I know, but the temptation to skip the
>> process and get lucky, is obviously very strong.
>[snip]

>I'm curious.  Are you serious about this statement: "90%+ of code is
>just cloning a skeleton and filling in details..."?

Yes, sorry, I seem to have left out a necessary qualifier -- in
commercial applications, 90%+ of code is (often) just cloning.

Quote:>If so, you should really read "The Pragmatic Programmer: From Journeyman
>to Master" by Andrew Hunt and David Thomas (ISBN: 020161622X).
>Especially the part concerning the DRY (Don't Repeat Yourself)
>principle.  They explain using meta data to fill in the details.

Yes, I am a big fan of metadata and have introduced exactly this
approach on several projects.  That is the thrust of the current
design trope using XML and XSLT as an application architecture (though
whether it effectively achieves it is another questions!)

Unfortunately, when such meta-facilities have to be built by hand,
they often are cost-ineffective compared to some degree of cloning, at
least cost-ineffective in the short term.  It is a matter of art to
decide what should be cloned, what factored into a framework, and what
driven by metadata.

In practice, in MRP systems and such, the framework and mechanics get
fixed early on, and the rest of the work tends to be either analysis,
or rather straightforward coding.  

Quote:>Also, what kind of software do you develop where 90%+ is what sounds to
>be highly repetitive?

How's that? :)

Joshua Stern

 
 
 

Article: Code Verification

Post by Univers » Wed, 16 Jul 2003 04:48:37



> ....It is a matter of art to
> decide what should be cloned, what factored into a framework, and what
> driven by metadata.

Art<=>Science

Art and science in dialectical interaction.

For some perhaps mainly art, but they are using *some* prior knowlegde
from books, articles, others experience summarized logically over time,
their own experience summarized logically over time, etc.  *That* is
*science*.

Elliott
--
OO software rests upon class abstractions expressed in class method
*behaviors*.  The sum of class method behaviors is the overall class
*role*.  Class role should have primary responsibility for managing class
data such that the impact of class data is driven by the operation of
class methods.
         ~*~   Get with OO fundamentals, get OO.  ~*~
-
      We Don't Need US Fat Cat's Geopolitical Hegemony!
           People of the World Demand US Stop Now!
-
               * http:\\www.radix.net/~universe *

 
 
 

Article: Code Verification

Post by JXSter » Wed, 16 Jul 2003 07:49:23



Quote:>Art<=>Science

>Art and science in dialectical interaction.

>For some perhaps mainly art, but they are using *some* prior knowlegde
>from books, articles, others experience summarized logically over time,
>their own experience summarized logically over time, etc.  *That* is
>*science*.

Well yeah, sure, but if I wanted to get you all e*d, I would
suggest that there may be a wide range of answers in the acceptable
range, that can be accessed in either way.

You seem to suggest (tell me if this is accurate), that somehow art is
necessarily less rigorous than science.  It's an old debate whether an
artiste needs or benefits from formal education or not.

I'm firmly on both sides of this issue.

J.

 
 
 

1. How Can I preserve the article of older MSDN articles?

I've recently received the January 2000 Issue of the MSDN CD.

For a beginner like me, it is very valuable information on all different
areas of computing.
So it got me thinking, what if I install each later release, and lose the
initial articles?

In other words, what do I do if I wish to preserve incrementally the
articles in the previous versions of MSDN ?

I would like to have them all for learning and reference. Without having to
re-install an older version, just to check out its contents.

I hope I've put it clearly.

Thank you in advance for your responses. : )

2. emTeX problem: driver cannot find fonts

3. Where to get the VC++ version source code of David George's article on MSDN

4. Can't get 11x17 figure to print from .m file

5. No Code Generated ?? Just lots of duplicate articles!!!

6. Cannot send email........but can receive

7. Good source code, articles and tutorials...

8. how to select a *directory*

9. Article ANN: Working Effectively with Legacy Code

10. Example code for RCM's Taskmaster article