> I'm currently planning compliance tests according to ISO/IEC 10918-2.
> I've noticed that the latter mentions a data set for the compliance test.
> Is this available on line or I must contact ISO (probably slow) ?
It's not available on-line, and I'm not sure that ISO ever even released
it officially. The fact of the matter is that the draft 10918-2 dataset
is almost valueless as a test set; you can accomplish more, more easily,
by cross-testing with any well-recognized implementation. (For example,
the free IJG code.)
Let's see, where did I put that screed ... ah, here it is...
10918-2 is completely useless for testing most production codecs anyway,
because it consists of random-noise data in a meaningless four-component
colorspace with randomly chosen sampling factors. Most of the codecs
I know about include colorspace conversion and upsample/downsample logic
and will not work on this data. If you can separate out those parts of
your codec and feed it raw subsampled data then you might have some
chance of using the 10918-2 data. It's still a pretty unfriendly test
set because it's a random-noise image --- you have no chance of getting
a go/no go indication by eyeball, you have to resort to nontrivial
statistical analysis to see whether your codec works at all! (Worse,
since your codec's roundoff errors are doubtless different from anyone
else's, you can't just compare output bits...)
And on top of that, the data set consists of one example of each of the
JPEG-defined processes. IIRC, there are only about two datastreams that
a standard baseline codec has any hope of reading. So it tells you
nothing at all about the robustness of your codec WRT real-world
variations in marker layout, restart markers, etc.
I'd suggest cross-testing with existing applications as a far more
useful and practical validation procedure than 10918-2. (The IJG code
is intended in part as a freely available reference codec for such
testing.) If you want to test the accuracy of your math, measuring the
degradation of an image over multiple compression cycles with a fixed
quality setting is a pretty sensitive check.
You can try the folks at www.jpeg.org if you want a more official
answer, but my opinion is that 10918-2 is a waste of time.
regards, tom lane
organizer, Independent JPEG Group