Very Computer

I am participating in a project using a Compact Flash card as the
harddrive, and I am trying to understand the ramifications of this
decision (ugly pun intended.)

In particular,

A)  What power failure problems can be anticipated and how might we
    work around those,

B)  What wear-leveling actually does, and what does that mean
    for instance, to a program that tries to implement hard
    disk based governmental wipe (writing 22 bit patterns to
    each byte.)  Would such a program actually clear the right
    bytes, or worse case, might it simply write 22 new bytes
    while leaving the original data alone.

    How we determine, given a CF device the consumer has put in,
    what wear-leveling has been implemented?

    And is there anyway to command the CF to write specific
    bytes regardless of wear-leveling algorithms.

C)  What wear-leveling actually does, and what does that mean
    when using Postgres, MySQL, or even Oracle (programs that
    like to write logs)

Are there any good Compact Flash tutorials out there?

Thanks,

Jerry

This gets discussed every month or so.
And I've yet to see anyone come to a 100% certain conclusion on any of your questions.
Take a look through the archives, and perhaps you'll be able to find out something
everyone else hasn't.
(and then write a good Compact Flash tutorial :)

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SnapGear ---                           ph:         +61 7 3435 2809
  | Custom Embedded Solutions          fax:         +61 7 3891 3630
  | and Security Appliances            web: http://www.snapgear.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ---  Free Embedded Linux Distro at   http://www.snapgear.org  ---

It seems completely undefined how a FC internally works.

So you can't relay on anything regarding wear leveling or any other of
your questions.

It even seems to be possible to destroy an FC just by removing the power
when a write has just occurred and is not ready internally.

-Michael

Quote:> I am participating in a project using a Compact Flash card as the
> harddrive, and I am trying to understand the ramifications of this
> decision (ugly pun intended.)

> In particular,

> A)  What power failure problems can be anticipated and how might we
>     work around those,

The only thing you can do is tie a big capacitor to the power line of the
flash, and use a power-fail chip to pull reset when the power fails. The
capacitor should be dimentioned to allow a worst-case write to complete.

The problem with losing power at the wrong time isn't as much that a data
sector will be partially written, but that any wear levelling map might be
partially written. This makes the CF useless.

Quote:

> B)  What wear-leveling actually does, and what does that mean
>     for instance, to a program that tries to implement hard
>     disk based governmental wipe (writing 22 bit patterns to
>     each byte.)  Would such a program actually clear the right
>     bytes, or worse case, might it simply write 22 new bytes
>     while leaving the original data alone.

There's a pretty good chance that the only safe destriction is to
burn/crush/physically destroy the chip.

Quote:

>     How we determine, given a CF device the consumer has put in,
>     what wear-leveling has been implemented?

>     And is there anyway to command the CF to write specific
>     bytes regardless of wear-leveling algorithms.

> C)  What wear-leveling actually does, and what does that mean
>     when using Postgres, MySQL, or even Oracle (programs that
>     like to write logs)

Wear-leveling could be as simple as just maintaining a redirect map of
sectors that start to take a long time to write to, or a full-blown
virtualisation map where each write will be directed to a different sector.

Quote:> Are there any good Compact Flash tutorials out there?

> Thanks,

> Jerry

That was my conclusion back when I was researching the topic.

Quote:> So you can't relay on anything regarding wear leveling or any other of your
> questions.

Exactly.  None of the manufacturers would spec any sort of wear-levelling or
power-down behavior.

Quote:> It even seems to be possible to destroy an FC just by removing the power
> when a write has just occurred and is not ready internally.

That's what I've read, though I haven't experienced it.

The number of sectors in a specified size of CF cards also vary rather
widely.  256MB usually means slight more than 256x10^6, but you never know
how much more.

--
Grant Edwards                   grante             Yow!  Where's th' DAFFY
                                  at               DUCK EXHIBIT??
                               visi.com            

Agree, don't base your design (or bet your job) on wear-levelling.

Quote:

> > It even seems to be possible to destroy an FC just by removing the power
> > when a write has just occurred and is not ready internally.

> That's what I've read, though I haven't experienced it.

> The number of sectors in a specified size of CF cards also vary rather
> widely.  256MB usually means slight more than 256x10^6, but you never know
> how much more.

Actually, it should be 2^28 physical cells, if they are telling the
truth.  What you see are usable cells.  However, it depends on how
many replacement sectors are available, and how many are already used.

Quote:>> The number of sectors in a specified size of CF cards also vary rather
>> widely.  256MB usually means slight more than 256x10^6, but you never know
>> how much more.

> Actually, it should be 2^28 physical cells, if they are telling the truth.

Depends on the definiton of "MB".

Quote:> What you see are usable cells.  However, it depends on how many replacement
> sectors are available, and how many are already used.

When they CF vendors say "256MB" they mean 256x10^6 not 256x2^20.  None of
the 256MB CF cards I tested had 256x2^20 bytes of usable storage. They've
followed in the disk-drive manufacturers weasly useage of the term
"megabyte".

--
Grant Edwards                   grante             Yow!  Where's th' DAFFY
                                  at               DUCK EXHIBIT??
                               visi.com            

Good questions.
I have an embedded app using a 2.4.18 kernal and slimmed down ext2 RFS
on a 64MB SanDisk. During development I had cycled power dozens of times
on the system without any obvious file corruption - it always booted and
ran at least. When I finally read about ext3 and JFFS2, I realized that
just flipping the power switch on ext2 was probably a bad idea. I ran
e2fsck and it found lots to fix, but nothing it could not handle.
Whatever SanDisk is doing inside the card it can't be too bad, but my
app does very little file IO.

Recently one of the flash cards on a duplicate system (with the same
power off methodology) failed to boot, but could be reformatted without
any obvious damage or loss of storage (it could have taken out some
extra blocks without me knowing I suppose).

More questions:
When faced with unknown wear leveling, does ext3 or JFFS2 add any value
in a CF configuration?

Has anyone implemented the 'hang a big cap off the flash' solution?
If you are designing from scratch this may work, but probably not as a
retrofit for existing boards.

 From what I've read I believe JFFS2 is not adequate for CF, since most
CF cards do internal wear levelling. Read prior posts to this group for
details.

Are you using industrial Sandisk or consumer grade ? I've always
wondered if industrial is better. I also have some of the same problems
you have, and I'm about to try industrial grade. As for ext3, I'm also
going to try it.

I am not sure how you tell. The disks I have been working with were
ordered thru a board vendor & were supposed to be validated somehow for
their product. They look different from what is available in stores,
since they have a large black bar code on the back and a date 5/15/02. A
disk I recently picked up at CompUSA had a different color scheme and
only had markings on the edge of the card.
This new disk has been working for more than a month in the app, but has
not been subjected to repeated power outages. When I emailed SanDisk
about possible quality differences between these disks, they said they
had changed the marking scheme, and made no mention of availability of
different grades. They gave me the impression that newer stock would be
of better quality than something purchased a year ago.

If the FS has no control or knowledge of wear-leveling, what advantage
do you think ext3 will provide as opposed to JFFS2? If the CF contains a
dedicated controller, it makes sense for it to implement a powerfail
scheme of its own, although a system level solution (UPS, supercap)
rather than a 'media level' solution would seem more promising.

Is Industrial grade the same as Utility grade ?
I just did a heap of speed tests on different CF cards, and Sandisk Utility grade
performed exactly the same as Sandisk Ultra.  They were orders of magnitude faster
than some of the generic brands.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SnapGear - A *Guard Company ---    ph:         +61 7 3435 2809
  | Custom Embedded Solutions          fax:         +61 7 3891 3630
  | and Security Appliances            web: http://www.veryComputer.com/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ---  Free Embedded Linux Distro at   http://www.veryComputer.com/  ---