Re: [ECOS] NAND flash and JFFS2

[Jonathan Larmour <jifl at eCosCentric dot com>]

Rutger Hofman wrote:
> Is this the right way to go:
>
>  = for our own system make/find a NAND flash device driver (we are
> switching from XScale to BlackFin BF547 and BF537, not least because of
> Marvell's defensive attitude in releasing its documentation). (Yes, we
> base our work on the eCos BlackFin port by Andre Liesk, University of
> Chemnitz.)
> 
>  = do I understand correctly that recent MDT/JFFS2 does the necessary
> work to have good support for generic NAND flash?

Yes and no. Certainly JFFS2 needs updating from upstream as a very first stage.

Linux MTD is fully GPL'd. As such it isn't very useful to eCos in practice
given the eCos licence is written with a special exception to avoid forcing
people to release all their application code. Use of vanilla GPL code in
eCos would oblige everyone to provide all code, including their application.

Instead eCos needs its own NAND Flash layer to perform the same sort of
things. I've included some excerpts from an internal proposal I wrote at
the bottom of this mail to give some pointers as to what is required. At
worst this should help anyone thinking about implementing it to see what's
needed so that it can be done The Right Way :-).

>  = so, then we must merge in a recent version of upstream MDT/JFFS2.
> The eCos JFFS2 ChangeLog has this as its most recent entries on MTD:
>     > 2005-07-30  Andrew Lunn  <andrew.lunn@ascom.ch>
>     >
>     >         * Merge to public MTD
> 
>     > 2005-01-22  Andrew Lunn  <andrew.lunn@ascom.ch>
>     >
>     >         * Merge from public MTD.
> 
>    How complicated is this going to be?
>     - I guess upstream JFFS2 is to do with linux-mtd git. I'll have to
> get familiar with that from the ground up.
>     - Would it be as easy as
>         git diff -date 2005-07-30 -date now | patch eCos JFF2?
>       My guess is, not at all. How can I find out at what versions of
> git and eCos CVS the various branches up/down occurred?

Not quite. If you look at http://git.infradead.org/ you'll see under
users/dwmw2 that there is a jffs2-ecos.git and jffs2-ecos-core.git
repository. The eCos parts need bringing up-to-date with the changes in the
core JFFS2 files, and bringing over any bug fixes and improvements from the
similar code in the Linux equivalents.

It's on my todo list, but then it's been on my to-do list for at least the
last year if not two, and I don't see any chance of that changing imminently.

>  = Or should we switch to YAFFS? It seems to have been ported to eCos,
> and it claims it is really, really easy to port anyway.

That is also fully GPL'd. I'm also not entirely sure the eCos port is
anything other than vapourware either, unfortunately.

Here's the proposal document I referred to:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Background
----------

First some background, JFFS2 does work with NAND on Linux. However the
majority of the real _work_ for NAND support is done in the MTD layer. That
layer, is very associated with JFFS2, and lives in the same source
repository and everything. However it is fully GPLd. And so to all intents
and purposes, for us it is unusable.

NAND differences
----------------

Why so much effort? NAND is very different from NOR flash really.

- Access is indirect, and via a special write buffer. Access to the buffer
is fast, but then you have to wait for the operation to complete. Although
NAND erases are faster than NOR erases.
- Page sizes are small.
- It is inherently unreliable. Bad blocks are common. So common that
manufacturers often have a factory-set bad block map.
- Failures in use are also common, so NAND has a "spare area" for
out-of-band (OOB) data. This is then used for ECCs. Some devices have
hardware ECC generation, some don't. Other OOB data can be used by higher
layers, e.g. for filesystem metadata.
- You cannot write to a sector over and over again, clearing bits, like you
can with NOR.
- It does not erase to all 1s.
- It can be bigger. So much bigger that it can expose lack of scalability.
This is a problem in JFFS2, and has only partially been addressed in recent
sources. Nothing we can do about that.

JFFS2 itself
------------

JFFS2 does need to behave very differently if it's layered over NAND. Not
everything is abstracted in the MTD layer. One particular aspect is write
buffering: you cannot write a NAND flash page as many times as you want:
you are only allowed to write 1-3 times (depending on the manufacturer),
including to the OOB area. The eCos JFFS2 port, being for NOR flash, has no
support for write buffering, and the Linux support is quite dependent on
Linux kernel inode and VFS knowledge. That's unfortunate as that will
require awkward porting.

JFFS2 also stores its own fs metadata information in the OOB data, such as
clean markers. Hopefully the existing Linux code will port quite easily.

Almost all the necessary code to port is what is in Linux's wbuf.c.

There are other more minor dependencies in various other places in JFFS2,
often to do with things like (flash) page alignment. Hopefully this should
in general work, but care is required.

The MTD substitute - the eCos NAND layer
----------------------------------------

Here's the functionality the eCos NAND layer (substituting for MTD) must be
able to cover:

* bad block management
  - Whether factory-marked in the flash, user marked in flash, or user
provided in some other way

* software ECC generation
  - if no hardware ECC available

* hardware ECC management
  - device dependent clearly, but we must have infrastructure support.

* OOB layout
  - due to hardware ECC, OOB layout is not fixed and varies between flash
parts. Therefore this has to be managed and abstracted for upper layers
such as JFFS2 (which needs to know the correct OOB locations for cleanmarkers)

* provide a well-defined interface to all the above to higher layers
 - Clearly it would be desirable if the eCos NAND layer is separate from
JFFS2 (just like the real MTD is), and so can be used directly or from
other filesystems than JFFS2, rather than being tied to it. Instead it
would be a component of the generic flash layer.

* provide a well-defined interface for all the above to underlying flash
drivers.
 - probably taking inspiration from the MTD API. Although we can't just
copy code obviously!
http://www.linux-mtd.infradead.org/tech/mtdnand/book1.html is very helpful
in this regard.
-=-=-=-=-=-=-=-=-=-=-=-=-=-

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["Si fractum non sit, noli id reficere"]------       Opinions==mine

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss