Re: [ECOS] NAND for STM32, YAFFS and UFFS

[Simon Kallweit <simon dot kallweit at intefo dot ch>]

Ross Younger wrote:
> > which is rather important for me at the moment. The port consists of 
> > the HAL part as well as a new ST NANDxxxx3A NAND driver. Both were 
> > started
> > as copies of Ross's code.
>
> This was my clear intention from the start; whilst clone+hack is often 
> frowned upon, in this sort of situation it keeps the individual driver 
> size down. Perhaps over time we might put together some more generic 
> chip drivers.

I think that a consolidated driver should emerge sooner or later. Code 
duplication is one of my issues towards your NAND framework. There is 
also quite a bit of code and configuration duplication for partition 
support, which I don't like either.

> > What is still missing is interrupt mode and the usage of STM32's 
> > onboard ECC computation unit.
>
> Interrupt mode adds a nontrivial overhead; if your application isn't 
> multi-threaded, it won't win you anything, and in any event it the 
> overhead will make your accesses slower overall (though won't hit the 
> CPU in polling). As ever, the trade-off is yours to make, which is why 
> it's an explicit CDL option.

Agreed.

> I have been benchmarking parts of my setup this week and found - rather 
> unsurprisingly - that the overheads are pretty punishing. The chip on 
> the EA board takes 25us to read a page, 300-700us to write a page, and 
> 2-3ms to erase a block. I'm finding that interrupt mode on that board is 
> adding about a millisecond to every NAND call, which adds up fast. 
> (There are 65,536 pages on this chip, and an exhaustive scan of a 
> nearly-full YAFFS fs has to read most of them.)
>
> It's clear that it's just not worth doing interrupt mode for reads on 
> this particular board; writes maybe, and erases are clearly worth it. I 
> will be disabling interrupt mode for reads in the EA2468 driver very 
> shortly and I would advise you to consider whether the situation on the 
> STM3210E is similar 8-)

I see, when time permits I'll do some tests myself and see whats most 
reasonable, or make it configurable.

> As for ECC, it should hopefully be straightforward for you to add an ECC 
> "driver" which will plug right on in in place of the Linux MTD 
> algorithm. The MTD ECC code is 3 bytes ECC per 256 bytes of data; if the 
> onboard is a different size, you will also need to add a new OOB layout 
> definition in nand_oob.c. (I got mine by reading the Linux MTD code.) If 
> you don't need to interoperate with anything else, the MTD alg may be 
> suitable, but of course if code space is really tight for you, you might 
> prefer to use the onboard ECC and add a CDL option to not compile 
> nand_ecc_mtd.c.

I'll look into it.

> > I wanted to do some quick tests with YAFFS, as my initial port of the 
> > UFFS filesystem is not yet done, and I wanted to get some rough 
> > numbers of what resources are needed to run YAFFS. Unfortunately it 
> > turned out that the YAFFS port does not handle small page NANDs, as 
> > one is found on the STM32 evalboard. Further investigation showed that 
> > YAFFS1 was using small page NANDs and YAFFS2 is used on 1k++ page 
> > NANDs. The eCos port is based on YAFFS2 I guess, so I think this 
> > should be renamed properly. Or was there a plan to support both YAFFS1 
> > and YAFFS2?
>
> There's a bit of a potential for terminology confusion here, so I shall 
> digress briefly in the hopes that it's useful for people not following 
> this debate so closely... In Aleph One's CVS repo, there are two 
> distinct modules `yaffs' and `yaffs2'. The former only supports 
> small-page, and is obsolete; the latter is effectively a patched version 
> which supports both small- and large-page devices. Confusingly, 
> internally to the code, `YAFFS1' refers to the small-page support and 
> `YAFFS2' the large-page. Personally, I consider there to be one YAFFS 
> with two tentacles.
>
>
> I have been working only with the yaffs2 source base. However, as I have 
> only been working with its large-page support (YAFFS2), I have compiled 
> out the small-page (YAFFS1) support meantime, and only written 
> large-page NAND hooks. This leaves a small gap waiting to be bridged.
>
> Off the top of my head, what's needed to make the small-page code work is:
> * to add a switch somewhere to tell eyaffs_mount() to set up the device 
> struct as YAFFS1 instead of YAFFS2;
> * to provide the couple of YAFFS1 NAND access functions which I haven't 
> yet (prototypes for which exist in ecos-yaffs-nand.c);
> * to unset CONFIG_YAFFS_NO_YAFFS1 in ecos-yaffs.h (perhaps make 
> YAFFS1/YAFFS2 modes properly CDL-switchable);
> * to add in the "missing" files from the yaffs2 distribution (just 
> yaffs_packedtags1.[ch], I think).

I did quickly implement YAFFS1 and made it configurable by CDL. There 
are some small changes necessary to the YAFFS codebase so we don't 
compile in too many obsolete functions (compat layer and ECC computation 
stuff). The fops test runs successfully on my STM3210E evalboard with 
smallpage NAND.


> As it happens I believe we do have an STM3210E somewhere here, so I will 
> probably end up looking into this myself in due course, but it rather 
> looks like I'm going to be distracted by commercial concerns for the 
> next couple of weeks :-(

I'll provide patches as soon as anybody wants to try it out.

> You mention that RAM is tight, I should raise a flag about YAFFS memory 
> usage. My benchmarking this week provided a rather alarming result. 
> Buried in YAFFS's own documents is the suggestion that one should budget 
> two bytes of RAM per page of NAND for YAFFS's internal data structures. 
> Now, the chip on the EA board is 1024 blocks x 64 pages per block, i.e. 
> 65536 pages total (x 2kbytes each, for a 128Mbyte chip) - so by the rule 
> of thumb one would expect to budget 128Kbytes. Worryingly, after I had 
> written 10,000 small files into YAFFS, my heap usage as discerned from 
> mallinfo() was around 1.6Mbyte! (I need to investigate this more; it's 
> next on my plate after sorting out the timings. However I am pretty sure 
> there is no memory leak, as the heap usage went to zero after umount.) I 
> am in the process of composing a post to the YAFFS mailing list raising 
> this and will see what the folks there say.

Well, that's good to hear. 1.6MB is well out of my league here. Even the 
RAM footprint calculations on the yaffs webpage are quite over the top 
for our platform. I think YAFFS will not make it into our products.

> > After some first investigation of the UFFS source code it seems that 
> > it needs quite a bit of work to make it work properly with eCos and 
> > not have hacks all over the place. But it seems that YAFFS does still 
> > use too much resources for our usage, so I think I need to work on a 
> > simpler one and UFFS still looks like a good start ...
>
> By the way, I'm still working on that proposal I mentioned here the 
> other day to let RedBoot use NAND as its FIS (by pretending it's a funny 
> flavour of NOR). While that won't be directly of use for you, it might 
> prove adaptable as a very very simple FS. But, so many things to do, so 
> little time...

My requirements are two-fold. First, I need a "standard" FS to store 
configuration and other data of variable size. For our platform it would 
be fine to not support stuff like symlinks, or have a fixed number of 
files allowed on the FS, to make it more compact. But, a standard 
compliant FS would still be very neat.

On the other hand I will need something very simple to store for example 
a firmware update, which is downloaded via GSM during the execution of 
the application. This should then be flashed during a reboot cycle from 
within my custom bootloader application. The bootloader should remain 
very small, so I don't have the option of supporting a FS.

Probably those too needs could be satisfied by a single very simple 
implementation of a filesystem. I have not yet found a public domain 
solution which would fit. So I think I'll have to roll my own.

Simon

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss