Re: Chroot testsuite

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Fri, 10 Jan 2014, Ryan Arnold wrote:

> Creating the chroot is out-of-scope for glibc since the methodology
> for doing so will vary based on operating system distribution (for
> example, Ubuntu might use schroot and debootstrap while for Fedora
> might use systemd-nspawn and yum --installroot).  A wiki how-to would
> be helpful in any case.  I imagine an OS distribution could also
> provide a package which does this for their specific src package
> manager.

> Is there any coverage that's lost by not performing builddir tests
> remotely if they're all repurposed as install/host tests?

glibc testing should not assume any particular packages on the host; it 
should work even if what you get from ssh is just a busybox shell (with no 
privileges to run chroot).  It should also not require that any OS 
binaries are available that are compatible with the newly built glibc 
rather than being linked with some other incompatible libc (consider, for 
example, testing an x32 glibc when nothing other than the kernel supports 
x32, or testing an ABI-changed glibc as part of bringing up a port).

So, except where testing something inherently requires chroot, I don't 
think tests for that thing should be limited to environments where chroot 
is a possibility - although it's useful to be able to run the same test 
both ways.  And where chroot is a possibility, being able to test in a 
chroot directory that is exactly what you get from install_root - no other 
software at all - would be desirable.

I do want to be able to run tests against installed glibc without the 
builddir from the original build (in support of a model of building a 
complete cross toolchain, then testing the final complete toolchain) and 
without needing to chroot, but the way I'm thinking of implementing that 
is using appropriate makefile variables that can be overridden for 
installed testing, so that e.g. tests link with $(ld.so-tests) which could 
be an installed file rather than a build tree file, so the dependencies 
don't force most of the library to be built.  That also requires 
distinguishing those tests that inherently examine properties of the build 
tree rather than just installed files, and should have a separate makefile 
target to run them at build time.  It also requires a mechanism to save 
any non-installed build-tree files that really are needed by tests.  I'm 
not planning on working on this until my changes to generate PASS / FAIL 
results are in.

I would not be adverse to the normal glibc testsuite moving to installing 
files in a staging directory within the build tree using install_root, 
then running tests the same way they would be run with a previously built 
and installed glibc, provided this does not require any extra software or 
privileges for chroot.

I've previously suggested a GLIBC_SYSROOT environment variable for use in 
testing, that would allow more of the effects of chroot to be emulated 
without needing any privileges or extra installed software, but there were 
some concerns about having such a variable.

-- 
Joseph S. Myers
joseph@codesourcery.com