Re: malloc, vfork, and OpenJDK

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 02/05/2019 09:46, Florian Weimer wrote:
> I recently looked at what would be needed to eliminate jspawnhelper from
> OpenJDK.  It turns out we'd need closefrom support in posix_spawn (bug
> 10353).

Yes, I was planning to send a message to ask if it would be worth
to add a closefrom action on posix_spawn file actions. Btw I recently
wrote about this very issue with jspawnhelper [1]

> 
> jspawnhelper is a fairly recent addition on OpenJDK on Linux, though.
> It turns out that the default vfork-based code for launching processes
> calls opendir from the vfork'ed subprocess, thus calling malloc after
> vfork.  This is needed for the OpenJDK-internal implementation of
> closefrom.
> 
> Obviously, calling malloc after vfork is completely undefined, but
> OpenJDK binaries have a very long shelf life, so future changes to
> malloc will have to take this compatibility constraint into account.
> 
> The good news is that I think that our current malloc implementation has
> this property.  Even if we expand the heap during malloc (with sbrk or
> mmap), the vfork implementation in Linux ensures that the change is also
> reflected in the parent process.  The futex-based locks do not use TIDs,
> so the arena lock is acquired and released as expected.
> 
> The only bug I can see is that destroying the subprocess from Java
> (which is supposed to be memory-safe) causes a memory leak in the parent
> process (if it happens during closefrom) or a deadlock (if it happens
> during malloc or free, while arena lock is held).  But this is a bug
> that exists today, and it does not seem to matter much in practice.
> 
> And OpenJDK recently switched to posix_spawn + jspawnhelper for its
> default process launch method, so that bug is going to go away
> eventually.  But Java binaries tend to stick around for a long time.

So the idea is to focus on making malloc work correctly with vfork,
so old OpenJDK which uses vfork for process spawning does not issue
undefined behaviour? Or the remaining bugs are not really affecting
OpenJDK and the idea is start to define a strict environment where
malloc after vfork would be safe?

It looks like even the *option* to use posix_spawn is a recent addition 
to openjdk [2]. So the question I have is how effective would to focus on 
backport the option to use posix_spawn on OpenJDK.

We can focus on extending posix_spawn to add the missing features to remove 
the requirement of jspawnhelper (which is orthogonal to the issue).

[1] https://zatrazz.github.io/More-About-Lauching-Processes/
[2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-October/056122.htm