Depends. I'm imagining e.g. your large parent string invasively giving it's sub-slices copies on finalization through it's own list, in which case the weak references clearing isn't much of an issue. Not easy to make fast and thread safe however.
In C#, WeakReference can take a boolean parameter during construction to specify if it tracks post-finalize or not (defaulting to the same behavior as Java which google tells me is as you say: not), and you can re-register your finalizer, but even so this kind of thing isn't what the GC is primarily designed for and it will probably show. I'm not sure if you can re-register finalizers in Java.
In C#, WeakReference can take a boolean parameter during construction to specify if it tracks post-finalize or not (defaulting to the same behavior as Java which google tells me is as you say: not), and you can re-register your finalizer, but even so this kind of thing isn't what the GC is primarily designed for and it will probably show. I'm not sure if you can re-register finalizers in Java.