embedded-linuxyoctosystemdsystemd-journald

systemd journald not working properly after reboot (until rotation)


I'm encountering strange issue with journald on a custom embedded Linux system built with Yocto.

On freshly prepared and booted system, journal works fine:

However, after issuing reboot and logging back in, current boot becomes somehow invisible:

All of above is fixed after calling --rotate - new file is created in journal/ dir, -b starts showing logs for current boot, listing boots shows one more entry as expected. It seems to me as if the system.journal file is inaccessible to journald for writing after reboot, and roatating somehow fixes that.
Interestingly, this only happens with reboot command ("soft" reboot). Switching power to the device off and back on (with or without calling poweroff first, and even with using reboot cutting power just after system starts booting again), after system is booted again journald is working as expected and new boot is correctly recognized.

Some additional info:

I would be grateful for suggestions on how to resolve or further debug this, and additionally (even if unrelated to the main issue) on how to resolve log partition unmounting failure mentioned (perhaps there is a way to delay unmount until after journald has stopped?).

Thanks


Solution

  • I'll answer my own question:

    The only necessary fix was to append log volume mount point to RequiresMountsFor in systemd-journal-flush service's unit file:

    RequiresMountsFor=/var/log/journal /log
    

    In yocto, that required patching systemd recipe to make a change in units/systemd-journal-flush.service.

    After this change, new boot is correctly recognized every time, and no more unmount failures during poweroff.