linuxshellpipe7zipstdio

Extract large 7z nested archives using pipe

I have a large password-protected 7z archive that contains another password-protected 7z archive, structured as follows:

OuterArchive.7z (200GB)
    └── InnerArchive.7z (200GB)
        ├── SomeFile.ext (1GB)
        └── ...

The sizes of the archives are extremely large (over 200GB), and I don't have enough free space to extract the entire archive as a file. Therefore, I am trying to use standard I/O pipes to extract only the requested file with the following command:

7z x OuterArchive.7z -so InnerArchive.7z -ppass1 | 7z x -si -so -ppass2 SomeFile.ext

Is this command correct (since it doesn't work), and is it even possible to execute such a thing since the file size exceeds the device's RAM (16GB)?

UPDATE: Here is the result of the command:

ERROR: 
Can not open the file as archive

E_NOTIMPL

You can see the first part of command is working:

┌──(user㉿dhcppc4)-[~/Desktop/x]
└─$ 7z x OuterArchive.7z -so InnerArchive.7z -ppass1 | head -n 1 | hexdump -C
00000000  37 7a bc af 27 1c 00 04  5f 0f 51 b0 a3 61 2f 29  |7z..'..._.Q..a/)|
00000010  1d 00 00 00 27 00 00 00  00 00 00 00 6f a6 f5 15  |....'.......o...|
00000020  a3 8c 32 aa 06 f8 aa c7  92 b0 42 a1 01 0c 66 a3  |..2.......B...f.|
00000030  77 09 6c c3 f9 da 72 28  20 d0 b6 ac 0b b8 ed fc  |w.l...r( .......|
...

So something must be wrong in this part: 7z x -si -so -ppass2 SomeFile.ext

Solution

  • Apparently this action is not supported. (cf. E_NOTIMPL).

    See: https://superuser.com/questions/1509609/combined-decryption-and-unzipping-of-a-file

    The answer from Igor Pavlov (the creator of 7-zip) is:

    7-zip needs "seek" operation for 7z archives. So you can't use -si to extract from 7z archive.

    This was dated 2016-03-30. My tests and the changelog indicate the statement is still valid in 2019.

    It might be possible to mount the external 7z archive as a filesystem so that 7z can have the seekable file it needs.

    Some projects that appear to support this using FUSE are:

    Using a web-server as a helper:

    Uses temporary files, so probably no good for the use-case of the question:

    Older, possibly non-working, projects include: