lofi(7D) compression support [PSARC/2007/569 FastTrack timeout 10/08/2007]

[Alok Aggarwal]

On Tue, 2 Oct 2007, Darren Moffat wrote:

> DJM-0 I don't quite understand how this is used.
>
> It seems that the process is like this:
>
> Create some image file that has a filesystem on it, lets make it UFS.
>
> # mkfile 1g /path/to/backingfile.img
> # lofiadm -C gzip /path/to/backingfile.img
> # lofiadm -a /path/to/backingfile.img
> /dev/lofi/1
> # mkfs -F ufs /dev/rlofi/1
> ...
> # mount -F ufs /dev/lofi/1 /mnt
>
> Is that correct ?
>
> In otherwords unlike the crypto case, and what your man page synopsis 
> suggests, the -C/-s arguments aren't things you pass in when you do a mapping 
> with -a but instead they are auxiliary functions.

Correct.

> DJM-1 How does this interact with lofi crypto ?
>
> (http://opensolaris.org/os/community/arc/caselog/2007/001/)
>
> Which happens first crypto or compression (hint it needs to be compression 
> otherwise it is waste of CPU resource) ?
>
> However if my understanding of how -C and -U work it seems that the 
> compression would be happening after encryption because by design it isn't on 
> the fly but actually a distinct admin action (which for building LiveCD 
> images is fine).

I need to think about this aspect some more, I'll come
back to this.

> DJM-2 What is the guidance for changing the segment size ?

Theoretically, segment size can be a multiple of 512 bytes.
For the LiveCD case we've seen that a segment size of 128k
gives us the best boot time performance.

For cases other than LiveCD, it would probably require some
experimentation to arrive at an optimal segment size.

> DJM-3 What happens if I make an lofi mapping with the default segment size 
> and later map the same file with a different size ?

Segment size can only be specified at the time of compressing
a file. That segment size is simply read in while mapping 
the file; you can't change it while mapping it in.

> DJM-4 What is the minimum segment size ?

That would be 512 bytes.

> DJM-5 How big is the header section ?

It is 12 bytes plus the length of compression algorithm used.
So, for gzip, it would be 16 bytes (12 + 4).

> The project to add encryption to lofi stalled on exactly the issue of the 
> adding metadata to the image.  Since this project is adding a header section 
> it seems to make sense that the crypto and compression projects share a 
> metadata section.

We should talk on this point, it does make sense to
have a shared metadata section.

> DJM-6 How big is the index section ?

It depends on the size of the file being compressed and
the segment size used. As an example, for a 100 Mb file
with 128k segment size, the number of index entries would
be 781 where each index entry is a uin64_t. This results in
the index section to be 6284 bytes.

> DJM-7 In the ZFS use of gzip it is possible to specify the level with gzip-6 
> being the default.   I think this case should do the same.

Can you think of a use case for providing gzip-6? For
the case of a LiveCD, we've found gzip-9 to be the most
optimal in terms of providing the best compression ratio.
gzip-9 is what is used in this project internally.

> DJM-8 The encrypted lofi case also proposed to add a new column to the output 
> to indicate if the mapping was encrypted, since this case also adds a new 
> column, coordination is needed (I think we are okay here though).

Yes, we should coordinate on this point too.

> Process issue:
>
> http://opensolaris.org/os/community/arc/caselog/2007/569/  gives 404 at the 
> moment.

I'll defer this to Tim.

Alok