Add sparse file support to cpio [PSARC/2008/727 Fast Track 12/03/2008]

[Don Cragun]

>Date: Mon, 24 Nov 2008 12:10:21 -0800
>From: "Garrett D'Amore" <gdamore at sun.com>
>
>It seems to me that there has been enough real discussion on this topic, 
>that I think it is worth promoting to at least a fast track status.  
>Please consider it so promoted, and set the time out for a week from 
>Wednesday.   (A longer timeout than 1 week is appropriate, IMO, given 
>that this week represents a short week in the US.)

The case has been recast as a fast track with timeout set to
12/03/2008.  Note that this is delaying a fix for an escalation for a
large external customer (whose name I cannot give on an alias with
non-Sun participants).

Note that a good portion of this discussion is about PSARC/2006/361;
not about this case.

>
>    -- Garrett
>
>Joerg Schilling wrote:
>> Don Cragun <don.cragun at sun.com> wrote:
>>
>>   
>>> and in the EXTENDED DESCRIPTION, where it says:
>>>
>>>     "SUN.holesdata    A Solaris extension to pax extended  header
>>>                       keywords. Specifies the data and hole pairs
>>>                       for a sparse file.
>>>
>>>     
>> ...
>>
>>   
>>>                         49 SUN.holesdata= 0 8192 24576 32768 49152 49159
>>>     
>>
>> I just verified this on a recent build and it seems that Sun pax now contains 
>> a similar conceptional problem as the GNU tar guys did introduce a few years 
>> ago: 
>>
>> The size of the POSIX.1-2001 file metadata is limited to 8 GB. Putting the
>> hole information into the file meta data area thus limits the number of holes.
>>
>> If you e.g. create a file with the maximum holyness (Block size == 8k),
>> you are able to archive a file of up to ~ 5 TB and aprox. 300 million holes.
>> I know that an archiver that is going to unpack this would need to allocate
>> 5 GB of virtual memory to remember the hole list. Files that fall into this 
>> category are already possible today, so this does not seem to be ready for the
>> future.
>>
>> I recommend to fix this by moving the hole information into the file data area,
>> to switch to a offset/datasize notation (to reduce the size) and to write
>> the information as comma separated pairs.
>>

That is not this case; you are asking to reopen 2006/361.

The holesdata for cpio presented by this case is stored in the file
data area as you are requesting; not in the header.  However, the file
size (including the holesdata) is still limited by the 11 octal digit
c_filesize field in the cpio file header to a maximum size of 8Gb.

>>
>> Jörg