[indiana-discuss] [zfs-auto-snapshot] replicating snapshots from time-slider to a remote server

[Brett Eisenberg]

I'd certainly vote for integration into zfs-auto-snapshot ;)

b

On Jan 7, 2009, at 12:51 , Richard Elling wrote:

> Tim Foster wrote:
>> hey there,
>>
>> On Tue, 2009-01-06 at 01:07 +0000, Calum Benson wrote:
>>
>>> On 17 Dec 2008, at 19:27, Tom Georgoulias wrote:
>>>
>>>
>>>> Is it safe to assume my options are to either wait for the next
>>>> release of time-slider or roll my own?  Are there any other  
>>>> options?
>>>>
>>> Well, I guess you could still try using Tim Foster's original ZFS
>>> Automatic Backup service: <http://blogs.sun.com/timf/entry/zfs_automatic_for_the_people
>>>
>>
>> Yeah. That stuff was really aimed at removable USB disks, and never
>> quite hit the mark [ it assumed FAT32 formatted disks, and send
>> send-streams as flat files, split into manageable chunks: it was a  
>> bit
>> clunky, and not having a decent "restore" story stopped it in it's
>> tracks ]
>>
>>
>> That said, the  backup-save-cmd stuff should work ok in the current
>> shipping version of the automatic snapshot service in 2008.11, it's  
>> just
>> not feature-complete yet.  That is, you should be able to send full  
>> or
>> incremental send streams of each snapshot with the right backup- 
>> save-cmd
>> string - some other folks on zfs-discuss@ were doing that  
>> successfully
>> recently.
>>
>>
>> A more complete solution for dealing with periodic incremental send
>> streams is a little more complex (and I haven't got cycles to  
>> implement
>> it at the moment unfortunately)
>>
>>  It would involve querying the remote server for a list of snapshots
>> per dataset we're interested in, then determining which snapshots are
>> common between the local and remote ends.  We could then send an
>> incremental send stream with differences between that common  
>> snapshot,
>> and the one we've just taken locally.
>>
>
> I've written such a beast.  But it isn't integrated with the
> zfs-auto-snapshot
> service (it could be).  I've only tested it on a limited number of  
> systems
> and there are still failure modes which need attention.  Is there  
> already a
> project created for such things?  Or should we glom into zfs-auto- 
> snapshot?
> -- richard
>
>
>> Right now, as backup-save-cmd is completely freeform, there's no set
>> protocol to determine how to retrieve that list of remote  
>> snapshots, how
>> to cope with differing levels of snapshots per local dataset[1]  
>> etc. so
>> we just ignore the problem(!).  The ZFS Automatic Backup Service did
>> this properly, checking the USB disk to see whether a full backup  
>> stream
>> was present for each dataset, before choosing to send incrementals,
>> sending a full backup stream otherwise.
>>
>> Incidentally, if anyone's interested in working on this support -  
>> that'd
>> be cool, Cc:ing zfs-auto-snapshot in case anyone there has free  
>> cycles.
>>
>> 	Does that help at all?
>>
>> 	cheers,
>> 			tim
>>
>> [1] As the auto-snapshot service allows you to snapshot multiple
>> unrelated filesystems at a time, any time we add a new local  
>> dataset and
>> choose to also snapshot/backup it, we'd need to ensure that we send
>> incremental streams for older datasets, and full streams for the new
>> dataset - gets a bit hairy.
>>
>>
>>
>>>
>>>>
>>>
>>> I don't know off-hand whether it'll still work on 2008.11 (or if
>>> they'll be confused by the presence of Time Slider on the same
>>> machine)-- I suspect it might need a bit of tweaking now.  But it
>>> might at least serve as a decent starting point for your own  
>>> solution
>>> until Time Slider supports backups as well as snapshots.
>>>
>>> Cheeri,
>>> Calum.
>>>
>>>
>>
>> _______________________________________________
>> indiana-discuss mailing list
>> indiana-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/indiana-discuss
>>
>
> _______________________________________________
> zfs-auto-snapshot mailing list
> zfs-auto-snapshot at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-auto-snapshot
>
> !DSPAM:1000,4965231d5309021468!
>