[indiana-discuss] replicating snapshots from time-slider to a remote server

[Tom Georgoulias]

Tim Foster wrote:
> hey there,
> 
> On Tue, 2009-01-06 at 01:07 +0000, Calum Benson wrote:
>> On 17 Dec 2008, at 19:27, Tom Georgoulias wrote:
>>
>>> Is it safe to assume my options are to either wait for the next  
>>> release of time-slider or roll my own?  Are there any other options?
>> Well, I guess you could still try using Tim Foster's original ZFS  
>> Automatic Backup service: <http://blogs.sun.com/timf/entry/zfs_automatic_for_the_people

> That said, the  backup-save-cmd stuff should work ok in the current
> shipping version of the automatic snapshot service in 2008.11, it's just
> not feature-complete yet.  That is, you should be able to send full or
> incremental send streams of each snapshot with the right backup-save-cmd
> string - some other folks on zfs-discuss@ were doing that successfully
> recently.

> A more complete solution for dealing with periodic incremental send
> streams is a little more complex (and I haven't got cycles to implement
> it at the moment unfortunately)
> 
>   It would involve querying the remote server for a list of snapshots
> per dataset we're interested in, then determining which snapshots are
> common between the local and remote ends.  We could then send an
> incremental send stream with differences between that common snapshot,
> and the one we've just taken locally.
> 
> Right now, as backup-save-cmd is completely freeform, there's no set
> protocol to determine how to retrieve that list of remote snapshots, how
> to cope with differing levels of snapshots per local dataset[1] etc. so
> we just ignore the problem(!).  The ZFS Automatic Backup Service did
> this properly, checking the USB disk to see whether a full backup stream
> was present for each dataset, before choosing to send incrementals,
> sending a full backup stream otherwise.

> 	Does that help at all?

Absolutely.  I've been hacking on a zfs replication shell script that I 
found on a blog and trying to make it work for my setup.  What you've 
said has just reassured me that the headaches I was running into aren't 
just a result of my minimal zfs knowledge, as I was struggling with a 
way to "rotate" the snapshots on the source server (making a new one, 
getting rid of the oldest, and renaming those in the middle to reflect 
their new status) and keeping the remote server sync'd up with the 
source.  So far I haven't been successful at that.

I know that the fishworks-based Sun Storage 7000 products have the 
ability to replicate snapshots to remote appliances and keep the systems 
in semi-sync, so there must be a way to do it.  Is that ability 
available in the open storage pieces of opensolaris?  Could we build 
upon that instead of doing something new?

I try pulling out time-slider and go back to trying to use 
zfs-auto-snapshots.  Can't hurt to give that a shot again.

Thanks,
Tom