[hpcdev-discuss] Transfer Large File Using Multiple TCP Connections

[Yu Xiangning]

Terry Dontje wrote:
> After rereading your post again, I guess the first question I should have asked was what is the problem you are trying to solve for?  Is it to get data into a compute node from a disk farm or is it long haul file transfers from anywhere on the net?  gridFTP seems to be geared towards the latter but I am curious if that is really what you are looking for or not?
>

Hi Terry,

The later one is what I'm looking for. I'll take a look at how pNFS does
and come back to you. Thank you very much!

- yxn

> --td
> 
>> Does the pNFS project
>> (http://www.opensolaris.org/os/project/nfsv41/)
>> provide you with what you want without having to
>> recode ftp and all the other applications?  I know my
>> group is looking at interfacing MPI-IO to work with
>> pNFS.
>>
>> --td
>>
>>> Hi OpenSolaris and HPC folks,
>>>
>>> I wonder if there is any existing efforts in the
>>> OpenSolaris or HPC 
>>> community to make OpenSolaris better support
>>> applications to transfer 
>>> large files using multiple TCP streams.
>>>
>>> Anyway, here's some thoughts I put together:
>>>
>>> Basically we may provide a user-land library (may
>> it
>>> be libhpc or some 
>>> other names) to provide APIs to allow applications
>> to
>>> do parallel data 
>>> transfer, partial file transfer, and other key
>>> features that are 
>>> required for such applications in the HPC
>>> environments. I think there 
>>> are quite a few applications want to use these
>>> features, a typical 
>>> example could be the FTP server to support GridFTP
>>> [1]. And hopefully 
>>> some guy from the community will be happy to port
>> FTP
>>> server to support 
>>> GridFTP based on this library.
>>>
>>> Please feel free to comment, thank you!
>>>
>>> - yxn
>>>
>>> [1] I. Mandrichenko, W. Allcock, T.Perelmutov,
>>> GridFTP v2 Protocol
>>> Description. May 2005.
>>> _______________________________________________
>>> hpcdev-discuss mailing list
>>> hpcdev-discuss at opensolaris.org
>>>
>> http://mail.opensolaris.org/mailman/listinfo/hpcdev-di
>>
>>> scuss
> --
> This message posted from opensolaris.org
> _______________________________________________
> hpcdev-discuss mailing list
> hpcdev-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/hpcdev-discuss