[hpcdev-discuss] Transfer Large File Using Multiple TCP Connections

[Yu Xiangning]

Hi OpenSolaris and HPC folks,

I wonder if there is any existing efforts in the OpenSolaris or HPC 
community to make OpenSolaris better support applications to transfer 
large files using multiple TCP streams.

Anyway, here's some thoughts I put together:

Basically we may provide a user-land library (may it be libhpc or some 
other names) to provide APIs to allow applications to do parallel data 
transfer, partial file transfer, and other key features that are 
required for such applications in the HPC environments. I think there 
are quite a few applications want to use these features, a typical 
example could be the FTP server to support GridFTP [1]. And hopefully 
some guy from the community will be happy to port FTP server to support 
GridFTP based on this library.

Please feel free to comment, thank you!

- yxn

[1] I. Mandrichenko, W. Allcock, T.Perelmutov, GridFTP v2 Protocol
Description. May 2005.