[gin-data] On SRB interoperability with gLite

[Reagan Moore]

Jens:
The SRB supports third party transfer, bulk transfer, parallel I/O, 
and registration.  There are several ways to support data movement 
between two sites:

1) Register the data that is in SRM A into a SRB collection.  Then 
issue a SRB replicate command to make a copy using third party 
transfer on storage system B.

2) For SRM version 1, a SRB driver exists that supports issues SRM commands.

3) The SRB support both a GridFTP drive, enabling access to a remote 
storage system through GridFTP, and a GridFTP client that can access 
files in any SRB collection.

The new technology we are developing is called iRODS.  This is a 
rule-based data management system that automates the application of 
management policies.  The iRODS system uses rules to control the 
execution of remote micro-services.  If there is sufficient interest, 
a micro-service could be created that talks to the SE.

Reagan Moore



>Just a quick update on the message below.
>
>I did get feedback from Erwin, but not from anyone else.  If
>someone is taking it off to discuss on srb lists, could you
>summarise the result to gin-data please.  Ta.
>
>Someone once told me that SRB has a GridFTP server, so I checked
>with Adil Hasan and he says his, yes, and his team got it working.
>
>As I see it, it is essential to be able to drive the SRB via
>GridFTP to avoid copying files out to the client.  I.e., if copying
>files from SRM A to SRB B from Client C, then you want the data
>to travel from A to B, not via C.
>
>If this is the case, then perhaps the FTS can drive the SRB like
>it does Classic SEs (I am told some countries still have Classic
>SEs :-)
>
>STFC - our organisation, RAL's research council, has got all the
>experience in-house, so we can play with it a bit and see what
>happens.
>
>I asked Adil to look into the GridFTP-for-SRB thing, and Matt
>Hodges from Tier 1 to test the FTS-to-GridFTP.  We can then plug
>it together and see if it works trivially.
>
>Regards,
>			--jens
>
>--
>Dr Jens Jensen
>STFC Rutherford Appleton Laboratory
>
>
>-----Original Message-----
>From: gin-data-bounces at ogf.org [mailto:gin-data-bounces at ogf.org] On
>Behalf Of Jensen, J (Jens)
>Sent: 11 May 2007 07:28
>To: gin-data at ogf.org
>Subject: [gin-data] On SRB interoperability with gLite
>
>Morning boys and girls.
>
>Arun and I had a discussion yesterday on how to make SRB and
>gLite interoperate.  We have discussed for a while to build
>SRB interfaces to SRM, and vice versa, and this is an old
>discussion, but there may be a "lower hanging fruit":
>
>What if FTS (File Transfer Service) - which can already talk
>to (at least) 3 SEs, SRM 1, SRM 2 (right?), and Classics
>- added support for SRB as another SE.
>
>Stepping back to the use case, I would think the primary customers
>would be
>(a) any VO which holds data in both worlds, e.g. (1st approx)
>all non-HEP VOs in EGEE;
>(b) Grids that wish to interoperate, like GridPP (SRM only)
>and the (UK) National Grid Service NGS (SRB).
>
>Questions.  People analysing SRB data tend to locate it and Sget
>it out of SRB.  Is FTS appropriate for this.  How does it work
>with the job submission workflow if an SRB file is needed
>(I didn't get enough sleep to figure that one out for myself :-)
>
>Probably a GLUE information service, say a BDII, is needed for
>SRB - FTS doesn't need it directly but other tools might if they
>are made SRB aware - worth thinking about now.
>
>So it would lose metadata information from the MCAT but
>perhaps a smarter egg could write something to transfer it
>to something else.
>
>Filenames.  Perhaps we now need "srm://srm.wombat.ac.uk/srm/file"
>and "srb://srb.wombat.ac.uk/srb/file" SURLs.  But the GUID and
>LFN stuff would work as before.
>
>Apart from everybody thinking interoperability is good, the real
>need should be identified.  We need to identify a real user who
>needs real data in both worlds, and work through the use cases.
>Any show of hands?
>
>Anyway, I am just thinking out loud as usual but thought I would
>suggest it so we can discuss it today.
>
>Thanks,
>--jens
>--
>   gin-data mailing list
>   gin-data at ogf.org
>   http://www.ogf.org/mailman/listinfo/gin-data
>--
>   gin-data mailing list
>   gin-data at ogf.org
>   http://www.ogf.org/mailman/listinfo/gin-data