[ogsa-naming-wg] RE: GGF/OGSA standards for hierarchical namespaces

[David Snelling]

Dave,

On 5 Apr 2006, at 20:53, Dave Berry wrote:

> Another question I've had raised to me is whether we can use WSRF/WSRT
> to handle the properties associated with entries in the directory.  I
> personally don't see how this would work, but given the context we're
> working in it would be helpful to establish what relationship  
> exists, if
> any.

If the "Directory" were referred to by an EPR, then with the WSRF/WS- 
Transfer operations one could obtain properties using the Get*  
operations. These properties could be both standardized across all  
domains (e.g. number of entries) or be domain specific (e.g. rwx flag  
settings) or be dynamic to a specific instance (e.g. "The 'files' in  
tis directory are all pipes connected to experiments.").


>
> Best wishes,
>
> Dave.
>
> -----Original Message-----
> From: Christopher Jordan [mailto:ctjordan at sdsc.edu]
> Sent: 05 April 2006 18:20
> To: Allen Luniewski
> Cc: Dave Berry; Ian Foster; gfs-wg at ggf.org; Andrew Grimshaw; Hiro
> Kishimoto; Mark Morgan; Gregory Newby; ogsa-naming-wg at ggf.org
> Subject: Re: GGF/OGSA standards for hierarchical namespaces
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> (cc list trimmed of people I know are on the mailing lists, to limit
> the number of duplicate copies received. as always, forwarding is
> encouraged)
>
> Thanks to everyone for useful responses, and to both Manuel and Mark
> for providing workable hierarchical, abstract naming schemes
> ("directory standards", in the context of this discussion), each with
> valuable features. I'm writing this note as a preface to the
> discussion of naming at the F2F meeting today, and as a response to
> some of the points already made in this thread. I'll mostly just be
> throwing out a few vaguely connected thoughts and pieces of
> information to help advance the starting point for the discussion
> today. As important as I feel it is for the community to quickly
> agree upon a definite direction/core draft standard, it's also
> crucial to get it right - we don't necessarily have to provide a
> perfect solution the first time, but we do have to provide a solution
> that can grow and change organically to fulfill new needs, as the
> community creates them.
>
> Just to update my previous e-mail regarding the options and
> suggestions out there, I've spoken with Osamu Tatebe, and it appears
> that the RNS item in my initial list of efforts and proposals is now
> coincident with Dave Berry's post-GGF16 note suggesting that RNS be
> re-factored into at least two separate standards. In this regard,
> Osamu says "At the last GFS-WG session in GGF16, we agreed to
> refactor RNS specs into (at least) three specs; RNS, iterator,
> logical name resolver." It's not immediately clear to me what is
> meant by "RNS" in this quote, but my guess would be that it refers to
> the directory standard/service itself. My understanding is that
> Manuel Pereira is currently working on this refactoring, and will
> offer a draft to the GFS (and likely OGSA-Naming) working groups for
> further discussion, once a workable draft is complete.
>
> To clarify my own ulterior motives, my goal is to get the work on
> naming and directory-related issues moved into the OGSA-Naming group,
> even if it is the same set of players doing the work, so that the GFS-
> WG can focus on the requirements for file systems within a "directory
> and logical/abstract naming system" - for example, a GFS directory or
> "file" entry should have properties such as size, support information
> about the source storage resource and access mechanisms, replication
> properties, the data's tier within an HSM, etc etc. I'm hoping that
> we can all work together to create a set of directory naming and
> resolution standard(s) that can support these needs, and I believe
> that if we do, they will also support just about any other function
> for which an abstract-name-based directory system is necessary.
>
>  From the above paragraph, I think it should be clear that I believe
> that it is a requirement to be the able to associate arbitrary
> attributes/properties with entries in the directory structure. What I
> envision here is a simple set of base standards, ideally to come out
> of the OGSA-Naming group; one for hierarchically organized
> collections of human-readable abstract names with arbitrary
> properties attached, each of which resolves to an EPR, and one for an
> iterator interface, where the items being iterated over are the
> abstract names. Each standard utilizing hierarchical collections of
> human-readable names, including the GFS-WG, would then define a
> "profile" for use of the directory service, including requirements
> for the properties which should be associated with a directory entry,
> and any higher-level services which would ideally build upon the
> "directory" standard I'm hoping to see.
>
> I believe that the WS-Directory proposal, certainly in the v3
> iteration Mark Morgan referred us to in an earlier e-mail, would
> fulfill the requirements I have for hierarchically organized
> collections of human-readable names, referring to EPRs and having
> attributes associated with them. We still need, in my opinion, an
> iterator interface standard to work in concert with these
> "directories"; hopefully Manuel's work with refactoring RNS will
> provide us with that.
>
> I'll stop here, since I believe the F2F discussion of naming will
> begin shortly. Thanks again to everyone for participating in this
> discussion so far, and I look forward to continuing it both in the
> F2F and through other channels.
>
> On Mar 28, 2006, at 11:17 AM, Allen Luniewski wrote:
>
>>
>> Chris,
>>
>> Thanks for the very thoughtful note!  A few comments.
>>
>> I agree with you that the current situation is, to put it mildly,
>> not acceptable.  We need to move to a position where the community
>> agrees on a single means for creating a hierarchy of pointers to
>> resources (a directory structure in Unix-speak).  Clearly one
>> aspect of this is that RNS and WS-Directory need to reconciled into
>> a single proposal.   Resolution sooner rather than later is clearly
>> in the best interests of the various WGs who depend upon
>> directories (including OGSA Data which I share responsibility for).
>>
>> You mention a couple of specific technical issues.  A few comments
>> on those:
>>         1. I go back and forth on attributes in the directory
>> system.  Since these are almost certainly cached from the
>> resources, today I am inclined to say that
>>                 attributes should not be part of the directory
>> system.  Ask me tomorrow, and I may have the other answer :-)  But
>> today, I would leave them out
>>                 of the base specification for simplicity but
>> consider an extension that included attributes if that were felt to
>> be vital.
>>         2. Dave Berry's suggestion to separate out the iterator
>> component is a good one.  There are going to be many places in the
>> overall grid standards
>>                 effort where an iterator-like structure will be
>> needed. Standardizing this seems like a proper goal for GGF.  If
>> that is accepted then using it
>>                 in a directory service seems quite natural.
>>
>> As for moving forward, I think that we need to see how this thread
>> plays out.  My instincts, however, are that getting the interested
>> parties in a room for a few hours would be the most effective way
>> to drive this to an early resolution.
>>
>> Allen Luniewski
>> IBM Information Management Division
>> San Jose, California
>>
>>
>>
>> Christopher Jordan <ctjordan at sdsc.edu>
>> Subject: GGF/OGSA standards for hierarchical namespaces
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>> All,
>>
>> Forgive the wide distribution on this e-mail, but this issue seems to
>> be to be both extraordinarily important to the future of GGF/OGSA
>> standards efforts and also in a state of either limbo or paralysis.
>> The topic I'm addressing here, both in my capacity as the  
>> secretary of
>
>> GFS-WG and as a generally interested participant on a few different
>> GGF working groups, is the question of adopting a single, possibly
>> minimal, standard for creating hierarchically organized  
>> collections of
>
>> pointers (WS-Names? GSR/GSHs, to date myself?) to "resources", where
>> the term "resource" could denote a service providing access to a
>> collection of files, computational resources, or database records
>> (that's a non-exclusive list), and where some items in the hierarchy
>> could actually represent directory-like structures, i.e. containers
>> for other collections of resources.
>>
>> The way I got involved in this discussion through the Grid File
>> Systems-WG, which at the time was bringing the RNS specification
>> forward for final approval as a GFD. Subsequently, there have been
>> numerous discussions outside of the GFS-WG context about the
>> suitability of the RNS standard for more general applications, as  
>> well
>
>> as the (perceived) complexity of the standard as a barrier to entry.
>> There have also been alternative directory construction standards
>> proposed by members of the OGSA-Naming-WG.
>>
>> The following are the activities/proposals I know of:
>>
>> RNS: I know the GGF editors have returned the final(?) RNS draft to
>> GFS-WG, with the suggestion that it is too specific to filesystem
>> needs, and the suggestion that it either be limited in scope to GFS
>> applications only (a non-optimal solution for obvious reasons) or  
>> that
>
>> the authors work with the OGSA-Naming people to help develop a
>> universal standard for hierarchical resource namespaces. If we are to
>> move forward with RNS, one of these options will clearly be a
>> necessity, given the points Greg Newby made in his responses on  
>> behalf
>
>> of the GFSG.
>>
>> WS-Directory: This is the hierarchical namespace standard  
>> developed at
>
>> UVa in response to their difficulty in implementing the complexities
>> and ambiguities in RNS. I like the simplicity of WS- Directory,
>> however it seems to be missing significant requirements for general
>> use such as attributes, both attributed which should be required such
>> as time-to-live, and the ability to add extensibility attributes such
>> as resource type, QoS, etc. This ability to add arbitrary attributes
>> is present in RNS but it still lacks some obviously fundamental
>> required attributes.
>>
>> Finally, Dave Berry sent an e-mail immediately after GGF16 in  
>> which he
>
>> mentioned the suggestion that we separate this functionality into two
>> logical functions, and therefore standards - a Directory Interface  
>> and
>
>> an Iterator interface, in which Directory interfaces were essentially
>> just pointers to Iterators, which would be standardized. However,
>> there would be no restriction that a Directory point to a particular
>> type of iterator interface. One point I wasn't clear on from the
>> e-mail was whether an entry in an interator could be another
>> directory, although I suspect it can.
>>
>> This short list is what I've got within easy reach. As I said
>> previously, I believe this is an important issue to resolve quickly,
>> and I'm sending this note in the hopes of initiating the conversation
>> among as many of the relevant parties as I can. Please feel free to
>> forward at will, respond with agreement, anger, or even unconcealed
>> rage.
>>
>> Possible ways forward would be for us to have a conference call (GFS-
>> WG meets rarely, and we could quite easily give up our call for a  
>> more
>
>> focused discussion of these issues), an extended e-mail  
>> discussion, or
>
>> a meeting at the next GGF (assuming we get a chance).
>>
>> Let me know how you feel about the options presented above, or feel
>> free to propose new ones if you like. The important thing is that we
>> begin to gain momentum, and then keep it going forward.
>>
>> Thanks.
>>
>> N.B. For anyone who may have missed any of the discussions reference
>> above, please let me know and I'll be happy to forward them to you
>> from my archives.
>
> - ----------------------------------------------------
> Chris Jordan
> HPC Systems Engineer
> High End Computing Systems Group
> San Diego Supercomputer Center
> ctjordan at sdsc.edu
> 858.534.8347
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (Darwin)
>
> iD8DBQFEM/xVPCVtcXn6kg8RArY1AKCO4lAdeWchGcMgDcmDl1YAx5jEQwCgybp7
> NZUrUXmTwpseVL9P0/WscUI=
> =AhSs
> -----END PGP SIGNATURE-----
>

Take care:

       David Snelling David . Snelling . UK . Fujitsu . com
       Fujitsu Laboartories of Europe