[ogsa-wg] Re: GGF/OGSA standards for hierarchical namespaces

[Hiro Kishimoto]

Chris's email forwarding.
----

Thanks to everyone for useful responses, and to both Manuel and Mark
for providing workable hierarchical, abstract naming schemes
("directory standards", in the context of this discussion), each with
valuable features. I'm writing this note as a preface to the
discussion of naming at the F2F meeting today, and as a response to
some of the points already made in this thread. I'll mostly just be
throwing out a few vaguely connected thoughts and pieces of
information to help advance the starting point for the discussion
today. As important as I feel it is for the community to quickly
agree upon a definite direction/core draft standard, it's also
crucial to get it right - we don't necessarily have to provide a
perfect solution the first time, but we do have to provide a solution
that can grow and change organically to fulfill new needs, as the
community creates them.

Just to update my previous e-mail regarding the options and
suggestions out there, I've spoken with Osamu Tatebe, and it appears
that the RNS item in my initial list of efforts and proposals is now
coincident with Dave Berry's post-GGF16 note suggesting that RNS be
re-factored into at least two separate standards. In this regard,
Osamu says "At the last GFS-WG session in GGF16, we agreed to
refactor RNS specs into (at least) three specs; RNS, iterator,
logical name resolver." It's not immediately clear to me what is
meant by "RNS" in this quote, but my guess would be that it refers to
the directory standard/service itself. My understanding is that
Manuel Pereira is currently working on this refactoring, and will
offer a draft to the GFS (and likely OGSA-Naming) working groups for
further discussion, once a workable draft is complete.

To clarify my own ulterior motives, my goal is to get the work on
naming and directory-related issues moved into the OGSA-Naming group,
even if it is the same set of players doing the work, so that the GFS-
WG can focus on the requirements for file systems within a "directory
and logical/abstract naming system" - for example, a GFS directory or
"file" entry should have properties such as size, support information
about the source storage resource and access mechanisms, replication
properties, the data's tier within an HSM, etc etc. I'm hoping that
we can all work together to create a set of directory naming and
resolution standard(s) that can support these needs, and I believe
that if we do, they will also support just about any other function
for which an abstract-name-based directory system is necessary.

 From the above paragraph, I think it should be clear that I believe
that it is a requirement to be the able to associate arbitrary
attributes/properties with entries in the directory structure. What I
envision here is a simple set of base standards, ideally to come out
of the OGSA-Naming group; one for hierarchically organized
collections of human-readable abstract names with arbitrary
properties attached, each of which resolves to an EPR, and one for an
iterator interface, where the items being iterated over are the
abstract names. Each standard utilizing hierarchical collections of
human-readable names, including the GFS-WG, would then define a
"profile" for use of the directory service, including requirements
for the properties which should be associated with a directory entry,
and any higher-level services which would ideally build upon the
"directory" standard I'm hoping to see.

I believe that the WS-Directory proposal, certainly in the v3
iteration Mark Morgan referred us to in an earlier e-mail, would
fulfill the requirements I have for hierarchically organized
collections of human-readable names, referring to EPRs and having
attributes associated with them. We still need, in my opinion, an
iterator interface standard to work in concert with these
"directories"; hopefully Manuel's work with refactoring RNS will
provide us with that.

I'll stop here, since I believe the F2F discussion of naming will
begin shortly. Thanks again to everyone for participating in this
discussion so far, and I look forward to continuing it both in the
F2F and through other channels.

On Mar 28, 2006, at 11:17 AM, Allen Luniewski wrote:

>
> Chris,
>
> Thanks for the very thoughtful note!  A few comments.
>
> I agree with you that the current situation is, to put it mildly,  
> not acceptable.  We need to move to a position where the community  
> agrees on a single means for creating a hierarchy of pointers to  
> resources (a directory structure in Unix-speak).  Clearly one  
> aspect of this is that RNS and WS-Directory need to reconciled into  
> a single proposal.   Resolution sooner rather than later is clearly  
> in the best interests of the various WGs who depend upon  
> directories (including OGSA Data which I share responsibility for).
>
> You mention a couple of specific technical issues.  A few comments  
> on those:
>         1. I go back and forth on attributes in the directory  
> system.  Since these are almost certainly cached from the  
> resources, today I am inclined to say that
>                 attributes should not be part of the directory  
> system.  Ask me tomorrow, and I may have the other answer :-)  But  
> today, I would leave them out
>                 of the base specification for simplicity but  
> consider an extension that included attributes if that were felt to  
> be vital.
>         2. Dave Berry's suggestion to separate out the iterator  
> component is a good one.  There are going to be many places in the  
> overall grid standards
>                 effort where an iterator-like structure will be  
> needed. Standardizing this seems like a proper goal for GGF.  If  
> that is accepted then using it
>                 in a directory service seems quite natural.
>
> As for moving forward, I think that we need to see how this thread  
> plays out.  My instincts, however, are that getting the interested  
> parties in a room for a few hours would be the most effective way  
> to drive this to an early resolution.
>
> Allen Luniewski
> IBM Information Management Division
> San Jose, California
>
>
>
> Christopher Jordan <ctjordan at sdsc.edu>
> Subject: GGF/OGSA standards for hierarchical namespaces
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> All,
>
> Forgive the wide distribution on this e-mail, but this issue seems to
> be to be both extraordinarily important to the future of GGF/OGSA
> standards efforts and also in a state of either limbo or paralysis.
> The topic I'm addressing here, both in my capacity as the secretary
> of GFS-WG and as a generally interested participant on a few
> different GGF working groups, is the question of adopting a single,
> possibly minimal, standard for creating hierarchically organized
> collections of pointers (WS-Names? GSR/GSHs, to date myself?) to
> "resources", where the term "resource" could denote a service
> providing access to a collection of files, computational resources,
> or database records (that's a non-exclusive list), and where some
> items in the hierarchy could actually represent directory-like
> structures, i.e. containers for other collections of resources.
>
> The way I got involved in this discussion through the Grid File
> Systems-WG, which at the time was bringing the RNS specification
> forward for final approval as a GFD. Subsequently, there have been
> numerous discussions outside of the GFS-WG context about the
> suitability of the RNS standard for more general applications, as
> well as the (perceived) complexity of the standard as a barrier to
> entry. There have also been alternative directory construction
> standards proposed by members of the OGSA-Naming-WG.
>
> The following are the activities/proposals I know of:
>
> RNS: I know the GGF editors have returned the final(?) RNS draft to
> GFS-WG, with the suggestion that it is too specific to filesystem
> needs, and the suggestion that it either be limited in scope to GFS
> applications only (a non-optimal solution for obvious reasons) or
> that the authors work with the OGSA-Naming people to help develop a
> universal standard for hierarchical resource namespaces. If we are to
> move forward with RNS, one of these options will clearly be a
> necessity, given the points Greg Newby made in his responses on
> behalf of the GFSG.
>
> WS-Directory: This is the hierarchical namespace standard developed
> at UVa in response to their difficulty in implementing the
> complexities and ambiguities in RNS. I like the simplicity of WS-
> Directory, however it seems to be missing significant requirements
> for general use such as attributes, both attributed which should be
> required such as time-to-live, and the ability to add extensibility
> attributes such as resource type, QoS, etc. This ability to add
> arbitrary attributes is present in RNS but it still lacks some
> obviously fundamental required attributes.
>
> Finally, Dave Berry sent an e-mail immediately after GGF16 in which
> he mentioned the suggestion that we separate this functionality into
> two logical functions, and therefore standards - a Directory
> Interface and an Iterator interface, in which Directory interfaces
> were essentially just pointers to Iterators, which would be
> standardized. However, there would be no restriction that a Directory
> point to a particular type of iterator interface. One point I wasn't
> clear on from the e-mail was whether an entry in an interator could
> be another directory, although I suspect it can.
>
> This short list is what I've got within easy reach. As I said
> previously, I believe this is an important issue to resolve quickly,
> and I'm sending this note in the hopes of initiating the conversation
> among as many of the relevant parties as I can. Please feel free to
> forward at will, respond with agreement, anger, or even unconcealed
> rage.
>
> Possible ways forward would be for us to have a conference call (GFS-
> WG meets rarely, and we could quite easily give up our call for a
> more focused discussion of these issues), an extended e-mail
> discussion, or a meeting at the next GGF (assuming we get a chance).
>
> Let me know how you feel about the options presented above, or feel
> free to propose new ones if you like. The important thing is that we
> begin to gain momentum, and then keep it going forward.
>
> Thanks.
>
> N.B. For anyone who may have missed any of the discussions reference
> above, please let me know and I'll be happy to forward them to you
> from my archives.

- ----------------------------------------------------
Chris Jordan
HPC Systems Engineer
High End Computing Systems Group
San Diego Supercomputer Center
ctjordan at sdsc.edu
858.534.8347

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEM/xVPCVtcXn6kg8RArY1AKCO4lAdeWchGcMgDcmDl1YAx5jEQwCgybp7
NZUrUXmTwpseVL9P0/WscUI=
=AhSs
-----END PGP SIGNATURE-----