[ogsa-wg] Re: [cddlm] Basic POSIX Component & template draft

[Hiro Kishimoto]

Hi Steve,

Your file & filesystem components are very good. They are what
I am looking for. One more question I have in my mind is how to
set user-id, group-id, and permissions to deployed files.

Can you call in OGSA-EMS session on Wednesday?
It starts 5:45pm JST = 9:45am UK.

The dial-in number for this session;
   US:    +1 718 3541071 (New York) or
          +1 408 9616509 (San Jose)
   UK:    +44 (0)207 3655269 (London)
   Japan: +81 (0)3 3570 8225 (Tokyo)
   PIN:  4371991

Thanks,
----
Hiro Kishimoto

Steve Loughran wrote:
> Hiro Kishimoto wrote:
>> Hi all,
>>
>> Jun Tatemura and I took a home work assignment about "Basic POSIX
>> Component" for BLAST application deployment.
>>
>> Attached is my first-cut draft and hope to discuss at GGF17 OGSA EMS
>> session.
>> Please have a look and give your feedbacks before or
>> at the GGF17. Your feedbacks are very important.
>>
>> Thanks and see you in Tokyo!
> 
> 
> This is really good, and I do agree, we do need a set of foundational 
> things to deploy.
> 
> One thing I would argue, having learned the lessons of both Ant and 
> SmartFrog, is that a consistent model of file system components comes 
> first. That is, we need components to model
> 
> -files (with a liveness test that verifies the file is present)
> -paths (an ordered list of files/directories)
> -directories (with components that can create a dir on deployment, 
> optionally to delete it and its contents on termination)
> -wildcarded sets of files (**/*.csv)
> -temporary files and directories
> -cached downloads
> -text files
> 
> I'm attaching the PDF file describing the current smartfrog set of 
> components that do this.  The underlying way that this is done is that 
> all components that consider themselves to be sources of filenames/paths 
> set the attribute absolutePath at runtime. They usually also export an 
> RMI interface for RPC-style operations, along side their RESTy state.
> 
> A standard set of filesystem components lets you include the operations 
> to create and clean up directories, temporary files and other 
> housekeeping operations into the workflows of a deployment.
> 
> Looking at the Posix proposal, I'd suggest starting with the reusable 
> set of filesystem types, including a <fs:mount> component that could 
> mount a local or remote filesystem, and which would then pass the local 
> location to interested parties as the filesystem:absolutePath attribute.
> 
> You absolutely don't need to introduce a new shared datatype 
> FileSystemName, because that implies some kind of alternate 
> naming/locating scheme outside of what CDL and the runtime has built in. 
> Dynamic CDL references are sufficient to crosslink information inside 
> the graph of deployed things, and offer flexibility as well as ease of 
> management (you see them when you walk the graph).
> 
> First, we'd need filesystem types:
> 
> <tmp cdl:extends="fs:TempFileSystem">
>    <fs:Description> … </fs:Description>
>    <fs:DiskSpace>
>       <fs:LowerBoundedRange>10737418240.0</fs:LowerBoundedRange>
>    </fs:DiskSpace>
> </tmp>
> 
> <home cdl:extends="fs:Directory">
>    <fs:Description>Chris's home directory</fs:Description>
>    <fs:dir>/home/csmith</fs:dir>
> </home>
> 
> When deployed, both of these would add absolutePath as an attribute,
> so that tmp/absolutePath woudl resolve to, say, tmp/work01234/ and
> home:absolutePath to /home/csmith.
> 
> 
> You'd then mount a blast dir, say on an nfs filesys whose URL you provide
> 
> <blastfs cdl:extends="fs:RemoteFileSystem">
>    <fs:url>nfs://filestore/csmith/blast</fs:url>
>    <fs:mountUser>csmith</fs:mountUser>
> </tmp>
> 
> The database file is under here, and you declare that it must exist. 
> Deployment will fail
> if it is absent:
> 
> <db cdl:extends="fs:File">
>    <fs:dir cdl:ref="/tmp/absolutePath" cdl:lazy="true" />
>    <fs:filename>db/ncbiblast/est</fs:filename>
>    <fs:filemustexist>true</fs:filemustexist>
> </db>
> 
> 
> To use it, just refer to its path
> 
> <blast:database cdl:ref="/db/absolutePath" cdl:lazy="true"/>
> 
> Paths could be represented by some list-like element, whose entries are
> evaluated at deploy time and then a platform-specific path created as a 
> result
> 
> <path cdl:extends="fs:Path">
>   <pathentry cdl:ref="/blastfs/absolutePath" cdl:lazy="true" />
>   <pathentry cdl:ref="/home/absolutePath" cdl:lazy="true"/>
> </path>
> 
> On Windows /path/absolutePath would become something like
>   "\\filestore\csmith\blast;c:\documents and settings\csmith"
> while on unix it could be
>   "/nfs/filestore/csmith/blast:/home/csmith"
> 
> This information now goes down to the jsdl component
> 
> <jsdl-posix:Environment name="PATH"
>   cdl:ref="path:absolutePath" cdl:lazy="true"/>
> <jsdl-posix:Environment name="TMPDIR"
>   cdl:ref="tmp/absolutePath" cdl:lazy="true"/>
> 
> 
> -Steve
> 
> 
> 
> 
> 
> 
>