[jsdl-wg] Regarding Data Staging Elements in JSDL

[Ali Anjomshoaa]

Gargi,

> Is the JSDL-wg also looking at data requirements of jobs for other than 
> file type data.
> For ex. is there scope of representing a database connection required by 
> a job in JSDL.

As far as I'm aware, there are no future plans for explicit data 
resource requirements specifications, such as you suggest, in JSDL.

The JSDL group are looking at the Resource Model development work in OGF 
for future resource requirements specifications, which may include data 
resource requirements.

Ali


> 
> Thanks
> Gargi
> 
> Gargi B Dasgupta
> IBM India Research Lab
> Plot No. 4, Phase II, Block C
> ISID Institutional Area
> Vasant Kunj, New Delhi 110070
> Email:gdasgupt at in.ibm.com
> Phone:+91 11 51292192
> 
> 
> *Ali Anjomshoaa <aanjomshoaa at mobability.co.uk>*
> 
> 04/11/2007 02:26 PM
> 
> 	
> To
> 	"Donal K. Fellows" <donal.k.fellows at manchester.ac.uk>
> cc
> 	Gargi B Dasgupta/India/IBM at IBMIN, jsdl-wg at ogf.org
> Subject
> 	Re: [jsdl-wg] Regarding Data Staging Elements in JSDL
> 
> 
> 	
> 
> 
> 
> 
> 
> Hi Gargi,
> 
> if you've got a workflow framework to use, use it for the data staging
> (either in or out, or both). If you're using JSDL for atomic job
> submissions and not wrapping a bunch of them with a workflow, then use
> the data staging elements provided by JSDL.
> 
> You could also use the JSDL data staging elements if you want to hide
> atomic job related data movement from the workflow layer.
> 
> Cheers and take care,
> 
> Ali
> 
> 
> Donal K. Fellows wrote:
>  > Gargi B Dasgupta wrote:
>  >> My question is:  In case of a workflow, with multiple job steps, and
>  >> data transferring in and out between these steps,
>  >> what is the natural home for these data staging elements.
>  >>
>  >> Is it in the workflow or in the individual JSDL elements. Will the Data
>  >> Staging elements remain in JSDL and then transfers needed in the
>  >> workflow be inferred from these JSDLs. Or will the staging be 
> explicitly
>  >> defined in the flow.
>  >
>  > Excellent question.
>  >
>  > Ideally, I'd imagine that they'd be in the flow, especially as transfer
>  > of a large dataset could take a long time (and considerable resources)
>  > in itself. JSDL deliberately doesn't do this, and this is because it
>  > doesn't assume the presence of an outer flow language (some use-cases
>  > specifically require data transfers in order for even a simple job to
>  > run, so they have to be there). If you've got such an outer language, do
>  > put the transfers there (though you could also use a JSDL document
>  > containing just a DataStaging section to describe the transfers,
>  > resulting in the flow language just joining up a bunch of JSDL docs).
>  >
>  > Donal Fellows.
>  > --
>  >   jsdl-wg mailing list
>  >   jsdl-wg at ogf.org
>  >   http://www.ogf.org/mailman/listinfo/jsdl-wg
>  >
>  >
>  >
> 
> 
> 

-- 
Ali Anjomshoaa PhD
Managing Director

Mobability Limited
Edinburgh, Scotland

www.mobability.co.uk