[jsdl-wg] Parameter Sweep discussions

[geoff]

Hi Steve,

The idea which we're trying to develop is that we allow for both 
changing elements within the JSDL document (as per current draft param 
sweep spec) and also, as you implied, try to define a means of sweeping 
over placeholders in input files.

Rather than use our own space for exchanging ideas we (Byungsang, David, 
Soonwook and myself) would like to use the SourceForge wiki if this were 
possible so as to allow all interested wg members to take part and/or 
follow ideas. I'll ask on Wednesday's phone session to see if this is 
possible.

gef

A S McGough wrote:
> Dear Geoff, all,
> 
> This is an interesting variation on the parameter sweep application. If 
> I understand you correctly rather than changing elements within the JSDL 
> document (or even specifying different input files) your approach is to 
> edit the contentse of an input file itself? I'm intrigued to know how 
> you do this - I can imagine some form of sed/awk/ed scripting could 
> achieve this in Unix.
> 
> Withouth knowing how you are doing this I can't say weather this is in 
> scope of JSDL or OGF. However, it could resolve one of the issues with 
> JSDL parameter sweep where "if you need to execute 1000 jobs each with 
> (minimally changed) input files you have to send 1000 input files" - 
> this is obviously inefficent.
> 
> We should definatly bring this up at a JSDL call.
> 
> steve..
> 
> geoff wrote:
> 
>> Dear Michel, Andreas, and others group members,
>>
>>
>> A group of us currently meeting in KISTI, S.Korea have been sharing
>> ideas and considerations for the implementation of a parameter sweeping
>> mechanism to support the research community. Input has principally come
>> from David Meredith (NGS), Soonwook Hwang and Byungsang (KISTI) and
>> myself. David is the creator of the UK National Grid Service JSDL
>> Application Repository portal (https://portal.ngs.ac.uk), and has been
>> tasked with creating a parameter sweeping mechanism for the NGS portal
>> (I am currently working with David on implementing this functionality).
>> Naturally we would like to adopt JSDL sweep related standards. Soonwook
>> and Byungsang are in charge of development of KISTI's AURORA Application
>> Repository (http://aurora.kisti.re.kr), which has been in use for some
>> time and uses its own schema definition for parameter sweeping.
>>
>>
>> AURORA's mechanism is loosely aligned with JSDL, but focuses on the
>> sweeping of parameters within input files, rather than sweeping over
>> elements defined within the JSDL document itself. We consider this to be
>> an important additional functionality. Indeed, David has already
>> reported on a number of (bespoke) parametric job definition schemes used
>> to sweep over parameters within input files which have been developed as
>> a matter of requirement by a number of different NGS user communities.
>>
>> The JSDL parameter sweeping specification in its current draft form
>> appears to provide the flexibility to sweep over any element within a
>> JSDL document, and as such provides powerful flexibility to the user.
>> >From our experience however, many applications (for example KISTI and
>> NGS supported NSCCS applications, e.g. CFD Solver, AutoDock) require the
>> ability to sweep over values within input files. Consequently, where
>> possible, we would like to propose further extensions to the existing
>> draft sweep schema specification for including; a) the definition of
>> parametric variables that can exist within input files, and b) referral
>> to the associated data\input files.
>>
>>
>> We were therefore wondering whether the same conclusion had been reached
>> by those contributors to the original draft specification or indeed,
>> whether you all feel that such an approach falls outside the remit of
>> the OGF standard specification. Maybe we could organize a joint phone
>> chat to discuss some of these issues and offer ideas and suggestions. We
>> will working together until late Friday so if it's possible to have some
>> feedback to these queries during this period it would be very much
>> appreciated.
>>
>>
>> Best wishes,
>>
>>
>> Geoff, Byungsang, David, Soonwook.
>>
>> Note,
>>
>> **KISTI – Korea Institute for Science Technology Information
>>
>> **NSCCS – National Service for Computational Chemistry
>>
>>
>>
>> -- 
>>   jsdl-wg mailing list
>>   jsdl-wg at ogf.org
>>   http://www.ogf.org/mailman/listinfo/jsdl-wg
> 
> 
>