[Pgi-wg] OGF PGI Requirement JD20 (115) Client requirements independent of the speed of the Computing Resource

Thu Jun 10 06:32:49 CDT 2010

Oxana and all,

Concerning OGF PGI Requirement JD20 (115) 'Client requirements 
independent of the speed of the Computing Resource' :

WLCG is not only discussing this issue, but is really working on it :

Using the current GLUE 1.3, WLCG provides a practical solution, and is 
currently improving its robustness permitting full operational deployment.

This solution, which matches my JD20.3 proposal below, is described at 
http://indico.cern.ch/getFile.py/access?contribId=6&sessionId=0&resId=1&materialId=slides&confId=72057

I will write down this information inside the OGF PGI detailed Wiki page 
for JD20.

Best regards.

-----------------------------------------------------
Etienne URBAH         LAL, Univ Paris-Sud, IN2P3/CNRS
                       Bat 200   91898 ORSAY    France
Tel: +33 1 64 46 84 87      Skype: etienne.urbah
Mob: +33 6 22 30 53 27      mailto:urbah at lal.in2p3.fr
-----------------------------------------------------

On Fri, 30/04/2010 22:22, Oxana Smirnova wrote:
> Hi Etienne,
>
> I have a feeling that there is sometimes a tendency to overcomplicate
> simple things.
>
> You are right in saying that many requirements are formulated in a
> confusing manner.
>
> In fact, it is very difficult to formulate a requirement or a
> definition. It is like describing a face. You often feel like "I
> recognize it when I see it, but can't describe".
>
> With unclear "requirements" one has to understand what *is* the
> requirement. And to do this, one has to understand the use case.
>
> Take your example of "scalable time". The use case is trivial: it makes
> *no sense whatsoever* to specify either CPU time or wallclock time in a
> heterogeneous system. Because there are different hardwares around. You
> *have* to characterize hardware on which this performance was achieved.
> Period. I hate the word "scalable" here because it really steels all
> attention, hinting at implementation details. Who said that execution
> time scales lineraly with benchmark values anyway? This is the real
> requirement:
>
> *** It must be possible to specify hardware characteristics (e.g. in
> terms of a benchmark value) corresponding to the indicated performance
> numbers ***
>
> And mind it, performance numbers in general are not limited to time. It
> might be also memory consumption from another foggy requirement of ours:
> it obviously will be different on 32- and 64-bit architectures.
>
> How will the tools use this information, is a different question.
>
> Cheers,
> Oxana
>
> 2010-04-30 17:24, Etienne URBAH пишет:
>> Balazs, Morris, Luigi, Johannes and all,
>>
>>
>> Concerning the capture and discussion of the requirements :
>>
>>
>> I think that the main problem is NOT the large number of requirements,
>> but the fact that many original requirements were formulated in a very
>> confusing manner.
>>
>> In particular, many original requirements simultaneously present :
>> A) mainly, the WAY the Execution Service would perform a desired
>> functionality,
>> B) sometimes, the INTERFACE which a Client would use to access this
>> functionality,
>> C) implicitly, the NEED for this functionality.
>>
>> This causes us to lose much time in useless discussions, because this
>> requires to use following process :
>> - First, focus on C) to precisely define the functionality, and assess
>> its usefulness,
>> - Second, focus on B) to define the Client interface,
>> - Third, verify that there is no garbage from A) left anymore.
>>
>> Working on the original explicit, implicit or hidden requirements found
>> inside the documents, this is exactly the process which I personally
>> followed, splitting complex confusing requirements into simpler and
>> clearer ones.
>> This permitted that requirements which I personally wrote down could be
>> often quickly understood, discussed, and even sometimes quickly labeled
>> YES (positively agreed) or NO (No agreement yet).
>>
>>
>> For the many requirements which are still unclear, I therefore urge you
>> to accept that I split them into simpler smaller requirements :
>>
>> - Apparently, it will increase the total number of requirements,
>>
>> - But in fact, after an easy work, that will increase the percentage of
>> requirements labeled YES and diminish the percentage of requirements
>> labeled UNCLEAR.
>>
>>
>> As example, please study following requirement, labeled 'Still UNCLEAR
>> after 10 mn discussion' :
>>
>> - JD20 (115) Job Description document should offer a new re-usable
>> structure to define 'scalabletime' requests optionally depending on
>> benchmark results.
>>
>>
>> I will split it into the 4 following requirements :
>>
>>
>> - Description of the needed functionality :
>>
>> JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution
>> Service SHOULD accept Client requirements independently of the speed of
>> the Computing Resource on which the Activity will run
>>
>>
>> - Description of the conceptual Client interface :
>>
>> JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description
>> document SHOULD offer a new re-usable structure permitting to describe
>> requirements independently of the speed of the Computing Resource on
>> which the Activity will run
>>
>>
>> - First option for a precise description of the Client interface :
>>
>> JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable
>> structure permitting to describe requirements independently of the speed
>> of the computing resource on which the Activity will run SHOULD be the
>> ordered pair (Value, Unit) where :
>> - 'Value' is the max number of operations allowed for the Activity,
>> - 'Unit' is as unit suitable for a total number of operations, from a
>> closed list to be refined (such as 'Specint2006 * hours' or 'Billions of
>> floating point operations')
>> This method is already used by BOINC and has proved it robustness.
>>
>>
>> - Second option for a precise description of the Client interface :
>>
>> JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable
>> structure permitting to describe requirements independently of the speed
>> of the computing resource on which the Activity will run SHOULD be a
>> 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue)
>> triplet as described in chapter 7.2 of the Specification Draft
>>
>>
>> I am absolutely certain that these 4 simple requirements will be quickly
>> understood.
>> I am quite sure that we will immediately obtain a YES for JD20.1 and
>> JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO
>> to the other one).
>>
>>
>> In the same way of thinking, I will re-propose following dropped
>> requirements. With sometimes an improved title, they describe the NEED
>> for a precise Client interface, WITHOUT specifying the precise component
>> which will implement them :
>>
>> - IS10 For requests querying Status of finished Activity, the
>> Information functionality MUST accept, as input parameter, 1 Activity ID
>>
>> - IS12 For requests querying Activity Status, the Information
>> functionality MUST accept, as input parameter, complex queries on
>> Activities expressed according to the GLUE2 model without explicit
>> specification of the Activity IDs. The Information functionality MAY
>> support multiple query languages in addition to the mandatory one.
>>
>> - LB4 For all requests, the Query functionality for Activity Logs MUST
>> accept, as input parameter, 1 Activity ID
>>
>> - LB6 For all requests, the Query functionality for Activity Logs MUST
>> accept, as input parameter, complex queries on Activities expressed
>> according to the GLUE model without explicit specification of the
>> Activity IDs. The Logging and Bookkeeping functionality MAY support
>> multiple query languages in addition to the mandatory one.
>>
>>
>> Besides, I just detected that we forgot to study following requirement,
>> which was mentioned in my mail dated 26 April 2010. Is it possible to
>> associate spreadsheet ID 172 to it ?
>>
>> - IS14 In order to prevent overloading the Execution Service (which has
>> performance requirements for Activity Management), the Information
>> functionality SHOULD be separated from the Execution Service
>>
>>
>> Thank you in advance for accepting that, when starting with original
>> confusion :
>> - rigidity prevents improvement and favors endless confusion,
>> - flexibility favors improvement and then clarity.
>>
>>
>> Best regards.
>>
>> -----------------------------------------------------
>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS
>> Bat 200 91898 ORSAY France
>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah
>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr
>> -----------------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Pgi-wg mailing list
>> Pgi-wg at ogf.org
>> http://www.ogf.org/mailman/listinfo/pgi-wg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5073 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.ogf.org/pipermail/pgi-wg/attachments/20100610/29095061/attachment-0001.bin 