[Pgi-wg] OGF PGI-JSDL : For resource requirements, proposal to use benchmarks as measurement units
Etienne URBAH
urbah at lal.in2p3.fr
Thu Sep 22 15:19:30 CDT 2011
Oxana,
During OGF PGI Session 2 at OGF 33 on Wednesday 21 September
11:00-12:30, for JSDL resource requirements, we agreed to present a
proposal for using benchmarks as measurement units.
This relates to PGI requirement 115 traced at
http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD20
Practical Problem
-----------------
For a given input data set, a user knows, by summing the CPU usage of
individual cores, that his application needs 3 hours of CPU in total on
a reference machine where the 'SPEC CFP 2006' floating point benchmark
rates each CPU at 30.
The user wants to send a Job running the same application on the same
input data set.
At Job submission time, the user is NOT able to know the 'SPEC CFP 2006'
rating of the machine which will eventually run the Job.
Still, the user wants to express in the Job Description document that
the total CPU time limit for his Job must be large enough.
Solution in English
-------------------
The user does NOT know how well the applications of the 'SPEC CFP 2006'
benchmark use multithreading. So, by security, this user rates each of
the cores of his reference machine at maximum 30 'SPEC CFP 2006'.
So, the total number of 'SPEC CFP 2006' operations needed by the Job is
at most :
3 * 3600 * 30 = 324000 (s * 'SPEC CFP 2006')
The user then wants to express following resource requirement in the Job
Description document :
Using the GLUE2 model, the 'MaxTotalCPUTime' attribute of the
'ComputingShare' entity must be at least 324000 divided by the 'Value'
attribute of the 'Benchmark' whose 'Type' is 'cfp2006'.
Solution in short human-readable pseudo-code
--------------------------------------------
Job Definition:
Job Description:
Resource Requirements:
GLUE2.ComputingShare.MaxTotalCPUTime >= 324000 /
(GLUE2.Benchmark.Value where GLUE2.Benchmark.Type = 'cfp2006')
Same solution in human readable pseudo-code with key=value statements
---------------------------------------------------------------------
Job Definition:
Job Description:
Resource Requirements:
Resource With Unit:
Resource Reference = GLUE2.ComputingShare.MaxTotalCPUTime
Lower Bounded Range = 324000
Resource Unit Reference = GLUE2.Benchmark.Type
Resource Unit Name = 'cfp2006'
Resource Unit Value = GLUE2.Benchmark.Value
Simple solution in JSDL using a new 'jsdl+dx:BenchmarkType' element
-------------------------------------------------------------------
This solution, which is simple for JSDL, requires that in the Execution
Service, a reference to 'jsdl+dx:BenchmarkType' is hard-coded as a
lookup from 'GLUE2.Benchmark.Type' to 'GLUE2.Benchmark.Value'.
I also use the fact that :
jsdl:TotalCPUTime == GLUE2.ComputingShare.MaxTotalCPUTime
<jsdl:JobDefinition>
<jsdl:JobDescription>
...
<jsdl:Resources>
<jsdl:TotalCPUTime>
<jsdl:LowerBoundedRange> 324000.0 </jsdl:LowerBoundedRange>
<jsdl+dx:BenchmarkType> cfp2006 </jsdl+dx:BenchmarkType>
</jsdl:TotalCPUTime>
</jsdl:Resources>
...
</jsdl:JobDescription>
</jsdl:JobDefinition>
Complex solution in JSDL using a new 'jsdl+dx:ResourceWithUnit' element
-----------------------------------------------------------------------
This solution, which is more complex for JSDL, provides the necessary
information permitting the Execution Service to perform the lookup
dynamically and prevents hard-coding.
<jsdl:JobDefinition>
<jsdl:JobDescription>
...
<jsdl:Resources>
<jsdl+dx:ResourceWithUnit>
<jsdl+dx:ResourceReference>
GLUE2.ComputingShare.MaxTotalCPUTime
</jsdl+dx:ResourceReference>
<jsdl:LowerBoundedRange>
324000.0
</jsdl:LowerBoundedRange>
<jsdl+dx:ResourceUnitReference>
GLUE2.Benchmark.Type
</jsdl+dx:ResourceUnitReference>
<jsdl+dx:ResourceUnitName>
cfp2006
</jsdl+dx:ResourceUnitName>
<jsdl+dx:ResourceUnitValue>
GLUE2.Benchmark.Value
</jsdl+dx:ResourceUnitValue>
</jsdl+dx:ResourceWithUnit>
</jsdl:Resources>
...
</jsdl:JobDescription>
</jsdl:JobDefinition>
Thank you in advance for studying these proposals, and for providing
comments and suggestions for improvement.
Best regards.
-----------------------------------------------------
Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS
Bat 200 91898 ORSAY France
Tel: +33 1 64 46 84 87 Skype: etienne.urbah
Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr
-----------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3882 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.ogf.org/pipermail/pgi-wg/attachments/20110922/e99a44d6/attachment.bin
More information about the Pgi-wg
mailing list