[Pgi-wg] Professional Software Engineering (was Gridiron, or standardization gone backwards)

Wed Jul 21 11:42:03 CDT 2010

Oxana and all,

Concerning OGF PGI :

-  Thanks to Oxana for having taken the time to read my long mail, and 
for having provided an answer.

-  For last proposed use case 'Marshal Activities between ARC, gLite and 
Unicore', I fully agree with Oxana that successful standardization 
SHOULD permit to avoid it.  This is why I wrote :
> But we hope that this will NOT be necessary.

Best regards.

-----------------------------------------------------
Etienne URBAH         LAL, Univ Paris-Sud, IN2P3/CNRS
                       Bat 200   91898 ORSAY    France
Tel: +33 1 64 46 84 87      Skype: etienne.urbah
Mob: +33 6 22 30 53 27      mailto:urbah at lal.in2p3.fr
-----------------------------------------------------

On Tue, 20/07/2010 21:25, Oxana Smirnova wrote:
> Hello Etienne,
>
> thank you for the thorough analysis!
>
> Of course, my gridiron analogy was half-joking, but only half ;-)
>
> You are right pointing out problems of non-professional software
> engineering; only PGI was not meant to engineer any software.
>
> PGI wanted to produce BES, JSDL and GLUE2 _profiles_ common for gLite,
> UNICORE and ARC. *That* simple. This is why pre-PGI documents are short
> and simple.
>
> Well, some also wanted to do more dramatic changes to BES, but that was
> not likely to happen.
>
> So, before getting bogged down in state diagrams and port-types again,
> I'd like to know, what are we trying to achieve now, on a high level?
> Are we still into profiling BES, JSDL and GLUE2 for the purpose of
> having a single infrastructure, or do we do something different? Like I
> wrote in other mails, I don't believe we can produce a single worldwide
> Grid infrastructure using all possible technologies; I don't even
> believe it is needed.
>
> And allow me a very high-level comment on the last proposed use case:
> why (conceptually) would it be necessary to "marshal activities between
> ARC, gLite and UNICORE", when the infrastructure will be mixed, when all
> services from different providers will be used interchangeably? When it
> will be straightforward to deploy at one site an ARC computing service,
> a gLite storage and a UNICORE monitoring service? That's why we need
> standards, no? Beckham can move from MU to Real and even to LA Galaxy
> without having to learn new rules. He may decide to move to Miami
> Dolphins, but something tells me they neither will need his bending
> kick, nor will be willing to play by his rules ;-)
>
> Cheers,
> Oxana
>
>
> On 20.07.2010 18:53, Etienne URBAH wrote:
>> Morris, Oxana, David and all
>>
>> Lot of thanks to Morris for his general leadership and his positive mail
>> oriented toward future success.
>>
>> I fully understand that many PGI-WG members are frustrated by the
>> non-achievements and internal struggles, and I try to present below some
>> explanations and proposals.
>>
>>
>> 'Very advanced draft' produced by the PGI founders in September 2008
>> --------------------------------------------------------------------
>> Did this 'Very advanced draft' ever exist ?
>>
>> Inside GridForge PGI documents, the 'Input_Documents/Pre-OGF_material'
>> contains various little files, some of which containing only some
>> fragments of requirements or specifications.
>>
>> Only on 11 May 2009 did Balazs KONYA upload the first version of the
>> 'Rendering of the GES strawman' in GridForge at
>> http://forge.gridforum.org/sf/go/doc15628?nav=1 with comment 'initial
>> rather incomplete draft'.
>>
>> NONE of the previous files clearly describes :
>>
>> - the context :
>> - Distributed resources managed inside separate administrative domains,
>> - Local and Grid-level credentials,
>> - Computing resources forbidden (or authorized) to have internet access,
>> - Input files available inside the Grid or only available on the local
>> machine of the Submitter of the Activity,
>> - ...
>>
>> - the problem(s) to be solved (AUTHN, AUTHZ, Delegation, Automatic /
>> Manual Data Staging, ...),
>>
>> - a summary of the solution(s) proposed to solve the problem(s),
>>
>> - the interactions between the software components used in the
>> solution(s),
>>
>> - the semantics and syntax of the data that the software components have
>> to exchange,
>>
>> - the sequence of the exchanged messages, and their relationships with
>> states of the software components.
>>
>>
>> Only on 31 July 2009 did Moreno MARZOLLA upload the 'Strawman
>> functionality' in GridForge at
>> http://forge.gridforum.org/sf/go/doc15736?nav=1
>>
>> At that time, PGI should have worked to validate this foundational
>> document, which contains at least some hints about the above topics.
>>
>> But instead of working on the validation of this foundational 'Strawman
>> functionality' document, PGI regrettably continued struggling on the
>> 'Rendering of the GES strawman'.
>> This document regrettably dives directly into Port-Types, which are
>> details of communication protocol totally irrelevant at this stage.
>>
>>
>> They didn't have to waste time on formalizing use cases and
>> requirements, because all these were in their heads.
>> -----------------------------------------------------------
>> That is the main cause of FAILURE for most projects :
>>
>> Founders ERRONEOUSLY think that the use cases and requirements are clear
>> in their heads, but in most projects, this proves FALSE, and
>> misunderstanding pops up everywhere.
>>
>> In particular, I is clear that the founders do NOT agree on what an
>> Activity ID should be, and do NOT understand even now the relationships
>> of this issue with the stability of the Endpoint of the Execution
>> Service (Single Endpoint, Factory + Manager, Transient Endpoints).
>> This is the reason why I have published the 3 corresponding Sequence
>> Diagrams inside the 'Input Documents / Execution Service' folder.
>>
>> Using the lessons of dozens of years of bad organization and thousands
>> of projects with bad software design, the most talented researchers
>> (Booch, Jacobson, Rumbaugh, ...) have gathered best practices and
>> propose methods permitting stakeholders to :
>> - understand the problems,
>> - write them down in a manner understandable for others,
>> - begin their work with foundations which will not collapse too quickly.
>>
>> Just like Alchemy used the rules of science to evolve into Chemistry :
>> Computing is using methods like CMMI or ITIL, and languages like UML, to
>> evolve into professional Software Engineering and Operation. In
>> particular, we have to consider :
>> - Use Cases,
>> - Requirements,
>> - Use Case Diagrams,
>> - Class Diagrams,
>> - Collaboration Diagrams,
>> - Sequence Diagrams,
>> - State Diagrams,
>> - Deployment Diagrams,
>> - Flow Charts,
>> - ...
>>
>> Professionals preferably begin with Use Cases. Therefore, Use Cases are
>> often the foundation of the whole Software Engineering work.
>> In order that these foundations are good, it is absolutely necessary
>> that the Use Cases correctly describe the System, the Actors, the
>> Preconditions, the Interactions between the Actors and the System, ...
>> This is the reason why I strongly reject a bad Use Case Template.
>>
>>
>> Analogies with 'Gridiron' and with 'Foreign people driving foreign cars'
>> ------------------------------------------------------------------------
>> I am afraid that one of the main problem is that 'Gridiron' is an
>> appropriate analogy for simple computing with different programs on
>> different machines, which is far simpler that our real problem.
>>
>> Though, 'Gridiron' can still propose some useful foundation rules, such
>> as :
>> - Players must respect each other,
>> - In order to accommodate variations in 'Gridiron' rules, the whole
>> playground must be large enough, the detailed surfaces of play should be
>> delimited with washable or removable paint, and there must be an
>> available assortment of balls of different forms and sizes.
>> - ...
>>
>>
>> But 'providing middleware for a distributed infrastructure managed
>> locally by separate administrative domains and authorizing users to
>> submit activities on remote resources' is VERY FAR from simple computing.
>>
>> This is a completely different story, of much higher complexity, having
>> analogy with 'Foreign people driving foreign cars' :
>>
>> - In each country, traffic is managed by the local police, which has the
>> right to stop any car and/or any driver.
>>
>> - In order to be allowed in a given country :
>> - A driver must have a valid driver license,
>> - A car must have a valid plate number and a valid insurance.
>>
>> - Generally, countries trust each other, so that
>> - A driver license issued in one country is valid in all other countries,
>> - An insurance issued in one country is valid in all other countries.
>> (But currently, my own car insurance is NOT valid for Irak and North
>> Korea)
>>
>> - In order to ease driving and prevent accidents :
>> - Traffic is everywhere on the right side of the road (except in the
>> United Kingdom !)
>> - All indications, and especially warnings, should be given with
>> commonly agreed pictograms
>> (Do you all understand what 'Pont effondré' means ?)
>> - Street names should be indicated using street plates
>> (But street plates are parallel to the street in France, and as far as I
>> remember, perpendicular to the street in the USA)
>> - If necessary, street plates should display a commonly agreed Latin
>> transliteration
>> (Ever tried to find 'Avenue Croutchef', who is a former Soviet Union
>> leader ?)
>>
>>
>> *** use cases must be provided by users
>> ---------------------------------------
>> Theoretically, yes.
>>
>> But computing scientists can only provide 'computing' use case.
>>
>> And our problem is NOT 'computing', but 'providing middleware for a
>> distributed infrastructure managed locally by separate administrative
>> domains and authorizing users to submit activities on remote resources'.
>>
>> Who are able to understand this problem, and write down relevant Use
>> Cases ?
>> - Surely the members of the teams having designed and now operating WLCG,
>> - Perhaps the members of OGF PGI-WG, if we accept professional Software
>> Engineering.
>>
>> Therefore, I suggest that each member of PGI-WG :
>> - carefully studies the 2 Use Case Templates and the 9 Use Case
>> documents published,
>> - if necessary seeks advise of professionals working daily on Use Case
>> capture,
>> - makes his/her own mind,
>> - presents his/her position.
>>
>>
>> use case where these different systems must be brought together
>> ---------------------------------------------------------------
>> 1) http://forge.gridforum.org/sf/go/doc16010?nav=1
>> 'Enforce Security of a Production Service Grid'
>>
>> 2) http://forge.gridforum.org/sf/go/doc16022?nav=1
>> 'from Service Grid to Desktop Grid'
>> The EDGI project will implement this Use Case for :
>> - gLite, ARC and Unicore as Service Grid middleware,
>> - BOINC, XtremWeb-HEP and perhaps also OurGrid as Desktop Grid middleware
>>
>> 3) If absolutely necessary, the EDGI project is also able to implement
>> following Use Case : 'Marshal Activities between gLite, ARC and Unicore'
>> But we hope that this will NOT be necessary.
>>
>>
>> Let's work with enthusiasm and professionalism !
>>
>> Best regards.
>>
>> -----------------------------------------------------
>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS
>> Bat 200 91898 ORSAY France
>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah
>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr
>> -----------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5073 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.ogf.org/pipermail/pgi-wg/attachments/20100721/e633d9d9/attachment-0001.bin 