[ogsa-dmi-wg] StartNotBefore element in Data Transfer

[Michel Drescher]

Hi Shahbaz,

of cure, you are right - well spotted.

I double-checked my implementation, and again the Functional  
Specification (draft 64). The Functional Specification is correct and  
clear when describing the Start operation and the [start time]  
attribute of the DTI. However, the Functional Specification is  
ambiguous when describing the [start not before] transfer requirement.

 From discussions with Steve around the same topic I remember what we  
resolved:
If no StartNotBefore (SNB) is given in the transfer requirements, then  
the DTI stays in the Created state until the EndNoLaterThan  
requirement has passed by (or until the system cleans up because of  
some policy), or until the user invoked the Start operation.
If a SNB has been specified in the transfer requirements, then the DTI  
MUST have a [start time] attribute, and it will automatically advance  
to the Scheduled state once the [start time] has passed. In this use  
case the user MAY try and start the transfer *before* the [start time]  
has passed by, ut the DTI MAY reject.

For the group: I think we need to slightly reword the Functional  
Specification better describing the StartNotBefore transfer requirement.

It currently reads as:
"This represents the [startNotBefore] information element. The data  
transfer encapsulated by the DTI MUST NOT start before the date, time  
and time zone specified by this element. If not specified its default  
value will be the date and time that the DTI was created. The DTI MUST  
NOT enter the transferring state before this time unless the “Start”  
operation on the DTI is invoked."

I propose to replace that with the following paragraph:
"This represents the [start not before] information element. The data  
transfer encapsulated by the DTI MUST NOT start before the date, time  
and time zone specified by this element. If not specified then the DTI  
MUST wait for a client to invoke the Start operation before it can  
enter the Scheduled state."

By standards process we are safe as we are allowed to submit a changed  
version of the Functional Specification along with the Interop  
Experiences document.

Cheers,
Michel

On 15 Oct 2008, at 15:18, Mohammad Shahbaz Memon wrote:

> Thanks Michel.
>
> I have another question pertaining to the plain-ws interop details
> posted on dmi wiki.
>
> There is a test case No. 5 (if its still valid)
>
> which says :
>
> ------------------------------------------------------
> Description:
> The Client asks the Factory to create a Data Transfer Instance.
> The source and sink EPRs, and the transfer requirements must be
> suitable to create a DTI instance. The TransferRequirements MUST be
> empty.
> After obtaining the EPR to the DTI instance the client then asks the
> DTI instance for its attributes.
>
> Expected outcome:
> The DTI instance returns its attributes without faults.
>
> Success Requirements:
> - The instance attributes list MUST contain a valid set of attributes.
> - The Status attribute MUST have the value "Created".
> - The instance attribute list MUST NOT have a value for StartTime.
> ----------------------------------------
>
> If you see the success requirements, it says the status must have a
> Created value and instance attributes with the start time. There is a
> contradiction between this success req. and the way you described in
> the last email, because it says if SNB is not specified then transfer
> will immediately start which could possibly lead this transfer to the
> scheduled/transferring/done state depends upon the job execution at
> the backend.
>
>
> Thanks
>
> Shahbaz
>
>
>
>
>
> On Wed, Oct 15, 2008 at 12:12 PM, Michel Drescher
> <Michel.Drescher at uk.fujitsu.com> wrote:
>> Hi Mohammad,
>>
>> for your reference, I am referring to draft 64 of the functional
>> specification.
>>
>> The main use case for the SNB requirement lies in resource  
>> allocation and
>> reservation. While either is out of scope for DMI they nonetheless  
>> must be
>> accommodated for. So consider a request for transferring 10 TB of  
>> data. Sure
>> enough you need to check and reserve disk space at the sink, and most
>> certainly also some bandwidth for the transfer itself.  
>> Implementations of
>> DMI my do reservations transparently, but often it is not. Hence a  
>> client to
>> DMI (not necessarily the user) reserves some bandwidth and storage  
>> space,
>> and makes sure that the transfer itself starts not before the  
>> reservation is
>> activated.
>>
>> The most important thing to understand is that the events and  
>> operations in
>> the state diagram for the DTI may happen explicitly, e.g. by  
>> sending the
>> corresponding WS message to the DTI, or implicitly or  
>> automatically. For
>> example, a client to a DTI may invoke  the "stop" operation,  
>> triggering the
>> FAILED  event causing a running transfer to fail. But equally the  
>> DTI itself
>> may trigger the FAILED event, possibly because the EndNoLaterThat  
>> (ENLT)
>> requirement must be observed.
>>
>> Having said that, the semantics for a DTI are as follows:
>> In any circumstance, when a DTI is created it is always in the  
>> "Created"
>> state.
>> From there on, if a SNB is *not* given, then the implicit SNB is  
>> set to the
>> time the DTI is created. Then the DTI waits until the SNB has  
>> passed before
>> triggering the STARTED event by itself. During that time the DTI  
>> MUST stay
>> in state "Created". That implicitly means that, if no SNB has been  
>> given
>> beforehand, that the DTI more or less immediately triggers the  
>> STARTED
>> event, because the implicit SNB has already passed. In other words,  
>> if the
>> client does not specify a SNB then the transfer is supposed to start
>> immediately.
>> It is slightly different though in the case when the SNB indeed  
>> lies in the
>> future. As said before, the DTI sits and waits until the configured  
>> SNB
>> passes by. In that time, a client to the DTI is allowed to invoke  
>> the DTI's
>> start() operation to attempt to start the transfer even though a  
>> SNB has
>> been previously given. The DTI however, may decide to either accept  
>> or
>> decline the start request, depending on the individual situation.  
>> If it
>> declines the request the DTI MUST send back a  
>> FailedStateTransitionFault
>> with the appropriate explanation. If it accepts the start request,  
>> it will
>> send back the correct response, and advance to the SCHEDULED state.
>>
>> It is important to note that the "Scheduled" state is named quite
>> ambiguously. This "Scheduled" state has nothing to do with the SNB  
>> in the
>> DMI sense. It is present in the state machine simply because in most
>> situations, there is a time gap between that point in time of  
>> telling the
>> underlying transfer protocol to move bytes, and the time of the bytes
>> actually hitting the wire. During this time gap the DTI MUST be in  
>> state
>> "Scheduled". Again, the proper state for the STI to be in while  
>> waiting for
>> the SNB to pass by is the state "Created".
>>
>> --
>>
>> I hope that, though quite long, the explanation gives you enough  
>> information
>> to proceed.
>>
>> Cheers,
>> Michel
>>
>> On 15 Oct 2008, at 10:01, Mohammad Shahbaz Memon wrote:
>>
>>> Hi Michel,
>>>
>>> in DMI, I am bit skeptical in the use of startnotbefore element of
>>> transfer requirement.
>>>
>>> Considering a scenario where client creates a transfer by specifying
>>> startNotBefore(SNB) element in the transfer requirements. Shall the
>>> transfer initialization be started automatically just after the  
>>> client
>>> calls DataTransferFactory.getDataTransferInstance() or the  
>>> scheduling
>>> will start after client calls the DataTransferInstance.Start()?
>>>
>>> If I guess, the scheduling starts after the GetDatatransferInstance
>>> then client doesn't need to send Start method call, however client  
>>> can
>>> send start calls which will result in a throw incorrect statefault.
>>>
>>> Thanks,
>>> --
>>> ------------------
>>> Mohammad Shahbaz Memon
>>> Distributed Systems and Grid Computing
>>> Jülich Supercomputing Center
>>> Forschungszentrum Jülich GmbH
>>> Jülich Germany
>>>
>>> Office: +49 (0)2461 61 6567
>>> Fax:     +49 (0)2461 61 6656
>>> http://www.fz-juelich.de/jsc
>>>
>>> Sitz der Gesellschaft: Jülich
>>> Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498
>>> Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe
>>> Vorstand: Prof. Dr. Achim Bachem (Vorsitzender),
>>> Dr. Ulrich Krafft (stellv. Vorsitzender)
>>
>> --
>> Michel Drescher
>> Fujitsu Laboratories of Europe, Ltd.
>> Hayes Park Central
>> Hayes End Road
>> Hayes, Middlesex UB4 8FE
>> Reg. No. 4153469
>>
>> +44 20 8606 4834
>> Michel.Drescher at uk.fujitsu.com
>>
>>
>>
>>
>>
>
>
>
> -- 
> ------------------
> Mohammad Shahbaz Memon
> Distributed Systems and Grid Computing
> Jülich Supercomputing Center
> Forschungszentrum Jülich GmbH
> Jülich Germany
>
> Office: +49 (0)2461 61 6567
> Fax:     +49 (0)2461 61 6656
> http://www.fz-juelich.de/jsc
>
> Sitz der Gesellschaft: Jülich
> Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498
> Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe
> Vorstand: Prof. Dr. Achim Bachem (Vorsitzender),
> Dr. Ulrich Krafft (stellv. Vorsitzender)

--
Michel Drescher
Fujitsu Laboratories of Europe, Ltd.
Hayes Park Central
Hayes End Road
Hayes, Middlesex UB4 8FE
Reg. No. 4153469

+44 20 8606 4834
Michel.Drescher at uk.fujitsu.com