[DAIS-WG] Large Size Data Handling Issue

[Mike Jackson]

Hi all,

On Mon, 15 Jun 2009, Sunil Ahn wrote:

> Hi All,
>
> This is Sunil Ahn, an AMGA WS-DAIR developer.
>
> I am facing a problem with handling large size data on AMGA WS-DAIR 
> implementation. 
>
> Currently entire SOAP message should be hold in a memory until tne 
> transfer is complete. 
> If the size of message is too large, it may cause server crash or 
> significant perfermance degrade. 
>
> Because WS-DAI & WS-DAIR specification does not take into account of 
> streaming (such as MTOM),
>  I think other implementations(DAIX, DAIR, DAIRDF, ....) face the same 
> problem. 
>
> I think, There should be some support by the specification to solve this 
> issue. 
> There are two possible ways. Maybe more..
> First one is to define a new fault message in the spec (for example, 
> TooLargeSizeDataRequestedFault),

This may be useful and I agree is within the scope of the spec. It is 
analogous to ServiceBusyFault or DataResourceUnavailableFault - providing 
information to the client about the service when the service cannot handle 
a client's request.

>  so that a user having this fault is urged to use indirect data service
>       or retrieve data with several requests.
> Second one is to modify the spec to allow streaming (such as MTOM 
> defined in SOAP 1.2).

The specs, by the indirect access patterns do support streaming. For 
WS-DAIR, for example, the data can be accessed a row at a time.

The suggestion to allow streaming such as MTOM is in my view not a concern 
of the spec. authors. There is nothing in the spec that says that it 
cannot be used, just as there is nothing that says that SOAP must be used.

The specs just specify the service interface to be offered and the 
behaviours of this interface - how this is actually realised is 
implementation-specific. This, of course, raises the problem in that
inter-operability can be affected if one server uses MTOM for WS-DAIR and 
another doesn't - could the same client talk to both? But then could one 
client talk to both SOAP 1.1 and SOAP 1.2 WS-DAIR servers, or a SOAP 1.1 
and non-SOAP WS-DAIR servers? But it could be argued that this is a 
problem for techology providers - ideally a web services client shouldn't 
have to care what binding a web service uses.

But I guess the question for now is how dependant upon other 
specifications or technologies do the spec authors want to be. Or, how did 
other OGF spec authors approach this?

cheers,

mike

> Any comments or ideas would be appreciated.
>
> Best Regards,
> Sunil Ahn.
>
>
> --
>  dais-wg mailing list
>  dais-wg at ogf.org
>  http://www.ogf.org/mailman/listinfo/dais-wg
>

-------------------------------------------------------------------
Dr. Michael (Mike) Jackson      E-mail: m.jackson at epcc.ed.ac.uk
EPCC                            OGSA-DAI Project
WWW:  http://www.epcc.ed.ac.uk  WWW: http://www.ogsadai.org.uk

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.