[cddlm] In preparation for the CDDLM/ACS joint session at GGF14

[Steve Loughran]

Keisuke Fukui wrote:
> Hi Steve,
> 
> Thanks for your comments.
> Although not complete, here are some comments:
> - We understand the File upload stuff is meant to be a transient/interim
>   and a real repository is expected.
> - We have one use case with commercial data center for ACS, where clients
>   exist outside of the data center and submit their task to the system, 
> then the
>   task is deployed and executed in the system. In this case, it might 
> require
>   much and/or indeterministic latency for component to pull the data 
> outside
>   of the system.
>   Thus, we assumed that it is more reasonable to have data repository 
> inside
>   the system, even though it may not be what the CDDLM specification 
> requires.
>   We assumed that the purpose of having AddFile is to "push" the data
>   from outside to inside of the system in advance. Then components can
>   "pull" the data easier from there.
> - As such, the file server in the case 1 diagram may not be a primary
>   or sole implementation of CDDLM, but we understand it is among possible
>   implementations of larger system under the current CDDLM specification.
>   (We may have had to color it neutral one rather than light blue.)
> - I understood all topics are about the case 1 diagram. Do you have any
>   issues on case 2 or 3?
> 
> BTW,
> Thanks for a heads up for JSR 277. We will keep an eye for it.
> It sounds like the improvement of the jar itself and pretty much java
> focus.
> 
>  -Keisuke
> 

Yes, having a real repository in the data centre is how things should 
really work.

As long as references to content can be supplied as URLS, urls that the 
programs/JVMs on the hosts can resolve, then we could do (2) and (3) 
without a CDDLM implementation needing to know about how those URLs are 
supported. That does imply HTTP/FTP/File, but since both .NET and Java 
have a way of letting you plug in new URL handlers. If you had a new 
url, something like

acs://app/124/component/12

then we could handle it, though I would strongly advocate using HTTP as 
way of retrieving things. not only do all apps support it out the box, 
it is easier to debug in a web browser

There are two more use cases,

-your asset store is used as the back end by an implementation of the 
CDDLM services. That is, someone uses <addFile> to add a file, and the 
request is forwarded to the ACS repository to add a new file to the 
application. Would that work?

-the results of a job are somehow placed into the asset store, for later 
retrieval by the submitter. This is out the scope of CDDLM; whatever you 
deploy needs to handle that submission process.

Asset stores are a trouble spot with me in the past; they have caused 
inordinate amounts of problems, at least when you are trying to use one 
built on MSSQL and classic IIS/ASP. Here are some things that I recall 
being troublesome
  -many file systems cannot have > 65535 files in a single dir, so you 
had better not use too flat a filesystem
  -if the NAS filestore is set to the GMT tz and the server in PST, it 
doesnt make any difference whether or not the clocks themselves are 
syncrhonized; the auto-cleanup process is going to delete new files 
under the assumption that they are out of date.
  -its very hard to secure stuff
  -any HTTP data provider must support HTTP/1.1 or at least 
content-length headers, so that the caller can determine that the 
supplied content was incomplete

As with most things, everything worked in development, it is only when 
you go to production, put the asset store 1200km away from the rendering 
service and keep the files on a different host from the database that 
things start to go wrong.

-steve