[gin-data] Re: gridftp between sites
William E. Allcock
allcock at mcs.anl.gov
Fri May 5 10:10:18 CDT 2006
Yes, I will be there to discuss it.
Bill
> -----Original Message-----
> From: owner-gin-data at ggf.org [mailto:owner-gin-data at ggf.org]
> On Behalf Of Erwin Laure
> Sent: Friday, May 05, 2006 9:12 AM
> To: gregor at mcs.anl.gov
> Cc: gin-data at ggf.org; Gregor von Laszewski
> Subject: Re: [gin-data] Re: gridftp between sites
>
> Hi Gregor,
>
> Many thanks for this input. Will you or Bill be in Tokyo next week? I
> think it would be best if we could discuss this in the data
> mgmt part of
> the GIN meeting on Thursday.
>
> Cheers,
>
> -- Erwin
>
> Gregor von Laszewski wrote:
> > PS: some screenshots are now at
> > http://wiki.cogkit.org/index.php/Testing
> >
> > Naturally we can make them more pretty, but the focus here was on
> > functionality. We could naturally report the results in some xml
> > document that you than could than render yourself, but we
> thought the
> > simple html page we have is descriptive enough to
> demonstrate that we
> > meet GIN goals.
> >
> > Gregor
> >
> > Gregor von Laszewski wrote:
> >
> >> Bill asked me to look into some programs that we developed
> in the past
> >> as part to verify the functionality between gridFTP
> servers between
> >> different sites.
> >>
> >> One of the things we have is
> >>
> >> a) a program that can test elementary gridftp
> functionality between a
> >> client and a service. This helps the user to test if he
> has access to
> >> the services from his preferred client. This was one of the most
> >> requested features by users before each SC in the past.
> >>
> >> b) this program can be run on a server and while using
> delegation you
> >> can create a matrix between sites that report on the ability to
> >> conduct a compatibility tests.
> >>
> >> We ran a form of this program before SC04, SC05 to test
> >> interoperability between the various versions of gridftp
> servers as
> >> well as various job execution clients.
> >>
> >> Some of the test have a history, this way you could see if
> this has
> >> worked in the past. It is easy to publish the result to
> public_html.
> >> As the output is all in HTML it can be easily browsed with
> a browser.
> >>
> >> I suggest that we collect some relevant machine names and
> port numbers
> >> in order to see if we can run it and to apply for accounts
> and place
> >> certs at the relevant locations.
> >>
> >> We are in the process of making screenshots and putting up a
> >> documentation that is easier to find than the one that we have at
> >>
> http://www.cogkit.org/viewcvs/viewcvs.cgi/src/cog/modules/test
> ing/karajan/README.txt?rev=1.1&content-type=text/vnd.viewcvs-markup
> >>
> >>
> >>
> >> Gregor
> >>
> >> PS: A subset of information from our documentation may
> show you what
> >> you can customize (please be aware that some text may :
> >>
> >>
> >> Constant example value description
> >>
> >> COLOR:FAILED "#ff4000" The
> background color of
> >> the table
> >> cell of a failed test
> >> COLOR:PASSED
> >> "#ffffff" The background color of the table
> >> cell of a passed test
> >> COLOR:TIMEOUT
> >> "#ffba00" The background color of the table
> >> cell of a
> timed-out test
> >>
> >> OUTPUT_DIR "output" The local
> directory in
> >> which the
> >> output files
> are created
> >>
> >> PUBLISH false Whether to
> publish the
> >> output files
> >> on a remote
> server after
> >> the tests
> >> are done.
> Copying happens
> >> using
> >> GridFTP.
> >>
> >> PUBLISH_HOST "wiggum.mcs.anl.gov" If publishing
> is enabled,
> >> the host
> >> to which
> files are copied
> >> PUBLISH_DIR
> >> "public_html/testing2" If publishing is enabled, the remote
> >> directory in
> which files
> >> are copied
> >>
> >> TEST_TIMEOUT 60*1000 The number of
> miliseconds
> >> after which
> >> a test is
> considered to
> >> have timed-out
> >> TEST_FILE "testfile" A file
> used in some
> >> of the file operation
> >> tests
> >> TEST_FILE_DIR
> >> user.home The directory containing the test file
> >> MAX_HISTORY_SIZE 365 The
> maximum number of
> >> samples stored in
> >> the history. Older
> >> samples may be
> >> discarded in order to
> >> enforce this
> >> setting.
> >> COG_DIR "{user.home}/cog-4_1_4" The path to a CoG
> >> binary distribution.
> >> This is needed for
> >> indirect tests
> >> (which use
> command line
> >> tools rather
> >> than library calls).
> >>
> >> 3. The hosts file
> >>
> >> The hosts.k file contains a list of machines and corresponding
> >> services on which the tests are run. The format of the
> file is a flat
> >> sequence of task:host elements (our manual that we write will have
> >> more detail on this). It is possible to specify multiple
> logical hosts
> >> for the same physical host for the purpose of separating different
> >> versions of the same services (like for example transferring files
> >> between the GT 4.0.1 and GT 4.0.2 GridFTP servers on the
> same machine).
> >>
> >> 4. Included tests and suites
> >>
> >> a) Execution (job submission)
> >>
> >> i) Direct - uses the task:execute call to submit jobs
> >> ii) Indirect - uses the cog/bin/cog-job-submit
> tool for the
> >> submission (effectively this would allow testing for GT2, GT3, GT4
> >> submission).
> >> b) File operations
> >> i) Direct
> >> A) Put - copies a file from the local host to the remote host
> >> B) Get - copies the above file back from the remote
> host to the
> >> local host
> >> C) List - lists files in a directory on the remote host
> >> D) Rename - renames a file on a remote host
> >> E) Remove - removes a file on a remote host
> >> F) Exists - tests the existence of a file
> >> G) Make Dir - creates a directory
> >> H) Is Dir - tests the isDirectory() implementation
> >> I) Remove Dir - removes a directory
> >> J) Bug - Runs a sequence of operations that used to cause a
> >> problem with
> >> certain combinations of client/GridFTP server versions (a
> >> list on a
> >> nonexistant directory/file followed by a simple
> operation -
> >> exists())
> >> c) Transfer
> >> Runs partial third-party transfers from /dev/urandom
> to /dev/null
> >> of 1MB of data between all pairs of hosts and displays the
> total time
> >> in a table. This test will only work properly with GridFTP
> servers.
> >> The transfer time will be noted within a table between all
> involved
> >> hosts.
> >>
> >
>
>
More information about the gin-data
mailing list