[gin-jobs] Standarized Execution Environment

[JP Navarro]

Thomas and Laurence,

My previous e-mail listed the environment variables the TeraGrid defines
in our default interactive and Grid jobs environment. I didn't explain
the method and tool we're using to manage environments.

The TeraGrid has standardized on an environment management tool called
SoftEnv, developed in the Mathematics & Computer Science division of
Argonne National Lab 15 years ago. It was developed to give users a
consistent way to manage their environment across over a dozen different
Unix variants.

Functionally, SoftEnv is very similar to modules used by DEISA. As a
matter of fact some TeraGrid sites use SoftEnv to manager TeraGrid user
environments, and modules for their local non-TeraGrid users.

Some links explaining the TeraGrid's implementation:
   http://software.teragrid.org/docs/ctss3/softenv/README.overview
   http://software.teragrid.org/docs/ctss3/softenv/README.teragrid
This document has a very detailed rundown of our SoftEnv keys standards:
   http://software.teragrid.org/docs/ctss3/softenv/README.ctssV3

The TeraGrid has been working to integrate SoftEnv with Grid tools so
that Grid users can manage their environments without having to login
to discover or alter (thru shell initialization) runtime environments.
Our two main integration activities have been to:

Work with the Globus team to extend the GRAM interfaces (pre-WS and WS)
so that Grid users can specify their runtime environment using abstract
SoftEnv keys. I think GRAM should support both SoftEnv and Modules.

Work with the Condor team so that SoftEnv keys are published in class- 
add
form, and so that jobs that specify SoftEnv prereqs have their runtime
environments setup automatically to include their prereqs.

The beauty of SoftEnv and Modules as environment management tools is  
that
they provide a standard interface for manipulating environments across
shell flavors, and an abstract namespace to represent target environment
configurations. Grids are all about standard abstract interfaces.

If Grid job interfaces could support environment management tools, we
would be able to define a standard GIN-Jobs environment that could be
requested by GIN Jobs, without GIN participants having to alter or
give up their own local or regional standard Grid environments.

Regards,

JP

On Mar 29, 2007, at 3:42 AM, Thomas Soddemann wrote:

> Hi Laurence,
>
> I have started to modify the GIN-Jobs Wiki page (https:// 
> forge.ogf.org/sf/wiki/do/viewPage/projects.gin/wiki/GINJobs) to  
> reflect your contribution.
> Here is a set of environment variables, we agreed to use in DEISA:
>
>    * Home directory ($HOME)
>    * Other home directory, accessible on each homogeneous site
>      ($DEISA_HOME)
>    * Working directory, accessible on each homogeneous site  
> ($DEISA_DATA)
>    * Temporary directory during the life of a job ($DEISA_SCRATCH)
>
> The rest is handled via the modules environment which is in the  
> user's default path.
> E.g. a user would load the initial standard DEISA environment as  
> described in the DEISA primer by issuing
> #1> module load deisa
>
> then he would proceed by loading appropriate subsequent modules  
> e.g. for CPMD
>
> #2> module load cpmd
>
> and then start his job.
>
> Cheers,
> Thomas
>
>
> <soddemann.vcf>
> --
>   gin-jobs mailing list
>   gin-jobs at ogf.org
>   http://www.ogf.org/mailman/listinfo/gin-jobs