[glue-wg] ATLAS, GLUE and JSON

[Alan Sill]

Hi JP,

I agree completely and may have missed stating my point clearly enough to say that.

GLUE in general (correct me if I am wrong) helps describe data center capabilities and characteristics to each other -- i.e., is designed to facilitate describing and providing information appropriate to entire data centers or subsets of them at a large scale.

Redfish applies to individual data center components, and is intended to replace IPMI for this purpose. IN addition, Swordfish aims to use the same schema structure and design to do the same sort of task in the storage domain.

Many of the things that GLUE needs to gather up and aggregate, and present at a higher level, are explicitly provided for in Redfish and Swordfish at the individual component level for servers, storage units, etc. In addition, there are provisions in the Redfish and Swordfish designs to accommodate the concepts of collections as groupings of related capabilities. This is all designed to be schema-based (using OData and some related concepts) and programmable with modern RESTful tools using JSON or XML for data exchange.

It seems to me that the capability to gather, parse, group, aggregate and present Swordfish and Redfish data using the concepts of GLUE would be a useful thing to have, especially because modern data center equipment is already being delivered with Redfish enabled and soon we will start to see Swordfish emerge for storage. Thus a layer to connect GLUE to these underlying sources of data, filling in from other sources where necessary, would seem to me to be a good idea.

I am also interested in other opinions, so second your request.

Hope this helps,
Alan

> On Oct 13, 2016, at 1:26 PM, Navarro, JP <navarro at mcs.anl.gov> wrote:
> 
> Alan,
> 
> Having looked briefly at the DMTF Redfish architecture I notes their scope is to “meet(s) the expectations of Cloud and Web-based IT professionals for scalable platform management”. They describe their data model as "The Redfish model is built for managing systems.”. Looking at their data model and API they are focused on describing real and virtual hardware elements, and performing management operations on those elements.
> 
> By contrast, GLUE2 is focused on user facing distributed resource metadata and interfaces to access those resources are out of scope (GLUE2 describes the interfaces, but they are implemented by other tools and specs).
> 
> In short:
> - GLUE2 focuses on user facing information while Redfish focuses on infrastructure management information
> - GLUE2 focuses exclusively on discovery, while Redfish focuses on both discovery and performing actions on infrastructure
> 
> While there is some modest overlap (both schemas have a concept of infrastructure location and administrative domains), they are trying to address different problems, IMO.
> 
> Does anyone have other opinions on this?
> 
> Thanks,
> 
> JP
> 
>> On Oct 12, 2016, at 6:02 PM, Alan Sill <kilohoku150 at gmail.com <mailto:kilohoku150 at gmail.com>> wrote:
>> 
>> Hi Paul,
>> 
>> I had approached JP a few months ago with the idea of creating a schema in the Distributed Management Task Force (DMTF)'s new Redfish architecture specifically to accommodate GLUE data. This would have the following features and advantages:
>> 
>> 1) Integrates with a standard already being built into hardware in data center server and storage by several vendors,
>> 2) Would map well to the basic ideas and approach of GLUE
>> 3) Provides existing implementation models using ODATA for organization and both JSON and XML representations for data exchange 
>> 4) DMTF would be willing to give us our own schema directory for Redfish for use with GLUE information.
>> 
>> Redfish is being built into hardware from several major vendors now as a replacement for IPMI (for servers and similar equipment) with some features specifically interesting to GLUE for things like storage through the partnership between the Storage Networking Industry Association (SNIA) and DMTF. SNIA's version of Redfish is called Swordfish, and includes a lot of information that would be very good to use as input for gathering up information we normally care about representing at the data-center level with GLUE.
>> 
>> I've thought of a name for this possible use of GLUE in the context of Redfish, which is of course "GLUEfish". The idea is to represent data center level information such as is typically gathered and exchanged in GLUE using the Redfish schema, thereby leveraging existing tools that could gather the information directly from current- and future-generation hardware.
>> 
>> If you are interested in discussing this further, please let me know. Otherwise, here are some useful links:
>> 
>> Redfish Developer Site: http://redfish.dmtf.org <http://redfish.dmtf.org/>
>> 
>> Specifications:
>> 
>>   Redfish standard:  http://www.dmtf.org/redfish <http://www.dmtf.org/redfish>
>>   OData: http://www.odata.org/documentation <http://www.odata.org/documentation>
>>   JSON-schema: http://www.json-schema.org <http://www.json-schema.org/>
>> 
>> I'd love to talk about this more with the GLUE group in general.
>> 
>> Alan
>> 
>> 
>> 
>>> On Oct 12, 2016, at 5:25 PM, Paul Millar <paul.millar at desy.de <mailto:paul.millar at desy.de>> wrote:
>>> 
>>> Hi everyone,
>>> 
>>> <background>
>>> 
>>> A little while ago, the WLCG VO ATLAS started investigating how to do storage accounting in a platform neutral fashion (i.e., not depending on SRM).
>>> 
>>> Their idea was that a site would "somehow" ensure that their storage system always has a file containing the necessary information.  The file would be stored in a standard location.  ATLAS could then read this file using whichever protocol is easiest.
>>> 
>>> The plan was for this file would be encoded as JSON.  They also put together a rough description of how this file should look like.
>>> 
>>> I suggested that they investigate GLUE 2 and the GLUE-JSON binding, rather than coming up with their own proprietary solution.
>>> 
>>> </background>
>>> 
>>> So, ATLAS are asking for some information:
>>> 
>>>  o	people with whom they can discuss how to use
>>> 	GLUE2 and GLUE-JSON.
>>> 
>>>  o	tools that parse GLUE-JSON.  I guess this would be mostly
>>> 	from XSEDE guys.
>>> 
>>> So, does anyone want to help them evaluate GLUE 2?
>>> 
>>> Cheers,
>>> 
>>> Paul.
>>> _______________________________________________
>>> glue-wg mailing list
>>> glue-wg at ogf.org <mailto:glue-wg at ogf.org>
>>> https://www.ogf.org/mailman/listinfo/glue-wg <https://www.ogf.org/mailman/listinfo/glue-wg>
>> 
>> _______________________________________________
>> glue-wg mailing list
>> glue-wg at ogf.org <mailto:glue-wg at ogf.org>
>> https://www.ogf.org/mailman/listinfo/glue-wg
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/glue-wg/attachments/20161013/3066eaf8/attachment-0001.html>