[occi-wg] New revision of the spec (Console definitions)

[Gary Mazz]

Inline.....

Thijs Metsch wrote:
> Again - see inline :-)
>
> On Thu, 2010-08-05 at 09:26 +0000, Csom Gyula wrote:
>   
>> Hi, 
>> thanks for your feedback! Comments for the open items only:)
>>
>>
>> [5] Id assignment: URI or id? Any:) I've got used to ids but URI might better fit the ws client 
>> space. Also the id is usually reflected within the URI.
>>
>> The general idea is that the identifier (either the internal id or the URI) is determined before 
>> the creation request is issued. So the sequence would be the following:
>>
>> 1. The client asks a new id/URI (for a resource type) and the server gives back the next free 
>>     one.
>> 2. The client assembles the id/URI into the request and issues it (if using URIs the assembling
>>     migth be nothing but using the URI to post the request to). The server checks whether 
>>     the id is free-and-unique or not, if yes it creates the new resource.
>>
>> In this scenario the worst thing that could happen is that an id/URI is requested but never
>> used - which is not an issue compared to the other case when the client creates the 
>> resource but cannot retrieve the id due to some problem (connection lost or kinda). 
>>
>> Similar could be applied when using (unique) names. Though in this case it is the client 
>> responsibility to determine the unique name, and the server has only 2 roles: validate the 
>> unicity and respond to resource reuqests given by name.
>>
>> A real life example: OpenNebula when deploying machines to the virtualization layer generates 
>> a so called deployment id (one-[VMID]) then assigns it to the Libvirt domain as the domain's 
>> name. The domain id itself is generated by Libvirt internally. We'll apply the same technique
>> to keep in synch our internal registry with the ONE's db.
>>     
>
> Well right now doing a POST on / will create /<some id> and a POST
> on /foo/ will create a /foo/<some id> if 'foo' exists. If you do a PUT
> on an non existing resource it will try to create it - otherwise update
> it. So a PUT on /foo/bar will create the resource /foo/bar...
>
> Maybe this helps?!
>
>   
>> [7] Graphics: I agree with Gary, console is a must:) His spec is definitve... here let me add just
>> some notes:
>> * We are using KVM/QEMU as the hypervisor so I can confirm that KVM/QEMU provides graphical 
>>   terminal support, namely VNC:)
>> * Security is an issue at least in two ways. First the terminal gives access to the running 
>>   compute resource hence it needs password protection or such. Second the terminal
>>   access operates on the host level not on the vm level. That is the terminal address would
>>   be the IP of the physical host the vm is running on. Generally it is not a secure thing
>>   to give direct access to interna hostl infrastructure. For instance in our solution (currently in 
>>   development) we use a VNC proxy that hides the internal locations.
>> * Isn't occi.console.status the same as (or projection of) the socci.compute.status? It could
>>    be a useful information though. I guess it should be dinamically queried from the corresponding
>>    compute resource.
>> * Either the console should be the part of the compute resource or it should link to the compute 
>>   resource it belongs to, that is:
>>   * either console should be moved under the compute namespace (ie. occi.compute.console.xxx)
>>   * or there must be a bidirectional link between the two (occi.console.compute_link -> compute
>>     and occi.compute.console_link -> console or such)
>>     
>
> Some good notes @Gary can you respond?
> How do you feel about the subject if those attributes are mandatory or
> optional?
>   
I think part of the issue is our deployment use case model that is 
somewhat incomplete in terms of a console.

*Computer **and Blade **Platform Use Cases*

*Generalized **VM Execution **Use Cases*
USE CASE] A VM executes on single computer platform with a single or 
multiple cpu cores.
USE CASE] A VM executes on blade platform with a single or multiple cpu 
cores.

*Generalized IO Console Configuration **Use Cases*
USE CASE] A single computer platform supports  one or more serial ports 
for console I/O
USE CASE] A single compute platform supports one or more video outputs 
and one keyboard input
USE CASE] A blade platform supports one or more serial ports for console I/O
USE CASE] A blade platform supports one more video output and one 
keyboard input

*Generalized IO Console Client to IO Console Server Connectivity Use Cases*
USE CASE] A blade platform management controller presents blade's video 
outputs and keyboard inputs via a network protocol including VNC, RDP 
and XWindows
USE CASE] A blade platform management controller presents blade's serial 
ports via a network protocol including TELNET and SSH
USE CASE] A KVM device controller presents single computer's video 
outputs and keyboard inputs via a network protocol including VNC, RDP 
and XWindows
USE CASE] A Terminal Server presents a single computer's serial ports 
via a network protocol including TELNET and SSH
*
Hypervisor Software Use Cases*
USE CASE] A Hypervisor Software executing on a single blade presents 
VM's video outputs and keyboard inputs via a network protocol including 
VNC, RDP and XWindows
USE CASE] A Hypervisor Software executing on a single blade presents a 
VM's serial ports via a network protocol including TELNET and SSH
USE CASE] A Hypervisor Software executing on a single blade element 
presents VM's video outputs and keyboard inputs via a network protocol 
including VNC, RDP and XWindows
USE CASE] A Hypervisor Software executing on a single blade element 
presents a VM's serial ports via a network protocol including TELNET and SSH

*IO Console Sharing Use Cases*
USE CASE] More than one user may access a blade's platform management 
controller's presented blade's video outputs and keyboard inputs via a 
network (console instance sharing)
USE CASE] More than one user may access a blade's platform management 
controller's presented blade's serial ports via a network (console 
instance sharing)
USE CASE] More than one user may access a Terminal Server's presented 
single compute serial port via a network (console instance sharing)
USE CASE] More than one user may access a KVM device's presented single 
compute serial port via a network (console instance sharing)

*IO Console Configuration Use Cases*
USE CASE] Network Address of Terminal Server's presented single 
computer's serial port can be set by VM configuration
USE CASE] Network port number of Terminal Server's presented single 
computer's serial port can be set by VM configuration
USE CASE] Network Address of Terminal Server's presented single 
computer's serial port can be set by Cloud Provider Administration
USE CASE] Network port number of Terminal Server's presented single 
computer's serial port can be set by Cloud Provider Administration
USE CASE] Network Address of Terminal Server's presented single 
computer's serial port can be read by Cloud User(s) though VM configuration
USE CASE] Network port number of Terminal Server's presented single 
computer's serial port can be read by Cloud Provider Administration 
though VM configuration
USE CASE] Terminal Server's Network Address presenting a single 
computer's serial port can be is common across all Terminal Server's 
serial ports
USE CASE] Network Address of KVM's presented single computer's graphical 
console can be set by VM configuration
USE CASE] Network port number of KVM's presented single computer's 
graphical console can be set by VM configuration
USE CASE] Network Address of KVM's presented single computer's graphical 
console can be set by Private Cloud Administration
USE CASE] Network port number of KVM's presented single computer's 
graphical console can be set by Cloud Provider Administration
USE CASE] Network Address of KVM's presented single computer's graphical 
console can be read by Cloud User(s) though VM configuration
USE CASE] Network port number of KVM's presented single computer's 
graphical console can be read by Cloud Provider Administration though VM 
configuration
USE CASE] KVM's Network Address presenting a single computer's graphical 
console can be is common across all Terminal Server's serial ports
USE CASE] Network Address of a blade's platform management controller's 
presented blade's serial port can be set by VM configuration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's serial port can be set by VM configuration
USE CASE] Network Address of a blade's platform management controller's 
presented blade's serial port can be set by Cloud Provider Administration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's serial port can be set by Cloud Provider 
Administration
USE CASE] Network Address of a blade's platform management controller's 
presented blade's serial port can be read by Cloud User(s) though VM 
configuration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's serial port can be read by Cloud Provider 
Administration though VM configuration
USE CASE] Terminal Server's Network Address presenting a blade's serial 
port can be is common across all a blade's platform management 
controller's serial ports presented
USE CASE] Network Address of a blade's platform management controller's 
presented blade's graphical console can be set by VM configuration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's graphical console can be set by VM 
configuration
USE CASE] Network Address of a blade's platform management controller's 
presented blade's graphical console can be set by Cloud Provider 
Administration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's graphical console can be set by Cloud 
Provider Administration
USE CASE] Network Address of a blade's platform management controller's 
presented blade's graphical console can be read by Cloud User(s) though 
VM configuration
USE CASE] Network port number of a blade's platform management 
controller's presented blade's graphical console can be read by Cloud 
Provider Administration though VM configuration
USE CASE] Terminal Server's Network Address presenting a blade's 
graphical console can be is common across all a blade's platform 
management controller's serial ports presented


*IO Console Authentication Use Cases*
USE CASE] A Terminal Server's presented single compute serial port has 
only one credential for all Private Cloud Administrators (user/customer) 
accessing the port
USE CASE] A Terminal Server's presented single compute serial port has 
only one credential for each Private Cloud Administrator (user/customer) 
accessing the port
USE CASE] A Terminal Server has only one credential for all Private 
Cloud Administrators (user/customer) accessing all presented single 
compute serial ports
USE CASE] Terminal Server's presented single compute serial port's 
credentials can be set with the VM configuration by the Cloud Provider 
Administrator
USE CASE] Terminal Server's presented single compute serial port's 
credentials can be set with the VM configuration by the Private Cloud 
Administrator (user/customer)
USE CASE] Terminal Server's presented single compute serial port's 
credentials can be set with a external management application by the 
Cloud Administrator
USE CASE] Terminal Server's credentials can be set with the VM 
configuration by the Cloud Provider Administrator
USE CASE] Terminal Server's credentials can be set with the VM 
configuration by the Private Cloud Administrator (user/customer)
USE CASE] Terminal Server's credentials can be set with a external 
management application by the Cloud Administrator
USE CASE] A KVM's presented single computer's graphical console has only 
one credential for all Private Cloud Administrators (user/customer) 
accessing the port
USE CASE] A KVM's presented single computer's graphical console has only 
one credential for each Private Cloud Administrator (user/customer) 
accessing the port
USE CASE] A KVM has only one credential for all users accessing all 
presented computer's graphical consoles
USE CASE] KVM's presented single computer's graphical console's 
credentials can be set with the VM configuration by the Cloud Administrator
USE CASE] KVM's presented single computer's graphical console's 
credentials can be set with the VM configuration by the Private Cloud 
Administrators (user/customer)
USE CASE] KVM's presented single computer's graphical console's 
credentials can be set with a external management application by the 
Cloud Administrator
USE CASE] KVM's credentials can be set with the VM configuration by the 
Cloud Administrator
USE CASE] KVM's credentials can be set with the VM configuration by the 
Private Cloud Administrator (user/customer)
USE CASE] KVM's credentials can be set with a external management 
application by the Cloud Administrator

USE CASE] A blade's platform management controller's presented blade 
serial ports has only one credential for all Private Cloud 
Administrators (user/customer) accessing the port
USE CASE] A blade's platform management controller's presented blade 
serial ports has only one credential for each Private Cloud 
Administrator (user/customer) accessing the port
USE CASE] A blade's platform management controller has only one 
credential for all Private Cloud Administrators (user/customer) 
accessing all presented blade serial ports
USE CASE] A blade's platform management controller's presented blade 
serial port's credentials can be set with the VM configuration by the 
Cloud Provider Administrator
USE CASE] A blade's platform management controller's presented blades 
serial port's credentials can be set with the VM configuration by the 
Private Cloud Administrator (user/customer)
USE CASE] A blade's platform management controller's presented blade 
serial port's credentials can be set with a external management 
application by the Cloud Provider Administrator
USE CASE] A blade's platform management controller's credentials can be 
set with the VM configuration by the Cloud Provider Administrator
USE CASE] A blade's platform management controller's credentials can be 
set with the VM configuration by the Private Cloud Administrator 
(user/customer)
USE CASE] A blade's platform management controller's credentials can be 
set with a external management application by the Cloud Provider 
Administrator
USE CASE] A blade's platform management controller's presented blade's 
graphical console has only one credential for all Private Cloud 
Administrators (user/customer) accessing the port
USE CASE] A blade's platform management controller's presented blade's 
graphical console has only one credential for each Private Cloud 
Administrator (user/customer) accessing the port
USE CASE] A blade's platform management controller has only one 
credential for all Private Cloud Administrators (user/customer) 
accessing all presented blade's graphical consoles
USE CASE] A blade's platform management controller's presented blade's 
graphical console's credentials can be set with the VM configuration by 
the Cloud Provider Administrator
USE CASE] A blade's platform management controller's presented blade's 
graphical console's credentials can be set with the VM configuration by 
the Private Cloud Administrator (user/customer)
USE CASE] A blade's platform management controller's presented blade's 
graphical console's credentials can be set with a external management 
application by the Cloud Provider Administrator
USE CASE] A blade's platform management controller's credentials can be 
set with the VM configuration by the Cloud Provider Administrator
USE CASE] A blade's platform management controller's credentials can be 
set with the VM configuration by the Private Cloud Administrator 
(user/customer)
USE CASE] A blade's platform management controller's credentials can be 
set with a external management application by the Cloud Provider 
Administrator


*Limits
*Desktop Virtualization created by the executing operating system in a 
VM is not in the scope of this use case model.


*Comments:*

I'm looking into a more robust way of defining the security identifier 
and credentials ,

I'm also looking at an interoperable way to incorporate the 
configuration issues into  OCCI. I put a proposal together  if we all 
agree  on the use cases.

>   
>> [8] Compute transitional states: Regarding long running jobs... I've got the point:) Just one
>> more question. Can someone query active jobs related to a particalur resource? Ie. is job linkage 
>> supported from the resources? Again its about robustness... w/o the query feature or kinda if the 
>> connection is lost during the transaction the client might miss a running job...
>>     
>
> That is not described yet...
>
>   
>> [9] Compute end state: yep, I can paint images:) Which UML tool do you use/prefer?:)
>>     
>
> I dunno - @Andy can you tell?
>
>
>   
>> [10, 11] Network related stuff: yep I can pick it up:)
>>     
>
> Perfect Thanks!
>
>   
>> [13] Golden images: Ok I see the point:) just let me add one note. Support through linkage means 
>> that you support 'clone before' operations only. In theory there could be (at least) 2 scenarios for 
>> golden image usage:
>>
>> 1. Clone before: Clone the golden image, retrieve its id then use it in the (new or updated)
>>    compute resource. That is cloning and compute creation/update happens in 2 transactions.
>> 2. Clone just-in-time: Give the golden image link, mark it with "clone=yes" and issue your 
>>     compute create or update request. That is cloning and compute creation/update happens 
>>     in 1 transaction.
>>
>> Strictly speaking the 2nd feature is not mandatory though it might increase usability (ie. 
>> "one click deployment"). 
>>     
>
> Anybody else got some thoughts on this?
>
> Cheers,
>
> -Thijs
>
>   
>> Cheers,
>> Gyula
>> ________________________________________
>> Feladó: Thijs Metsch [tmetsch at platform.com]
>> Küldve: 2010. augusztus 5. 10:16
>> Címzett: Csom Gyula
>> Másolatot kap: occi-wg at ogf.org
>> Tárgy: RE: [occi-wg] New revision of the spec
>>
>> Hi,
>>
>> Thank you for your replies - I'll try to comment inline :-) If you see
>> any changes needed - do not hesitate to edit them in the Wiki as well!
>>
>>     
>>> Core:
>>>
>>> [1] Action - Http mapping: Which Http method does actions map to? POST?
>>>       
>> yes
>>
>>     
>>> [2] Return values: Return values could be specified both for CRUD and resource-specific
>>> actions. For instance when creating new resources the generated id should be returned as
>>> a minimum (see also id assignment below)
>>>       
>> Currently it should either return a 200 OK with Location or a 202 with
>> Location to another intermediate resource for a long running job.
>>
>>     
>>> [3] Creating subresources: it could be clarified whether subresources can be created
>>> along with the parent resource or not? What is right creation seuqence:
>>>
>>> 1. Create a compute w/o network and storage then add network and storage
>>> 2. Create a compute with network and storage in 1 transaction then maybe adding more
>>>     resources later...
>>>
>>> Which scenario is valid? Which should be supported by an OCCI-complient solution?
>>>       
>> Probably something we need to be more clear on in the Infrastructure
>> part.
>>
>>     
>>> [4] Adding subresources: Do you plan to support adding subresources? For instance
>>> attaching an existing storage to a compute? If yes where does it fit within the spec? Is it a
>>> resource UPDATE procedure by providing a link to an existing resource?
>>>       
>> It would be a PUT with an Link on an existing resource...needs to be
>> more clear I guess :-)
>>
>>     
>>> [5] Id assignment: Generally speaking server assigned ids doesn't provide a robust solution.
>>> What if the client lost the connection? It can lead to zombie resources... Think of system
>>> integration and cloud management automation... Either unique names or sequences (like
>>> in RDBMS) could be supported.
>>>       
>> What is an id in this question? An internal ID used by the Backend
>> system or a URI?
>>
>>     
>>> [6] 'Attributes in creation': the specification deals with attribute mutability however it doesn't
>>> clarify attribute 'creatibility'. That is what attributes can be filled by the client and what can be
>>> filled by the server when creating resources? What can be null? What is always assigned
>>> either dynamically or manually? Generally it would be nice to document how attributes behaves
>>> when creating a new resource: can be null | must be manually filled | dynamically assigned |
>>> manual or dynamic etc.
>>>       
>> Might be an idea - for now mutability means: can be set by the user -
>> all described attributes are mandatory (if not stated otherwise)
>>
>>     
>>> Infrastructure:
>>>
>>> [7] Graphics: It is a basic service within clouds to provide OS-independent monitor that is terminal
>>> access to vms independent from the OS running in the virtual machine. VNC is a typical solution.
>>> It would be nice to support this within compute (eg. graphics type, ip, port, password, and keymap).
>>>       
>> What do you think of the solution Gary proposed on the list?
>>
>>     
>>> [8] Compute transitional states: It might be useful to introduce substates for transitions. For instance
>>> virtual machine deployment typically takes some time while the resource is either pending (not yet
>>> scheduled) or being deployed. Typically a user or agent would be interested in such transitions...
>>>       
>> See long running jobs...
>>
>>     
>>> [9] Compute end state: Also an end state (like deleted) should be introduced that is for systems that
>>> support logical deletes (destroys the vm but maintains the record for audit purposes or kinda).
>>>       
>> Already an todo in the Wiki - can you paint new images? :-)
>>
>>     
>>> [10] Virtual network: vnets are basic services within cloud/virtualization solutions. That is the
>>> admin or user create a virtual network and the user assigns the compute's NIC to the selected
>>> virtual network.
>>>
>>> This is related to the following question: "Semantics of provisioning a compute in relation to
>>> networking is undecided". In my limited practise I've seen 2 types of provisioning:
>>>   * when the client request an explicit address
>>>   * when a client request a virtual network and it is the server who assigns the address
>>>
>>> That is in the first case IP address is filled by the client, however in the second case it is filled
>>> by the server (see also [6] above).
>>>       
>> This is a area we need to work on - can you pick this one up?
>>
>>     
>>> [11] Private vs. public IP: Also there's a difference between public and private IPs. Public IP
>>> allocation is definitely a cloud role, related to the administration domain too. However managing
>>> private IPs at the cloud level might not make sense: since vlans are separated anytime any
>>> private IP could be configured at the VM level, it is the user responsibility. So for private IPs
>>> it is usually the MAC address that must be managed by the cloud (that is unique MAC address
>>> is a generally a must).
>>>       
>> see above :-)
>>
>>     
>>> [12] Cooked storage: Do you plan to support cooked storage, that is creating a storage with
>>> a file system on it? If yes a disk type attribute might be introduced...
>>>       
>> Probably more in scope of CDMI if I understand you correctly...but not
>> sure...
>>
>>     
>>> [13] Golden images: It is a basic service within clouds to create a storage by cloning
>>> an existing image. If this is to be supported then a source attribute (link) might be introduced to
>>> the storage resource.
>>>       
>> This is generally support through links - the spec might need to be more
>> clear here...
>>
>>
>> Thanks again,
>>
>> -Thijs
>>
>>     
>>> Cheers,
>>> Gyula
>>>
>>> ________________________________________
>>> Feladó: occi-wg-bounces at ogf.org [occi-wg-bounces at ogf.org] ; meghatalmaz&#243;: Thijs Metsch [tmetsch at platform.com]
>>> Küldve: 2010. augusztus 3. 9:26
>>> Címzett: occi-wg at ogf.org
>>> Tárgy: [occi-wg] New revision of the spec
>>>
>>> Hi,
>>>
>>> I would like to point you to the newest version of the spec:
>>>
>>> http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/CoreAndModels
>>>
>>> http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/HTTPHeaderRendering
>>>
>>> http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/Infrastructure
>>>
>>> Some changes have happened including:
>>>
>>> * renaming Kind to Resource
>>> * Cleanup of namespace for schemes in the category and links
>>> * added some diagrams etc.
>>>
>>> This version includes a lot of rewording and should be more clear on
>>> many issues!
>>>
>>> Please review it and give us a first shot of feedback!
>>>
>>> Cheers,
>>>
>>> -Thijs
>>>
>>> --
>>> Thijs Metsch
>>> Senior Software Engineer Grid and Cloud Technology
>>> Platform Computing GmbH
>>> Europaring 60
>>> D-40878 Ratingen
>>> http://www.platform.com
>>>
>>> http://www.nohuddleoffense.de/ - http://www.twitter.com/befreax
>>> _______________________________________________
>>> occi-wg mailing list
>>> occi-wg at ogf.org
>>> http://www.ogf.org/mailman/listinfo/occi-wg
>>>       
>> --
>> Thijs Metsch
>> Senior Software Engineer Grid and Cloud Technology
>> Platform Computing GmbH
>> Europaring 60
>> D-40878 Ratingen
>> http://www.platform.com
>>
>> http://www.nohuddleoffense.de/ - http://www.twitter.com/befreax
>>
>>     
>
>