[occi-wg] VM memory ranges and weight (was "XHTML5 Rendering Proposal...")

Wed Nov 11 12:03:22 CST 2009

Hey Andrew
The assumption in your mail that I see (correct me if I’m barking up the wrong tree) is the hypervisor allows the OS grab all memory and that it is the host OS that has control over what the hypervisor does. However, hypervisors aggressively optimise and reuse inactive memory in a means to do more with less. My view on what you are discussing are more to do with internal optimisations either within the service provider’s domain or within the internal technology domain of hypervisors. As OCCI is a “boundary” specification and protocol it shouldn’t be too concerned with such things. Also when talking of operating systems, with respect to OCCI we fall into a grey area between IaaS and PaaS. What might be worth consideration is the characteristic of “burstability” or in more general terms, “elasticity” in various attributes and that these considerations could pave the way to countering scenarios that you have outlined.

Andy

From: occi-wg-bounces at ogf.org [mailto:occi-wg-bounces at ogf.org] On Behalf Of Andrew de Andrade
Sent: 11 November 2009 17:08
To: occi-wg
Subject: [occi-wg] VM memory ranges and weight (was "XHTML5 Rendering Proposal...")

(new topic: VM memory ranges and weight)

As I am not super technical, the following may be a bit confusing to understand. I'm going to try my best to explain. Keep in mind that this is not a critique of the XML currently being discussed, but in the way we are defining memory allocation for a VM in general. Here goes:

I noticed that the XML is defining memory in the following line:
<li>Memory: <span property="occi:compute.memory">2048</span>

I have one concern about defining a fix memory amount in our VM specification. The way I see it, allocating a fixed memory amount to a VM caters to a legacy issue associated with Operating Systems. The typical modern operating system for x86 chipsets (Windows, Linux variants, Unix variants etc.) all having one major memory management problem --  they're greedy. When the OS starts up, it first checks the machine it is running on, in the case of a IaaS cloud this is a virtual machine. Upon discovering how much memory the hypervisor has allocated to it, it automatically grabs all this memory whether it needs it or not. It assumes full ownership of the memory.

Long-term this is very wasteful of memory, because there is no way (other than the black magic VMware is doing) to return this memory to the hypervisor when the applications running on top of the OS are not using it.

i.e. if we have 48GB of RAM on the host and we start up 12 identical VMs with 4GB allocated to each via the OCCI specification, we then can end up with a lot of waste. If one of these VMs needs more than 4GB of RAM, it cannot take this RAM from another VM on the host that is not really using their memory.

We should create a specification that allows this legacy memory allocation issue to degrade gracefully:
-- The specification permits a memory management model for future operating systems, whereby the machine only takes the memory it needs and can get more memory when necessary and also return memory that it is not using. This would most likely best be defined not as one memory value, e.g. 2048MB like in the XML above, but instead would be expressed as a range and a weight.
-- The typical modern OS today should still be able to get memory via this old legacy model (it gobbles up all the memory allocated to it). For these systems we can simply set the minimum and maximum to be the same value.

-- The range would specify the minimum and maximum amount of RAM available to the Operating System. e.g. the machine is guaranteed a minimum of 2048 MB of RAM and can use up to  8192 MB of RAM.
-- In non-multi-tenant environments, i.e. private clouds, the weight would be use to prioritize the machine relative to others as memory is freed up. This would only be necessary when all the host's memory is used and the host owner wants to prioritize some VMs over others.
-- In a multi-tenant environment, i.e. a public cloud, all the machines would ideally have the same weight (equal priority) or in some cases the cloud may offer service levels (e.g. a website would pay for a lower service level while a VoIP box would pay for a higher service level).

I realize that this is a part of the specification that clearly would not be used now or soon, but it permits OS developers to modify/optimize their memory management to best use the host memory in a IaaS cloud environment. This is a chicken and the egg problem. We need to create a specification that permits OS developers to make this change, so they have a reason to improve OS memory management procedures at the kernel level. If we don't do this, it's very likely that the modern OS will continue to function as a very resource-expensive runtime environment.

I wouldn't be surprised if in the future this permits the creation of memory management algorithms at the hypervisor level analogous to the way TCP/IP and FCP congestion algorithms manage competition for a finite amount of  bandwidth.

Feel free to break off the above discussion into a new discussion thread if necessary.

Andrew de Andrade
Product Manager - Cloud Computing
Locaweb (São Paulo, Brazil)

On Nov 11, 2009, at 1:24 PM, Gary Mazz wrote:

Nicely done...

This proposal provides OCCI with significantly increased options while
maintaining support for more traditional representation methods. If
adopted, this proposal could be used to establish baseline for a
capabilities road map without requiring significant industry investment
in custom technologies.

cheers
gary

Edmonds, AndrewX wrote:

Hi all,

From the previous discussions had and agreeing in principle to OCCI providing a XHTML5 rendering, I'm submitting the following proposal [1] for the group's review. XHTML5 is used to make the representation machine readable and as it is well-formed any XML toolset can digest and manipulate it. Complimenting XHTML5 is the use of RDFa which is used to demarcate information that has semantic significance and in our particular case, used to expose attributes defined by the OCCI specification. The combination of XHTML5 and RDFa is a W3C recommendation [2]. Your thoughts are welcome as ever.

Regards,

Andy

[1] Attached; source available here: http://code.google.com/p/occi/source/browse/docs/occi-xhtml5.xml
[2] http://www.w3.org/TR/rdfa-syntax

Andy Edmonds
skype: andy.edmonds
tweets: @dizz

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

------------------------------------------------------------------------

_______________________________________________
occi-wg mailing list
occi-wg at ogf.org<mailto:occi-wg at ogf.org>
http://www.ogf.org/mailman/listinfo/occi-wg

_______________________________________________
occi-wg mailing list
occi-wg at ogf.org<mailto:occi-wg at ogf.org>
http://www.ogf.org/mailman/listinfo/occi-wg

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/occi-wg/attachments/20091111/cacc64f4/attachment.html 