Virtual and distributed computing technology and applications
Thursday, March 22, 2007
IDC: Server shipments slow on spread of virtualization:
"Growth in the x86 server market revved slightly in Q4 2006, growing 7.0% in the quarter to $7.2 billion worldwide, its fastest growth rate in five quarters, but unit shipment growth continued to moderate with growth at 1.1% year over year, to 1.85 million servers as customers continued to consolidate their IT infrastructures, .. "For the first time in more than 10 years, average selling values in the quarter increased year over year as IT managers move to consolidate IT workloads. This shift toward a shared compute infrastructure is driving additional scalability, memory attachment and I/O needs, which in turn, lead to higher average selling values." ..
Microsoft Windows servers .. revenue grew 9.4% and unit shipments grew 5.1% year over year. Quarterly revenue of $5.3 billion for Windows servers represented 34.9% of overall quarterly factory revenue, the single largest revenue segment in the server market, IDC reported.
After two consecutive quarters of single-digit revenue growth, Linux server revenue growth accelerated once again, growing 15.3% to $1.8 billion when compared with Q4 2005. Linux servers now represent 11.9% of all server revenue, up more than one point over Q4 2005. But Linux server shipments declined 0.8% year over year after 18 quarters of double-digit shipment growth, as IT consolidation extends its reach into the open source domain...
Unix servers experienced 2.8% revenue growth year over year when compared with Q4 2006. Worldwide Unix revenues were $5.1 billion for the quarter, representing 33.5% of quarterly server spending." Itanium, z/OS and blades sold about $3.5B combined." 10:38:40 PM
Friday, March 16, 2007
The Globus Consortium Journal
: Overview of Virtualization Technology in Distributed Computing workshop. "Among the highlights was an interesting paper from Intel dissecting the
performance of Xen networking. A wonderful adoption scenario was
represented in the work from the University of Marburg where
suspend/resume properties of VMs are being used to improve backfill
strategies in the local scheduler - computations running in VMs are
simply suspended when a large parallel job is scheduled to run and
resumed afterwards. The remarkable part of this work was that it was
very much requirement-driven and has been voted into production by
users. Another interesting talk came from the Australian Partnership
for Advanced Computing (APAC) described their experiences using virtual
machines in production Grids for a couple of years now." 11:20:49 PM
Monday, February 19, 2007
IE6 on XP SP2:
Nov 2006: Microsoft offers a free download of "a VPC virtual machine image containing a pre-activated Windows XP SP2, IE6 and the IE7 Readiness Toolkit to help facilitate your testing and development. The image is time bombed and will no longer function after April 1, 2007. We hope to continue to provide these images in the future as a service to web developers." Even with only 6 weeks to go, it might be a useful device for testing. 5:11:29 PM
Automatically download VMware images:
Thoughtpolice.co.uk offers VMware images ready-to-use for Fedora, Ubuntu, and FreeBSD. When they issue a new one, they place a torrent link in a file you can download via rsync to torrent the image, all automatically. Subscribing to virtual appliances is an idea that's been much talked about, and there may finally be a sufficient audience for it. 12:53:29 AM
Running Windows on the Amazon Elastic Compute Cloud with Qemu and Linux:
At first this seems like a kludge, but it has some other benefits:
- "Significantly lower per/hour costs by running several
additional Virtual Machines per AMI. This would require a Qemu
abstraction / accelerator. You could foresee up to 15+ addition concurrent operation systems.
- Able to more effectively simulate varied operating/hardware environments for easy migration from legacy systems.
- Backups to S3 - Using the Qemu-img tool, instant snapshots
can be saved to S3. Used with a versioning system, this could provide
for unlimited roll backs.
- VMware compatible - Qemu-img supports vmdk conversion as
well as several other formats enabling easy migration from existing
virtualized server environments"
Use of QEMU's virtual disk differencing system has appeal. The big question is the speed of the resulting systems, and any possible limitations from the size of the EC2 virtual machines.
Wednesday, February 14, 2007
Dell is posting VMmark results:
"Dell has recently measured
various configurations of two- and
four-socket PowerEdge servers running VMware ESX Server 3 using a beta
version of the VMmark benchmark. The graphs shows a 4-socket PowerEdge
6950 with dual-core AMD Opteron processors has 57% higher
virtualization performance than a 2-socket PowerEdge 2950 with
dual-core Intel Xeon series 5100 processors. The 2-socket PowerEdge
2900 with new quad-core Intel Xeon 5300 series processors shows 51%
better performance. Equally impressive, all three 8-core servers were
able to support the same number of heavily-loaded virtual machines." 10:47:41 AM
Tuesday, January 23, 2007
Amazon Tries Hand at Virtual Appliances:
"Amazon Machine Images or AMI [is] a packaged environment that includes all the necessary bits
to set up and boot Amazon EC2 instances. The company launched their Elastic Compute Cloud (EC2) - Amazon's
hosted, on demand virtual datacenter based on the Xen technology - with
hopes of creating demand for a pay-per-use model of virtual machines. .. Amazon is now asking its EC2 community members to share their AMIs with other Amazon Web Services developers." Amazon keeps a listing of public AMI's
with RSS feed
. There's even a Windows AMI running under QEMU under Xen (whew!). XML manifests are required for each AMI. 6:42:50 PM
I resumed blogging for my new company today, with comments on operating systems and virtuailzation. Enjoy. 5:48:35 PM
Wednesday, January 03, 2007
Monday, November 27, 2006
VMware Reporting Huge Growth in 3Q 2006:
"The company's revenue continues to climb. For the first half of 2006, VMware reported its revenue as $288 million. Compare this to the annual revenue recognized in 2005 - $387 million, 2004 - $218 million, and 2003 - $100 million, the growth is staggering. [approx 6x in 3 years.] More than 20,000 companies now [use] VMware technology, including 99 of the Fortune 100 companies" 12:19:36 AM
Server Virtualization Becoming Norm: "Server
virtualization no longer has the same cache it did a year or two ago.
And the reason is simple: Now that everybody is starting to do it,
there is nothing to boast about. According to IDC, more than
three-quarters of companies with 500 or more employees use virtual
servers, and 45 percent of all new servers purchased this year will be
virtualized." 12:16:26 AM
Tuesday, November 21, 2006
Hackers Use Virtual Machine Detection To Foil Researchers:
Funny benefit for users of virtual machines -- if you run a vm, increasing amounts of malware will refuse to run. "Hackers are adding virtual machine detection to their worms and Trojans to stymie analysis by anti-virus labs, a security research said Sunday. The tactic is designed to thwart researchers who use virtualization software, notably that made by VMware, to quickly and safely test the impact of malicious code. .. "Three out of 12 malware specimens recently captured in our honeypot refused to run in VMware," said Lenny Zeltser, an analyst at SANS Institute's Internet Storm Center (ISC) in an online note Sunday.
Malware writers use a variety of techniques to detect virtualization, including sniffing out the presence of VMware-specific processes and hardware characteristics, said Zeltser. "More reliable techniques rely on assembly-level code that behaves differently on a virtual machine than on a physical host," he added. " 4:55:43 PM
Monday, November 06, 2006
Microsoft debuts virtual appliances:
Following VMware's lead in promoting software appliances, Microsoft is promoting the idea among its third-party developers for its Virtual Server VHD (virtual hard disk) format. "More than twenty Microsoft partners have already committed to
distributing software via VHD Test Drive Program by the end of the
year, including Altiris, BEA Systems, Check Point, Citrix, CA,
CommVault, Dell, FullArmor, HP, Network Appliance, Platespin, Portlock,
Quest Software, SourceCode Technology Holdings, Symantec and UGS." I wonder how Microsoft will finesse the licensing of their operating system in this format. 10:29:34 PM
For PG&E customers, it pays to virtualize:
A major California utility creates an incentive for saving energy through server consolodation with virtualization. "They said the company had recently approved a three-year, $950 million plan whereby PG&E will reimburse 50% of the costs of a server consolidation project, including software, hardware and consulting, up to a maximum of $4 million per customer.
"We will calculate the energy consumed by existing servers and subtract the difference in the energy consumed by the new servers," said Randall Cole, senior project manager for PG&E's Customer Energy Efficiency program. "Then we'll pay 8 cents for every kWh saved over the first year of the server virtualization implementation." Electricity rates for PG&E's non-residential customers currently stand at 12 cents to 15 cents per kWh. ..
What's in this incentive plan for PG&E? The motives are fairly obvious: "We don't want to build any more power plants," said Bramfitt. At the same time, "regulators have told us loud and clear that we need to meet certain energy efficiency goals," he said. "We want customers to save energy, and we will pay them to do so." ..
"Virtualization isn't mainstream yet, and once it ramps up, I'm not going to incent it anymore," Bramfitt said. "But, in the meantime, I'm going to do whatever I can to light a fire under peoples' butts."" Nice to see a coincidence between my interests in energy efficiency and my main work in computers. And I wish I knew more about how the electric utilities are regulated in California, so they make money with this program, compared to other states where they do not. 10:25:11 PM
Thursday, October 26, 2006
Tuesday, September 12, 2006
VDI Server Sizing and Scaling
: VMware white paper on the capacity of its latest software to host multiple desktops. "We looked into two different workloads for capacity planning guidelines. For a light worker workload an HP DL 385 G1 server could support 42 Windows XP virtual machines. For a heavy worker workload the same server supported 26 Windows XP virtual machines... [The] server considered to be at capacity when the client fails either due to a 10 percent increase in the canary time observed for the workload or when a client script fails due to predefined timeouts." This was at approx 70% CPU utilization.
"Hardware Configuration for Server Running Desktops: One HP ProLiant DL385 G1. Two 2.2GHz AMD Opteron dual-core processors, 16GB RAM, 2 Ultra 320 SCSI drives (2 × 146GB disks, 15,000 rpm)"
As expected, powering a VM on takes more CPU than un-suspending it. Buth there's another option: you can "power on approximately the number of desktop virtual machines you estimate the host can support, with the virtual machines configured to go into standby mode after a certain fixed interval. ESX Server 3.0 introduces new power management options [e.g.] Windows XP desktops are configured to Wake on LAN .. When any network traffic arrives for a virtual machine, it seamlessly wakes up from standby mode. .. Compared to the resume from suspended state or cold reset option, this activation scheme offers very fast activation (on the order of a few seconds).." 6:10:35 PM
Saturday, August 19, 2006
The future of appliances is virtual
: Good summary of the advantages of virtual appliances: "Virtual appliances have some distinct advantages over traditional appliances
and servers. So many, in fact, that the age of the traditional hardware
appliance could be coming to an end. .. although hardware appliances are usually much easier to deploy and manage than traditional full function servers, they are wasteful when it comes to power consumption, heat output and space requirements. Each appliance also needs cabling to networks and power outlets, and before you can use them, most also need you to connect a serial interface to them to assign an IP address. This last point may seem trivial, but finding a PC with a serial port and a copy of HyperTerminal thatís within a cableís length of a new appliance is not always easy. Plus, [you may need] two appliances if you want a reasonable degree of fault tolerance.
In contrast, virtual appliances can be run on whatever hardware you see fit,
ranging from a cheap single-socket desktop for appliances that you just want to
evaluate, through to extremely fault-tolerant multiprocessor server hardware and
SAN storage for a virtual appliance that you need to be constantly available." And of course they have advantages for the vendor, allowing them to distribute easily-changed software rather than inventory and ship hardware. 12:31:29 AM
Friday, August 18, 2006
Parallels Compressor Server
: In addition to a virtualization engine for the Intel Mac, Parallels offers the Compressor, which "improves the performance of any Windows-powered Parallels, VMware and Microsoft virtual server or virtual workstation by reducing virtual hard disk size by 50% or more. .. Compatible with virtual servers built with Parallels Server (expected late 2006), VMware Server, VMware GSX Server, Microsoft Virtual Server.[or] Parallels Workstation 2.1, Parallels Desktop for Mac, VMware Workstation, and Microsoft Virtual PC." Two versions for sale, $50 and $180, with 15-day free trials. 11:55:38 PM
Monday, August 07, 2006
Let's get virtual
: How to convert an old, cheap PC into an iSCSI server (and use an dual Athlon white box for VMware ESX). 11:37:48 PM
Monday, July 31, 2006
Creating Smaller Virtual XP Machines:
Explanation and how-to for manual reduction in Windows images form >2 GB to under 700 MB. The commentary after the article has pointers to many sources, including: 12:27:52 AM
Wednesday, February 08, 2006
Wednesday, January 25, 2006
Small unit that provides voip and gateway functions. Interfaces for ethernet, and for "normal analogue telephone (or cordless) and a standard PSTN line. In
technical terms, this has both an FXS and an FXO interface - the FXS
interface allows a normal telephone to be turned into an IP phone and
the FXO interface provides connectivity to a PSTN line (or of course
another voip adapter which is locked by the provider). These interfaces
can be configured independantly using the onboard web interface where
when you log in as an admin user and switch to advanced mode, there are
hundreds of settings ..." Has instructions for remote control by Asterisk. About $100. It ought to would work with a virtual machine Asterisk
, I suppose. (Spec sheet here
). 3:41:26 PM
Sunday, January 15, 2006
Sunday, January 08, 2006
Thin client server:
If you run XP as a VM, you can use RDP into it to control from a desktop. To convert old PCs into cheap terminals, this product from X2 seems promising. 11:25:47 PM
Monday, November 07, 2005
Topologilinux - [Running Linux inside Windows]: Based on slackware and colinux. Last updated 1Q 2005. Also runs native. "Topologilinux is a free Linuxdistribution to be run on top or inside your existing windows system. The main thing with Topologilinux is that it does not require any partitioning at all. (uses a single file as linux root system)..
Topologilinux needs a PC 486 or higher with about 100MB free diskspace and about 8MB ram. If you want to install everything you need about 2,5GB free space. If you want to have a X system it is recommend to have a Pentium PC with 32MB ram or more. And If you want to use colinux and X in windows you will probably need a pention 300Mhz or above with Win2k and about 96Mb ram. .." 9:15:51 AM
Monday, September 26, 2005
Update your Linksys router with Sveasoft's firmware
: "In its GPL Code Center
, Linksys provides the source code for most of its devices. However, unless you're a programmer, this isn't going to do you much good. What can help you out is what Sveasoft has done with that source code. Based in California, this company has taken Linksys' source code and created new versions for replacing factory firmware
. Basically, installing this firmware takes a limited functionality $50 consumer router and adds many of the features of an enterprise router. ..
Sveasoft actually sports three different families of firmware: Sveasoft firmware for Linksys WRT54G and WRT54GS routers, Alchemy firmware that works with a list of routers (which is free and adds a lot of the features listed above), and the aforementioned Talisman firmware." 12:05:41 PM
Wednesday, September 07, 2005
IBM open source rHype:
Back in Feb 05, IBM released an open source (GPL) virtualization engine called rHype, for 'research hypervisor'.
(Updates are posted through July 05). It is a paravirtualizer, so most O/S need modification before running. It runs on several hardware platforms (Power, Sony Cell, x86), and has emulators for others (Mambo, QEMU). "In addition to the rHype project, IBM has a commercial hypervisor running on machines that use its Power processors. Because rHype uses the same interfaces as the commercial hypervisor, Linux doesn't have to be modified to run on an rHype-Power foundation. With rHype on x86 chips, Linux must be modified to work. " rHype is expected to help Xen authors advance their platform. Charles King, principal analyst of Pund-IT Research, notes a trend I'm watching: "Juggling numerous tasks has long been a useful ability for corporate computing centers. Now such abilities are increasingly useful at home as computer networks get more complex and useful, King said. "It's fascinating to me that something that's been seen as a benefit for enterprise data centers is percolating its way down into the set-top box," King said. " 8:48:23 AM
Wednesday, August 31, 2005
An Introduction to Virtualization
: A history from 1959 to now (with a quote using "virtual machine" in 1961), plus a directory of various virtualization projects and products. 10:57:20 PM
Monday, August 22, 2005
Tuesday, August 16, 2005
New projects at Sun
: "Sun's Squawk project was noted by Gage. Written in Java, Squawk is intended to provide multiple virtual machines and treat an application as an object, Gage said. Squawk would provide more efficient use of memory and power, according to Gage. Squawk was described on Sunís Web site as a compact, high-performance Java environment. "This is perfect for wireless sensor devices," Gage said." 7:54:50 PM
Monday, August 15, 2005
What is a Hypervisor
: Brief summary of how a virtual machine hypervisor relates to its hardware, and of upcoming hardware support. "Intel and AMD are both working on providing full Hypervisor support at the CPU level, introducing a new CPU Ring (-1), with respective project codenames "Vanderpool
" and "Pacifica
". A thin layer of Windows virtualization technology will move to execute at Hypervisor CPU Ring -1 to take full advantage of hardware innovations from Intel and AMD." 2:37:46 PM
Saturday, August 13, 2005
OSx86 in the Wild
: Running OSX as a guest VM will be interesting (once it gets released legally). There are several bio and grid apps that run natively on OSX, plus various media products that are better in OSX. "We can confirm that a torrent containing a VMware image of Tiger x86 has been leaked to a major torrent site. This virtual hard drive image has supposedly been hacked to circumvent TPM and run on non-SSE3 machines. .. We obviously have not tested this image (and will not, due to its blatant violation of the DMCA) .." 7:43:06 PM
Wednesday, July 27, 2005
Q&A: Microsoft's Bob Muglia discusses virtualization plans: Senior vice president of Microsoft Corp.'s Windows Server division: "Today, we have a product called Virtual Server that sits on top of Windows and provides virtualization capabilities. In the future, we're going to build the hypervisor and the virtualization stack into Windows. So while it's a whole new set of technologies, much, if not all, of what Virtual Server does today goes into the operating system. It becomes an operating system feature. .. I think about the '07 generations of the operating system, say '07-'08 as all being Longhorn, maybe even to '09 for Longhorn R2. .. the virtualization features are in the Longhorn time frame, but it's not in the initial release of Longhorn .. "
The Register's interpretation: "Microsoft has planned a long, slow death for the Virtual Server product it acquired in 2003. Redmond will stop selling a standalone partitioning product when the server version of its "Longhorn" operating system arrives... It looks like Microsoft plans to "featurize" at least basic server partitioning functions and give the technology away with the OS. VMware, however, apparently has little to fear until 2009. The virtual machine market will keep kicking back cash to the software maker and its owner EMC." 12:54:06 AM
Tuesday, June 28, 2005
Friday, June 24, 2005
Meiosys: Interview about an application virtualization technology. The company was recently purchased by IBM. "From our perspective, virtual machine technology is really about resource isolation. It's about creating OS containers that are isolated from other containers on a physical machine. You can run multiple virtualized machines in parallel on a physical machine and have complete isolation. Technology like VMware gives you the capability to run multiple OSs in parallel on the same machine, so you can run Windows and Linux in parallel. You get some utilization benefits out of that on a machine-specific basis, but the real value of that technology is resource isolation.
We're application virtualization so we're above that infrastructure and resource level, and our technology is much more fine-grained. We build a container dynamically around a specific application that runs either on a physical or virtual machine, and we allow the application to be moved from physical machine to physical machine, or virtual machine to virtual machine. We are more granular because we can wrap and move specific applications and application processes, and of course multiple applications can be running within a particular virtual machine.
We also have developed a patented TCP/IP socket migration technology that allows us to preserve state and connection during the relocation of an application from one machine to another. .. Because we provide a fine-grained virtualization at the application layer, our overhead is under 1%, compared to 15% to 40% overhead for some other technologies which enable mobility. .. We abstract the application from the OS. We run for the most part in user space, not kernel space. "
Also has interesting comments about selling data center software into enterprises through channels. 9:23:21 AM
Saturday, June 18, 2005
Red Hat virtually supports Xen:
Fedora Core 4 released with Xen. Future versions of Fedora will probably be driven by a new Fedora Foundation rather than Red Hat corporate. Plus, here's a tutorial about Xen on SuSe
. 9:22:53 AM
VMware Technology Network (VMTN) for Developers: VMware aims to be a platform for software distribution and development. As John Sequeira recognized years ago, software can be delivered better in vm's (especially open source software). "Pre-built application environments in VMware virtual machines.. [are now] available for download to any software developer. Industry-leading software vendors BEA Systems, MySQL AB, Novell, Oracle and Red Hat are among the first to distribute their software in virtual machines. Entire application environments can be pre-installed, pre-configured and "saved" within a best-practice virtual machine. Developers can eliminate many of the traditional stumbling blocks associated with testing, evaluating and deploying new software by using these pre-built applications within virtual machines."
Also announced: VMTN Subscription for a suite of most VMware products with support and upgrades priced at $299 per developer per year.