There are two things to understand.
Virtualization & Economies of Scale.
First let’s remind you what you should already know about the world wide web. It’s millions and millions of computers that are web servers hosting content that you can see in your browser. It doesn’t matter where these computers are or what they look like. Some geek is going to make it easy for you, and the structure of the internet disappears. You just use it. That geek is like your mechanic, or your attorney, or your dentist. He understand the complexities of technical stuff, and there isn’t really an easy way to understand the how, but you can understand the why.
Virtualization is like a game of ‘telephone’ played by robots who can speak any language perfectly.
There are several layers of software that run on the hardware of your computer. Each computer’s CPU requires specific instructions to do the same thing. Your smartphone, your calculator, your desktop, your videogame console. They can all compute 1 + 1 =2. But they all have different CPUs that need to be programmed differently.
But you can write a program that makes your smartphone run the same exact program as the one that runs on your desktop computer. All it needs is what’s call an abstraction layer. What is an abstraction layer? In the game of telephone, one robot whispers English to the second robot, and the second robot speaks perfectly translated French into the ear of the third robot. The third robot has no idea what language the first robot whispered. That makes the second robot an abstraction layer.
A virtual machine puts an abstraction layer above the hardware of a computer CPU. It now can pretend to be any computer. It can speak any language. It can run any program. It can pretend to be
Virtualization has been around for about 40 years. IBM invented something called VM/CMS back in the 70s. (I think). It is now perfected to the point at which the performance cost of the abstraction layer is about 2–3%.
Virtualization works because of something called Turing’s Law or Turing completeness, which essentially states that any string of 1s and 0s can be translated into any other string of 1s and 0s given a set of proper rules. So any computer can emulate any other computer. The limiting factor is efficiency.
It also means any 10 computers can be made to seem like 1 big computer. Also, any 1 big computer can be made to seem like 10 ordinary computers, or 6 and a half bigger computers, or 37 very small computers. You just do the equivalent of turning a dial.
It is important to remember that over the years, compute hardware has become specialized. So you have special hardware for your self-driving car and special hardware for your talking refrigerator. But given the right software, your refrigerator could drive your car. Anything can be done in software, just a little bit slower, again, depending on the efficiency of the abstraction layer.
Economies of Scale
So what if I decide to take virtualization to the max? Lets say I build 20 data centers and each one has 500 racks of compute hardware. I could set up my virtualization to make those computers pretend to be web servers, or car driving programs, or videogame servers or basically any compute thing there is. If I have good business sense, I can rent out compute hardware like Hertz rents cars. I can sell compute capability to anyone. I take the risk and make a big capital investment in my data centers, and now all I have to do is get people to rent the compute power.
At some point, the volume discounts I can get buying chips from Intel exceeds whatever any single business can get. So to handle the equivalent amount of work in my data centers it’s cheaper for me to sell computer time to you than it is for you to buy your own computers and do it yourself.
At a big enough scale, I can borrow money cheaper than your business, I can hire smarter folks. But I still have to solve all of the details of virtualization so that you can do on my compute platform the same thing you would do in your own data center.
A cloud is a large set of compute hardware that runs many different virtualizations at an economically sustainable level.
Now if you want to look under the hood and see what makes a cloud good, you need to get into details of both what the technology has done and what the market has sustained over the past 20 or so years of cloud computing.
The three leaders in the space, Amazon Web Services, Google Compute Platform and Microsoft Azure all started differently. Amazon by doing e-commerce and building out data centers to sustain that business. Google by doing web search and building out data centers to sustain that business. Microsoft partially by doing search and then determining not to be left behind. Because of this history, they are developing different kinds of services, and thus have different priorities.
The cloud will offer particular architectural advantages that are interesting to the geeks who dig looking under the hood. But these will always be dictated by market forces. The question about what is computable and what computers can do was answered a long time ago. 1s and 0s can represent just about anything humans can see and hear. The game is only constrained by compute efficiency and money.