This graph from the National Oceanographic Data Centre, looking at Ocean Heat Content in the upper half of the ocean shows the following:
Roughly 5 x 1022 Joules since 2003. Since the IPCC's graph above up to 2003 shows that most of the energy from global warming is in the oceans, to a first approximation, Ocean Heat Content change since then is going to be close enough to the Total Heat Content change.
So, total heat content change from 1961 to 2011 - 50 years - is approximately 21 x 1022 joules.
A BIG number but somewhat unreal. So how much heat is this. What could it do? What is it in the real world, where we don't routinely look at numbers that big.
This is a rate of heating of 133 Terrawatts. Or 0.261 Watts/m2
133 Terrawatts is 2 Hiroshima bombs a second. Continually since 1961.
It would boil Sydney Harbour dry EVERY 12 HOURS!
One of the worlds great harbours boiling dry twice a day! And this has been happening for the last 50 years.
Where could this heat have come from?
Since the extra heat, mainly in the the oceans is the equivalent of warming the atmophere by 42 °C, if this heat had been extracted from the atmosphere to warm the oceans we would have seen a drop in Air temperatures of a similar scale: ≈ 40 °C or so of atmospheric cooling. I think we can agree that this hasn't happened.
Similarly, if freezing of ice & snow was supplying heat that could warm the oceans - hard to imagine what the process might be but theoretically possible so we need to consider it - this would require the freezing of around 12,500 Billion tonnes of extra ice per year. In contrast actually 500 Billion tonnes of ice is melting each year. No, that isn't the source.
Could it be Geothermal heat - heat coming from inside the Earth?
Sorry, no, that doesn't work either.
The rate of geothermal flux to the surface, for the entire Earth is around 47 Terrawatts. This comes from residual heat remaining from when the Earth was first formed and heat from radioactive decay of minerals within the Earth. And it doesn't change suddenly. What is sudden? To a geologist, 100,000 years is sudden.
But this flow is only around 1/3rd of what is needed to account for the increase in heat. And since this flux has been very steady for a very long time, it can't be the source of the change in heat content. Since it has been so constant for so long, the normal geothermal energy flow must be part of the normal heat balance. Therefore to account for the increase in heat content, the global geothermal heat flux would need to have increased by a factor of 4 over the last 1/2 century - 1 normal flow and 3 extra flows.
Suggest an idea like that to any geologist and just watch for the look on their face!
And now we have run out of terrestrial heat sources that might do the job. Since Geothermal Energy is the only energy source on the planet large enough to even conceivably supply this much energy and even it is too small, that leaves only one option left.
An imbalance in the heat flows to and from Space. Nothing else fits the evidence.
Lets be very clear about this: NOTHING ELSE FITS THE EVIDENCE
Not theory, not ideology, not political views, not internal variability, not questions over surface temperature records, not fudged or not fudged data, not hockey sticks or Medieval Warm Periods, or perhaps missing 'hot-spots'.
The Earth is experiencing an energy imbalance with space!
NOTHING ELSE FITS THE EVIDENCE.
So what are left standing as viable explanations?
Greenhouse Gases and Clouds. Known Greenhouse Effect impacts of the GH gases, and possible changes in cloud behaviour.
Exactly where most informed discussion of AGW occurs. The known impacts of GH Gases and the recognised uncertainties over cloud behaviour.
More at the link - this is an excellent overview which addresses nearly all counterarguments to man-made global warming.