Global primary energy by source
So, what does renewable energy have to accomplish before it can compete with fossil fuels in an open market? Well, in short, we will have to overcome the diffuse and intermittent nature of renewable energy more efficiently than we can overcome the declining reserve qualities and unrefined nature of fossil fuels.
In other words, renewables need to overcome the following two challenges in order to displace fossil fuels in a fair market:
  1. Solar panels and wind turbines need to become cheaper than raw fossil fuels. This is the challenge posed by the diffuse nature of renewables.
  2. Storage solutions need to become cheaper than fossil fuel refineries (e.g. power plants). This is the challenge posed by the intermittent nature of renewables.
Point number 1 is the way in which we procure our energy (mining/drilling fossil fuels or deploying solar panels and wind turbines) and point number 2 is the way in which we make this energy useful to society at higher levels of penetration (refining fossil fuels to electricity or smoothing out the intermittent surges of renewable energy). Without point number 1, point number 2 cannot exist and without point number 2, the energy procured in point number 1 cannot sustain a complex society such as ours.
Thus, if renewables are to challenge fossil fuels in an open market, technology must advance to the point where renewables can compete under both these points. This article will examine whether this is in fact possible.

The diffuse nature of renewables

Renewable energy advocates often point to the total energy fluxes of the Earth (below) and proclaim that renewable energy resources are essentially boundless. Yes, it is true: we are surrounded by incredible amounts of diffuse renewable energy (e.g. solar radiation and wind). Unfortunately, however, this energy is useless to us unless it is concentrated into forms such as electricity or fuels.
RenewableEnergyPotentialVsFossilFuels
The reason behind this is called the second law of thermodynamics which states that energy must flow from a concentrated form to a more diffuse form in order to do work. Our entire society was built on the work performed through transforming concentrated fossil energy to diffuse heat and, in order to compete, renewable energy technologies also need to deliver such concentrated energy.
Now there is no question that renewable energy can be sufficiently concentrated by solar panels and wind turbines. The only question is whether this can be done more cost effectively than simply digging already concentrated fossil fuels out of the ground.
This challenge is two-fold. Firstly, energy does not like to be concentrated, hence the fact that the average commercially available solar panel is only about 13% efficient. And secondly, because the energy source is so diffuse, vast areas need to be covered in order to harvest this diffuse energy. The video presentation below gives a very informative discussion on the size of the areas we are talking about here (the video is long, but definitely worth the time).
As a result of this challenge, it was previously calculated that the solar panel price needs to fall to about $0.31/W installed in order to compete with coal at $100/ton. It is therefore clear that installed solar prices still need to fall about one order of magnitude [10 times] before we can see a sustained market driven displacement of coal by PV.
Is this possible? Well, the most optimistic projection in the OpenEI database sees solar PV levelling off at about $1.44/W installed which is more than quadruple the required level. Perhaps we will be pleasantly surprised by some technological miracle in the medium-term future, but achieving the required prices with current PV technology will unfortunately be completely impossible.

The intermittent nature of renewables

If intermittent renewables like solar and wind are ever to contribute a sizable portion of our global energy mix, a large amount of additional infrastructure will need to be deployed in order to counter the large fluctuations in output varying over timescales ranging from seconds to years.
As an example, the variability of wind generation in Germany for 2012 is shown below. On a countywide basis, the output varied over more than two orders of magnitude [over 20 times] from a minimum of 0.115 GW to a maximum of 24 GW. It is clear that a large amount of extra infrastructure will be needed in order to smooth out this erratic output to something that better resembles the demand profile. Solar PV is of course even worse because it generates no power whatsoever for the majority of the time and delivers most of its energy in the few hours around noon.
German wind output 2012
Again, there can be no doubt that we have a wide range of technically proven solutions to this problem. When just looking at the area of energy storage there are many proven ways to store energy in chemical, kinetic and various potential forms. But again, the challenge is to deploy these solutions at a lower cost than that involved in the refinement of fossil fuels.
A coal power plant is the most expensive kind of fossil fuel refinery. For example, a standard coal-fired power plant must sell electricity for about $0.06/kWh, but coal at $100/ton costs only $0.015/kWh. The remaining $0.045/kWh represents the price of refining coal to electricity and arises primarily from the low efficiency and high capital costs of coal plants.
So, how does energy storage compare? Well, a recent test of lead acid and Li-ion batteries found that these technologies could store energy for about $0.34 and $0.40 per kWh over their respective lifetimes. Hence, we again have to conclude that the most ideal renewable energy storage solution is still about one order of magnitude away from challenging fossil fuels on a level playing field.
The Li-ion battery throughput cost of $0.40/kWh mentioned above was calculated for an initial cost of $600 per kWh of capacity. Most optimistic projections for Li-ion battery costs give longer-term prices at about $200 per kWh of capacity. At these prices, battery storage would be about triple the price of refining coal to electricity. Again, we need a technological miracle.

Final word

So, these are the facts. In order for intermittent renewable energy sources such as solar PV to effectively compete with fossil fuels like coal, both the price of installed solar panels and the price of battery storage will need to reduce by a full order of magnitude. In addition, optimistic long-term projections state that both solar panels and battery storage will reach technological maturity at roughly triple the cost of their fossil fuel counterparts.
Does this mean that it is fundamentally impossible for renewable energy to trump fossil fuels? Well, I would stop short of saying that, but, from this analysis, it appears unlikely that we will see a large scale market driven displacement of fossil fuels by renewable energy in the first half of this century.