Thursday, August 2, 2018

Can Renewable Energy and Machine Learning Enable an Energy Independent India?


These are high-times for Artificial Intelligence (AI), specifically its Machine Learning (ML) form. Both are increasingly coming into the public consciousness, as they are applied to an array of sectors. Applications in the Power sector, though less visible than say physical robotics, are the most fundamental, especially in an Indian context, as its evolution will also transform many other sectors.

As a tropical country we are endowed with abundant sunshine and wind, but still import much of our energy in the form of oil and coal. The current government is pushing to transition us to a ‘Solar Nation’, in no small part to reduce the national import bill. Achieving this could accelerate the Make in India campaign, because national energy independence will mean the cheapest power possible. The vision is for a 100% renewables-powered grid, giving 100% reliability of supply, enabling prices to be a fraction of what they are today.

Technically India already has ample power, with approximately 300GW of installed capacity, and only 150GW of peak demand. However, this is not a practical reality. Rather, it reflects the challenges of managing supply and demand. Our per capita consumption remains low because most people still have only a limited access, in terms of connectivity and availability. Further, though abundant, Solar and Wind are inherently variable, and will exacerbate grid-volatility as integration increases. Even with the small penetration of renewable generation today, the intermittency has caused a significant rise in balancing costs, which in the future could be huge.

Added into the mix, the future will see increasing Distributed Energy - both localised green generation, and storage. Though in some instances this will take pressure off the central grid, for example by reducing the load on transmission infrastructure, overall it will increase volatility. Today generation and consumption locations are essentially fixed, we at least know where they occur, even if not quite what volumes and when. But we have little idea of the future profile, especially with Electric-Mobility coming into the frame.

It is already difficult for humans to manage this complexity, but these challenges aren't dire portents for the sector. What we need is a modern approach to address key deficiencies, by analysing disparate real-time data for optimal decision-making. This must be cost-effective, and of all available avenues software will always be much cheaper and more efficient than physical buildout.

This is where AI & ML excel, with by far the greatest benefit-to-cost ratio. Crunching the exponentially growing volumes of systems data required to balance gird-volatility, forecast future trends, optimise existing assets and plan future infrastructure. They are already handling these issues in the real-world, from forecasting of solar generation to prices on the power markets, and even detection of power-theft.

In a renewable and distributed energy future, AI & ML can manage key aspects of the energy system, with human-oversight replacing human-intervention for the highest decision levels. It thus has the capability to help us realise the vision of an energy-independent India, with affordable power for all.


Credit to Niladri Roy for co-writing this article.

Human vs Machine Learning

In previous articles I've written quite a bit about Artificial Intelligence and Machine Learning. This article hopes to explain these terms in fine detail and compares them to the workings of the human brain.

Learning Machines - Not So New

Though the terms Artificial Intelligence (AI) and now Machine Learning (ML), have become widely vaunted for an array of areas in industry and life, they conceptually have a very long history, being prevalent in prior decades and even centuries. We can actually go back several millennia, to the ancient world, for the first instances. 

There have been examples of ‘intelligent’ machines peppered throughout ancient mythology, and even an allegedly real one in ancient China. Given there is still limited understanding of what AI and ML actually are, examples from myth can help with differentiating and understanding both concepts, and ultimately explore ML in more detail.
The Automatons of Hellenistic Greek mythology, were metal statues crafted by Hephaestus, the smith-god of the Olympian deities. The best of them could think and feel just as humans do, like the Kourai Khryseai - ‘golden maidens’. This is the most common perception today of AI (just without the moving limbs, or heart and charm).

But more usually, the Automatons were only able to think, feel and ‘learn’ within the boundaries of what they were designed for. Like those created to assist Hephaestus in his weapons forge up on Mount Olympus. Or Talos, the giant who patrolled the shores of Crete to defend against attacking fleets. Or the Keledones, which were also golden maidens, but only able to sing and learn new songs and music. All were excellent at doing and learning about their respective tasks (or datasets as it were), but not much else. So the Automatons are pretty apt early examples of what AI, and particularly ML actually are.

What Are Artificial Intelligence and Machine Learning?

In simple terms, Artificial Intelligence is the broad notion of machines being able to perform intellectual tasks in a manner similar to human cognition. This is usually imagined to be the much conjectured ‘technological singularity’, where human cognitive capabilities are surpassed, possibly resulting in drastic and dire existentialist threats to humanity (so say messers Hawking, Musk, Gates, and co anyway). This concept is also known as ‘Artificial General Intelligence’ (AGI), which though of course a future possibility - whether malevolent, benevolent, or indifferent - is far removed from current reality. One reason being that human intelligence is not singular in nature, nor is the physical brain yet fully understood.

Figure 1. Types of human intelligence.

Machine Learning is a subset of AI, based on giving machines access to datasets, then letting them ‘learn’ from these. Specific problems are solved without the need for explicit programming, by feeding in data and training the system to derive information out of them. Once trained, any new data given to a machine will yield desired outputs - solutions to the initial problem. Starting from this basic and well-defined premise, better insights can be gained as to how a machine learns.

Current (mis)Perceptions

There is very varied understanding of AI and ML. Both are frequently misperceived as interchangeable, adding to the confusion. AI is indeed a rather nebulous term, but ML is a well-defined subset of it with tangible applications. It’s really ML that is making great strides in an ever more diverse range of fields.

However, because of the misperceived interchangeability, customers – in the energy sector and more widely - expect ‘AI’ to deliver everything now. Customers are often unaware of the numerous remaining incremental steps that are necessary to achieve their respective visions. As with all technology, and indeed life, development takes time, an often frustrating universal truth.

Supervised, Unsupervised, and Reinforcement Learning

Machine learning is commonly split into three main types: supervised, unsupervised, and reinforcement learning. A simple example of supervised-learning is having labels for sorting colours, or animals, whereby the machine is supervised (by a human) in how it categorises these. In unsupervised learning, the machine comes up with its own labels and logical groupings. So just like the mythological Automatons, a supervised machine can adapt and learn within the parameters it is given. However, it can only expand its range, by reprogramming through external intervention - from Hephaestus or a human respectively.

Also in unsupervised learning, whereas a human may use groupings such as age for classification, a machine may form entirely different classifications from the same dataset that are otherwise unapparent to a human. In reinforcement learning, the algorithm tries to find the best way to earn the greatest reward. These now lead to to a more dynamic and interesting space, where the lines get blurred a little.

Deep Learning

Deep Learning is a subset of Machine Learning, which is closer to human learning - give a machine massive amounts of data and let it come up with something by itself. This can be both supervised and unsupervised, so some real projects dedicated to deep learning from Google Brain’s DeepMind, can illustrate these.  

Neural networks are designed with the purpose of modelling or mimicking the human brain, better than previous approaches. One such network was given access to unlabelled YouTube images, and nothing else, to see what it came up with. Eventually, the machine learned to identify….cats! Machine image recognition is quite analogous to the human visual cortex and neurological processing, so has been well replicated. Though humans can still pick a face out in a crowd more easily, machine image recognition has made great strides recently with the advent of convolutional neural networks, which are specifically designed for shift-invariant object recognition in images.

Figure 2. Convolutional neural network for image recognition.

Of more interest and complexity is AlphaGo. The game of Go was invented in China over 2,500 years ago, and is believed to be the oldest board game still being played today. It has simple rules but has long been regarded as a difficult challenge within the field of AI and ML. Some of the complexity is attributed to the sheer number of possible game variations, which are on a scale similar to the number of atoms in the universe. Building a system that masters this game requires considerably more components that mimic human thought processes than other games, such as chess.

The AlphaGo neural network was given no rules, just historical games and datasets. The network was able to incrementally learn the rules, or ‘policies’, through self-simulated games, and what the best moves were to defeat a human. Millions of games were simulated, with high scores used to justify, or gradually reinforce its learned behaviour. So in very similar manner to a human, it post-rationalised, learning tactics to get high scores, which reinforced its behaviour.

Previously, it was generally believed that a decade or more would need to pass before a machine could defeat a professional player. Yet in 2016, a 5-match human vs machine contest, between one of the best professional human players and AlphaGo, saw the machine unexpectedly triumph 4-1. This was through finding ways of winning which ran contrary to professional convention, and has actually redefined the way the game will be played in the future by humans.

While the original AlphaGo needed the expertise of humans to get to such a level, the latest evolution of the program, AlphaGo Zero, has been able to consistently demonstrate superhuman performances well exceeding its predecessor. All without any human inputs or historical datasets, by knowing only the rules of the game and by playing games with itself. This is definitely a significant achievement in both ML, and wider AI.

Explicability, Predictability, and Volatility 

Despite these strides, there is still no way to understand why machines make any of these decisions. Nor can we really get any insight either, given the inherent black box nature of these algorithms. This lack of explicability is significant from a human perspective. Humans still always look for explicability and predictability - formulas for ‘what’s going on behind the scenes’. Classical ML techniques which were derived in the 80’s, are all statistical in nature – support-vector-machines (SVM), tree-based, or knowledge-based algorithms. In other words, they can be grasped by (capable) human minds. It is very challenging for humans to accept anything otherwise. Even with all the volatility and uncertainty around us in daily life, humans are neurological deeply hardwired to search for patterns and predictability.

Yet almost dichotomously, the human mind also functions in a very similar ‘inexplicable’ way to these black-box machines. What is called ‘gut’ feeling or instinct, is simply the brain making superspeed assessments based on a lifetime of data points and bypassing conscious consideration. We can’t explain it, but we do trust it without hesitation, even when we can’t fathom the pattern. So why not the same for machines? Machines can and have been proven to see patterns that humans can’t – patterns that sit outside of classical information theories and conventional explicability.

As the share of renewable generation on the grid increases, so will grid-volatility. So ‘inexplicable’ ML that can process patterns at super-speeds far beyond what humans are capable of, is exactly what will be needed. ML is an essential tool to manage extreme grid volatility, through forecasting of weather and demand based on past behaviour and data points. ML doesn’t need to be explicable to be useful, and its applications in forecasting within the context of energy specifically, will be delved into more deeply in the follow up article. This will include the 5-to-10-year incremental journey from reality today to where customers think capability should be.
 Figure 3. The Turing Test

The Big Question

With ‘creativity’ and learning in mind, can machines mimic that so cherished of human traits, the one held up as the highest of virtues and only possible with a ‘soul’? Well let’s flip the question around - how unique is human creativity really? When analysed, much of what humans produce is surprisingly repetitive. Take Art, even with its seemingly endless variety, it all still falls within set bounds - for a machine with enough processing power. There are already examples of computers painting and drawing pictures, which are indistinguishable from those by a human hand and heart. Then there’s music, which has wide and ubiquitous commonalities across genres and cultures.

Human creativity then maybe isn’t that special, and of course “all art is quite useless”, as Oscar Wilde famously said. However, there is another dimension that should be separated out. The preceding examples are all about production, rather than interpretation, and understanding ‘why’ humans react to art is still beyond machines (and it should be noted, many humans too). Another aspect of human creativity is the ability to do research, design and innovation - the highest level of insight, generated from a combination of multiple cognitive faculties, or modules. This is the ultimate ability of the human brain, which is what AGI aims to achieve. Currently, there is narrow AI within bounded modules, but no sense yet of how to plug them together. Though raw computational power has already far surpassed human cognitive capacity, it still doesn’t enable multidimensional integration of these modules. Development is certainly far from anything that could pass the Turing Test. This experiment was hypothesised by arguably the Godfather of ML, Alan Turing, to assess a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from a human.

Conclusion

Hence true machine creativity is still far from possible yet - a neural network will not learn to paint then suddenly decide to become a sculptor unprompted, nor write a new company energy policy. So, is human learning better than machine learning? The ideal scenario is to combine both - ML’s ability to crunch enormous volumes of data, with the still uniquely human ability of creativity, to achieve the highest levels of decision making.


Credit to my colleagues Niladri Roy, Rohan Nongpiur, Sho Akama and Aseem Khanduja for co-writing this article.

Harnessing the Wind and Sun to Achieve an Energy Surplus Future

Imagine a future where a 100% renewables-powered electricity grid harnesses the abundance of energy beamed down on us daily, to give 100% reliability of supply. Enabling energy prices to be a fraction of what they are today, or perhaps even universally free. Let’s consider two paths of thought, one into the far future, the other more nearer-term for India, and rather imperative.

Looking to history to inform our future-gazing, the indications are that we’ll find ever newer ways to consume energy. So we may take indeterminate time to reach ‘surplus’ in the strictest sense, though that does not preclude universal free access. Even if we developed the ability to absorb all the energy emitted by a star - perhaps through the hypothesised Dyson Sphere - and become a ‘Type 1 civilisation’. Tech advancement will likely hog whatever we produce. More likely, we’ll develop ships that are electric instead of solid-fuel run, and hyperloops to replace trains. Energy is like money, we will always find ways to consume it - our way of life will just change to match.

Today however, India already has an energy surplus, with approximately 300GW of installed capacity, and only 150GW of peak demand. But this is a technical rather than practical reality. There is still load shedding in most parts of the country after all, and our per capita consumption remains low because most people still have only limited access, in term of both connectivity and availability of power. Further, India is a tropical country blessed with ample sunshine and wind, even in the monsoon months. Yet at present, India still imports much of its energy from abroad, notably in the form of oil and coal. The government is convincingly pushing for renewables, and wants to transition us to a ‘Solar Nation’, in no small part to reduce the national import bill. Achieving this could accelerate the Make in India campaign, because national energy-independence will mean the cheapest energy possible.

Once India reaches national energy independence, another equally fundamental development and national health challenge that can be addressed, is the stark water distress. Cheap energy will allow unlimited pumping of water across immense distances, enabling this demon to finally be dispelled. Nascent water desalination technologies - which already exist, but are hamstrung by huge scaling costs - will then take off along the coast. This will enable pumping of clean water to the northern inlands, and onwards to other countries in similar need.

Technologies such as distributed energy generation and storage will take some pressure off the future central electricity grid - for example by reducing the load on transmission infrastructure - there can be no such ‘distributed desalination’ equivalent for water. The oceans are fixed, so plants must remain coastal, or possibly at some suitable inland locations, such as Sambhar lake in Rajasthan. This will mean building significant new pumping infrastructure, with massive energy demands to fully service the country,

Thus, a prerequisite for clean water is vast amounts of cheap energy, and this can only be possible from solar and wind, which are inherently variable, so will increase grid-volatility. Today, even with just a small penetration of renewable generation, the intermittency has caused a significant rise in balancing costs, and in the future this could be huge.

Given this, we must identify solutions with the highest benefit-to-cost ratio across the energy-ecosystem, and software will always be cheaper and less cumbersome than physical buildout. This is where artificial intelligence and machine learning (AI & ML) come into frame and excel - crunching the exponentially growing volumes of data required to balance gird-volatility, forecast future trends, optimise existing assets, and plan future infrastructure. We are already seeing AI & ML starting to handle these issues in the real-world, from forecasting of generation and demand, to scheduling of supply. This bodes well for the future water-energy nexus.

So, there are a multitude of ways we can imagine using an envisaged future energy surplus, but solving clean water access for all, with the help of AI & ML, seems to be the one that will most benefit the country and its people.


Credit to Niladri Roy for co-writing this article.

Thursday, June 28, 2018

Aggregated Weather Forecasts for Large Load Areas


Introduction

When checking the weather forecast in the mornings to see whether you’ll need an umbrella that day, you’ll probably be looking at the forecast for your current point-location, giving data specific to your neighbourhood. If you’re planning a weekend away, the forecast display can easily be changed to another specific point-location. But what if your business need to know what is happening across an area much greater than what can be represented by a single point location? For large areas, obtaining weather forecasts from point-locations is unlikely to suffice and may lead to strategic errors for the business, and ultimately, financial losses.

One example of where aggregated weather forecasts are required is in the energy industry. Electricity demand (load) is strongly dependent on the weather conditions, and so weather data is a vital component of electricity load forecasting. Aggregate-level load forecasts must consider the weather conditions across the entire area, as opposed to using just one location to represent the weather conditions everywhere. This article aims to address the challenges of weather location selection and proposes the best methodologies for determining how many, and which, weather locations should be used when forecasting load over large areas.
The Indian state of Uttar Pradesh, India’s most populous state, is taken as an example case study. Uttar Pradesh covers 243,290 km2, equalling the size of the United Kingdom. Weather therefore varies significantly across the state – there are higher altitude areas in the north, and more rainfall occurs in the north and eastern areas. As a result, the impact of weather on the state’s load will also vary significantly across the area.
Uttar Pradesh - India’s most populous stare.

In particular, localised extreme weather events can be hugely problematic. For example, there could be heavy rainfall in Agra (west Uttar Pradesh), but sunny skies in Lucknow (central Uttar Pradesh). If the weather data is only received for Lucknow, the forecasting system would have no knowledge of the rainfall elsewhere.
(Include a relevant radar image once available)
The geography and climatology varies across Uttar Pradesh.

Methodology

In order to give an aggregated state load forecast, a virtual weather station must be formed, which best represents the weather across the entire state.
Initially, the weather forecast locations you do choose must have data availability, reliability and accuracy. All weather stations that do not, must be ignored. Let’s assume that we begin with 20 weather station locations across Uttar Pradesh that have reliable and accurate data, and provide a good geographical spread across the state.
The virtual weather station can then be built by using multiple point-locations. But how many point locations should be used? Too little will not represent the area well enough, but too many point locations may actually increase the forecast error and increases the amount of data, which has its own issues.
Once you’ve decided how many point locations to use, how do you decide which point locations should be used? Do you choose a roughly even spread of locations across the entire area? Or do you choose more in the west because population is higher there? Ultimately, you want to represent the impact of weather on load, not just the average weather of the state. This will vary with population density, economic and demographic diversity, as well as the end-use diversity of the residential consumers.
Another challenge is that weather forecasts are rarely perfect. Inaccuracies are lower the closer in time we are to the forecast time, but will often still contain significant errors. To rectify this, real-time observational data can be used to compare the forecasted values with the actual observed values and adjust the weather forecasts accordingly. This is particularly effective during days of extreme weather, because the forecasting system is able to quickly react to sudden weather changes.

Methods for aggregated weather forecasts


a)      Simple Averaging

The first method simply takes an average of every forecast variable, from all 20 weather locations. Although this is a simple method, it can be very effective at capturing the impact of weather on load across a large area, as shown by multiple studies (Lloyd 2014). However, there are of course issues to using this simple average method. Firstly, averaging may actually reduce accuracy - many studies have shown that sometimes you can use too many weather locations and cause unwanted smoothing of the data. Additionally, taking the average fails to deal with:
·         the effect of geographical diversity – differing weather across the state;
·         demographic diversity – differing economic and classes across the state;
·         end-use diversity – differing efficiencies of appliances, difference uses of electricity, etc.
Also, why 20 weather locations? Why not 5 or 10, or 100? This number has been heuristically (randomly) picked and is unlikely to be the optimal number of weather locations to use.

b)      Best-fit to Load

In this second method, we progress on the simple averaging technique, by building a load model for every weather location (20 in this case). The accuracy of each model is then tested against actual load and the individual models are ranked in order of their accuracies. Next, the weather data from the top 5 best-fit models are combined, perhaps through simple averaging. This method only uses weather locations that have a strong impact on load, and so reduces the chance of errors creeping in from unimportant weather locations (Charlton and Singleton, 2014). But how do we know if these 5 models have captured the effect of geographical diversity, economic diversity or end-use diversity? And why the 5 best models and not the top 3, top 7 or top 15 models? Again, these numbers have just been heuristically selected.

c)      Zonal best-fit to Load

The third method is very similar to one above, but this time we select the best model for each ‘zone’ (Nedellec et al., 2014). Zones are defined by the operators, covering a small, defined territory within the greater load area. Again, a separate load model is built, one per weather station location. The accuracy of each model is then tested against the actual load. Next, within each zone, the weather station that achieved the highest load accuracy is used. All other weather stations within that zone are ignored. Finally, the average is taken of all the zonal weather stations.
This deals with the geographical diversity problem – all zones are represented. But as a result, it may neglect the issue of capturing the population, economic or end-use diversities. And again, the number of zones and the initial number of weather locations is randomly decided.

The state can be split into zones.

d)      Ranking and Rating

All three of the above methods have heuristically selected the number of weather stations to use; they do not answer the question “How many weather locations should be used?” The Ranking and Rating method attempts to answer this question (Hong & Wang et al., 2015).
A load model is built for each individual weather location. These loads are then compared against the actual aggregate load and ranked in order of their load error, in terms of the mean average percentage error (MAPE) or root mean squared error (RMSE). Obviously, the highest ranked model contains the weather station that has the highest impact on the load. The next step then combines the weather data from x number of the highest performing load models, where x is a number ranging from 2 to the total number of locations; e.g. for x = 2, 3, 4, 5, etc. Again, we fit the virtual weather station of each combination scenario with load data, and arrange in ascending order of error (MAPE or RMSE). The virtual weather station that produces the lowest error is therefore the combination of weather stations locations to use.

Combining weather stations – weighted means

How should the weather locations be combined? A simple average may be effective, but it may be more effective to apply a weighted mean. This weighting could be by population density, giving greater influence to weather in areas with higher populations. This is often used in developed countries, where population correlates well with local electricity demand, but this may not be the case in developing countries. For example in India, highly populated areas tend to be poorer areas with less energy-use per capita, and so population density does not correlate well with local electricity demand. Therefore, a different type of weighting should be applied, such as the economic density of the area. This may correlate better than population density for developing countries, because economically powerful areas tends to be co-located with areas of high residential demand.
The population density of Uttar Pradesh, India.

An important point to make here is that weather insensitive loads should be taken out of the equation. Weather-insensitive loads are sources of load that are not affected by the weather. These are mostly from industrial sources.
Weather insensitive load centres, like this car factory, must be ignored when applying the Rating and Ranking method.

Conclusions

There are multiple methods for producing an aggregated weather forecast, applicable to electricity demand forecasting and management for large load areas. The best of these is the Rating and Ranking method, which ranks the weather station locations by their load error when inputted into a single-station load model. This answers the question “what weather station locations should be used?” Next, the question of “how many weather station locations should be used?” is answered by combining the weather data from x number of the highest performing load models. The virtual weather station that produced the lowest error is therefore the combination of weather stations locations to use.
These weather stations can be combined using a simple average or a weighted mean. It is suggested that a weighted mean by population density is suitable in developed countries, but a weighted mean by economic density is more suitable in developing countries.

References

Charlton, N., & Singleton, C. (2014). A refined parametric model for short term load forecasting. International Journal of Forecasting, 30(2), 364–368.
Hong, T., Wang, P. and White, L., 2015. Weather station selection for electric load forecasting. International Journal of Forecasting, 31(2), pp.286-295.
Lloyd, J. R. (2014). GEFCom2012 hierarchical load forecasting: Gradient boosting machines and Gaussian processes. International Journal of Forecasting, 30(2), 369–374.
Nedellec, R., Cugliari, J., & Goude, Y. (2014). GEFCom2012: Electric load forecasting and backcasting with semi-parametric models. International Journal of Forecasting, 30(2), 440–446.

Sunday, May 6, 2018

How Machines Can Learn to ‘See’ the Weather and Enhance Future Energy Management



Sight or Intuition?

How do we understand the world around us? That is a surprisingly complex question. Let’s try to explain using a real-life example. In the past, whilst enjoying a merry late night meander back from the bar, you may have experienced being barked at, or even chased by a dog.

It was obviously a dog, but did you know what kind of dog? German Shepherd, Labrador, Beagle? In that situation it didn’t matter, it was just a ‘dog’, a barking one with sharp teeth.

But how was that understood? For humans, there is a degree of intuition. Aside from the barking and teeth, it is because we all share a common abstract image of 'dog'. The internal image is likely to vary from person to person. If you grew up with a Golden Retriever, your internal ‘dog’ will look more like that, with some positive emotional attachment. And if you grew up with a Rottweiler snarling at you, your image will be one with scary teeth, and some frightful memories.

Perhaps surprisingly, this is not so dissimilar to how a visual processing AI 'sees' the world.
A Machine's Brain

Having established how humans understand the world in an abstract sense - by having representations inside our brains - enables us to start conceptually understanding how a machine ‘sees’. It obtains its own internal image of reality, which is in a sense the mission of Machine Learning (ML), to obtain a representation of whatever we want it to learn, but in a systematic way. We can encourage machines to form their own internal image of reality using ML, by giving them a mathematical foundation from which they can construct these.

So if we have a picture of a dog, how can we enable a machine to learn how to see that dog?
Figure 1: Dog - a friendly, drooling one (Image credit: Wikimedia Commons)
Figure 1 shows just one dog. But there are hundreds of different breeds, and a machine cannot obtain a general image of ‘dog’ from just looking at a single example. Figure 2 shows several dogs, and we could assemble thousands. Each a different type and in different poses.
Figure 2: Selection of different dogs (Image credit: Wikimedia Commons)
They are simple enough to differentiate. But, now squint your eyes, and you’ll see something like Figure 3.
Figure 3: Blurred dogs - but still identifiable by a human (Image credit: Wikimedia Commons)
As a human, you can probably still tell them apart, but don’t those dogs now look at least slightly similar to each other? Certainly more so than the clear pictures (Figure 2). All we did was decrease the resolution of the image (known as down-sampling). More precisely, we picked a group of pixels and took their average (Figure 4), then repeated for all areas.
Figure 4: The operation of blurring (Image credit: Wikimedia Commons and Sho Akama Images)
Now imagine if we apply the same process to all the picture several times. They will look increasingly similar. We might then only be able to obtain abstract images of ‘dog’, containing some distinct features, like the shape of its head and body, its saucer bowl eyes, wet button nose, drooling tongue, floppy ears, and maybe sharp white teeth. But whereas human intuition can still make sense of these incomplete datasets, machines typically cannot, they lack that intuition.

Convolutional Neural Networks

We all see the world through our own personal 'lens'. Elderly people tend to have slightly blurrier eyes than young people in a physical sense. More conceptually, a politician has a better lens to understand parliamentary dynamics. And a physicist has a lens to see the invisible laws of the universe.

But what does the sight lens of a machine look like? Within the ML domain, this lens is called a filter (or kernel). For any computer, an image is a 2-dimensional array with numbers. It cannot recognise a complex object like a dog in one go. It starts from understanding edges and lines, and by stacking different filters, it can eventually ‘see’ a dog. An ML model specifically designed for computer vision is called a Convolutional Neural Network (CNN).

Traditionally the set of filters is handcrafted by specific ML-area experts. For a human face, you may want to consider a filter that can detect the general geometric shape of faces, like the ‘T-line’ between the nose and eyes (Figure 5).
Figure 5: T-Line, on just some random guy (Image credit: Wikimedia Commons)
To identify a chameleon, you need a very different set of filters than to identify a human (maybe less so if they are of the Karma variety). This means every time you want to detect something new, you need a group of experts for R&D. Imagine trying to develop the best filters for a chameleon. What would they be? To detect rounded edges and stack those up to find two big eyeballs? Why not a filter to find their signature long tongue? Now how are those filters different from the filters for a human? Can our filter distinguish the two creatures in figure 6?
Figure 6: Chameleon vs human (......probably) (Image credit: Wikimedia Commons)
So there are a seemingly infinite number of possibilities, and even with years of research, it is just not practicable to obtain a ‘best’ set of filters. This is where CNN enters the frame and excels - a machine capable of learning filters relative to given objects.
Figure 7: How a Convolutional Neural Network recognises an object (Image credit: Wikimedia Commons)
In Figure 7, each blue block is applying a filter, then extracting core information relative to that filter (also known as ‘convolution’). The size of subsequent blocks decreases because each time a filter is applied, the resulting image shrinks.

Each sequential group of blue blocks is called a layer. By exploiting sets and combinations of filters, a CNN can recognise complex shapes, like dogs (or chameleons).

CNN for Weather Prediction and Better Energy Management

So now we have understood CNN and how a machine can ‘see’. But the link between recognition of a dog and Energy management probably still feels very abstract.

Rather than jump to how machines can ‘see’ and interpret weather patterns to make forecasts. Let's also consider how applications to forecasting are very different to simple object recognition. There are some important future possibilities that could provide compelling solutions to energy industry problems, but it is important to first smooth that gap of understanding.

ML-enhanced visual recognition can be used to recognise, and more crucially interpret, specific visual weather patterns, in much the same way that they can learn how to recognise dogs. Then, these machines can combine image interpretation data with other weather data-sets, to perform some of the higher level integrative decision-making tasks that a human meteorologist currently does. This automation will enable a speedier response by renewable energy generation and distribution management systems. To better match energy supply and demand, by predicting changes in weather patterns over specific local areas.

For example, CNN could be applied to imagery from various weather sources (satellite and radar images, forecast model output graphics, etc), to forecast the temperature, humidity and precipitation. Such a weather combiner would greatly help with the operations and management of electricity grid load, which is heavily dependent on the weather. So such forecasts are hugely important for balancing energy supply with demand. The use of meteorological imagery in the energy industry has started to garner more interest, yet the intersection of ML (CNN), Energy, and Meteorology remains under-explored. But with the integration of renewable generation rapidly growing across the globe, the need to focus on this is becoming more pertinent.

Taking weather satellite imagery specifically, CNN can observe and forecast solar irradiation at the surface. As it can ‘see’ everything, any weather data represented in the form of an image can have filters applied. These are both inherently complex and seemingly similar, even to the human eye. Human eyes are attentive to fine detail, in principle all-encompassing, and hugely experienced due to the constant repetition of situations. Nevertheless, humans can and do make mistakes, and one set of eyes can only see what is in front of it. This is where CNN wins over human eyes. If given enough data, a CNN can help interpolate multiple images, around the clock. In an ideal future, perhaps Meteorologists would no longer have to do night-shifts – a CNN-driven forecasting model could do the job instead.

CNN Can Bring Human Intuition to Numerical Weather Prediction Models

Within the field of Meteorology, weather forecasts are generated by Numerical Weather Prediction (NWP) models, which are then interpreted by Meteorologists to accurately communicate a weather forecast for a given location. Due to the complexity of weather, the human interpretation of these outputs is a crucial step – the model cannot be relied upon alone.
Figure 8: Typhoon Mawar - didn't require much interpretation (Image credit: Wikimedia Commons)
For example, an NWP model output may show there to be short, sharp, heavy showers over Buckingham Palace in London tomorrow. But in reality, these mesoscale-sized heavy showers may occur further to the north than the model predicted. Or maybe the convective rain clouds developed slower than forecasted and ended up causing rainfall later in the day and further downstream from Buckingham Palace. Clearly, there’s uncertainty in the NWP forecast. This is where the job of a trained meteorologist comes in.

The meteorologist would hopefully have understood the complete picture of the NWP forecast and made solid interpretations with that understanding. They see that the predicted rain clouds are small in size and so covering only a very small part of London. They also see that other NWP models are showing the rain clouds to cover different locations. With this added layer of understanding, the meteorologist can then communicate a forecast along the lines of: “Short, heavy showers expected to be scattered around central London.”
Figure 9: ‘Not raining’ at Buckingham Palace (Image credit: Wikimedia Commons)
CNN applied to NWP model graphics could replicate the meteorologists’ interpretation. The CNN can be trained to interpret the spatial relationship of components within the output image, and produce an accurate local forecast. This would be achieved by reading every model output graphic and learning their evolution. This can then be matched with historical model graphics, and the observed conditions seen at every time-step and every location. Once the CNN has enough repetition of each scenario, it will have learnt to understand the complete picture of the forecast, just like a human would. Though the above example of Buckingham Palace is light-hearted, there are far more destructive potential cases in much of the world, where weather patterns are highly volatile and severe.

Similarly, CNN visual recognition can be applied to radar imagery. Radars typically scan the atmosphere every 15 minutes and output a value for the reflectivity that’s bounced back to it by water droplets within clouds and rainfall. Currently a meteorologist must use these radar images to decide if and when rainfall may occur, say over an airport. Use of CNN visual recognition could replicate the human interpolation, and eventually avoid equivalent human error.

Conclusion

Given how fundamental weather forecasting is for the Energy sector, especially as we rapidly integrate more renewables to the grid, there is a clear need to develop weather specific CNN. Computer-vision researchers used to hand-craft filters so that machines could see, now a CNN can learn these filters by itself. In similar spirit, a CNN can start to interpret weather images instead of a meteorologist, eventually achieving far greater accuracy and efficiency. Though development is still very nascent, it echoes what is happening across the energy sector, and many other domains. However, though machines can learn 'how' to see, they don’t yet know ‘what’ to see. So for now, machines have only gained sight. To enhance the sector and address our pressing global energy needs, we must help them gain vision.


Wednesday, December 20, 2017

How Weather Forecasting Will Drive Down Electricity Bills in the Renewable Energy Future

Renewable energy sources like wind and solar power are inherently variable due to their dependence on natural phenomena. For renewable systems to be integrated into our energy mix, comprehensive system management must be in place, where energy sources are accurately scheduled ahead of time to ensure electricity demand is always matched by supply. This therefore requires precise forecasting of renewable generation. The dependence of these systems on the weather means that accurate meteorological data is essential as an input into renewables forecast modelling. As renewables increase in the energy mix globally, the need for accurate and valid weather data is becoming increasingly more important.

The Rise of Renewable Energy
The renewables revolution is well under way. Today, renewable energy (excluding hydro) only accounts for 8% of global electricity generation [1], but its current growth is rapid - almost 40% of the additional global power generation in 2016 was from renewables. The fastest rises compared to 1990 levels have been from solar photovoltaic (PV) (45.5%) and wind power (24%) [2], due to national and international policies, and sharply falling costs. Record-low solar PV auction prices below 3 cents per kilowatt-hour have occurred across the globe, including in Argentina, Chile, India, Jordan, Saudi Arabia and the United States. Similar records have been seen for electricity from wind developments, spearheaded in Chile, India, Mexico, Morocco, Denmark and the Netherlands.

The Rise of Electricity Bills
But does lower renewable kWh prices result in cheaper electricity bills for everyday consumers? End-user electricity prices have been on a continuous upward trend in many countries. In the UK, there was a 63% rise in electricity prices between 2004 and 2014 [3]. A similar picture has been seen across Europe and further afield. According to the US Energy Information Administration, US utility residential electricity rates have increased by around 20% in the last 10 years [4].  The most influential driver of this has been the rising price of natural gas, and this trend is expected to continue. For electricity bills to come down, cheaper energy sources must be realised – can renewables provide this?

The Renewable Energy Future
The growth of renewables is expected to continue. The International Energy Agency (IEA) predict that by 2040, the installed capacity of solar and wind will increase by a factor of 14 and 4, respectively. This will result in the share of global electricity generation accounted for by solar and wind rising from 5% today to 34% in 2040 (figure 1) [1].
Figure 1. Global electricity generation mix to 2040. Source: Bloomberg New Energy Finance, New Energy Outlook 2017. 
But as governments remove feed-in-tariffs and other supportive incentives, renewables need to be able to stand on their own feet. The only way this can happen is if they are cheaper than conventional sources. Ever more efficient solar PV modules and wind turbines will reduce costs, but price decreases must also come from more efficient operating and maintenance procedures. Yet due to the uncontrollable variability of wind and solar PV systems, their integration into power systems poses challenges.

Balancing Supply and Demand
Supply must always balance demand. This is achieved by grid operators scheduling power generation in advance to match the expected demand at all times. The limited ability to anticipate variable energy sources on a minute-by-minute basis results in supply and demand deviating from each other in real-time and as a result, grid operators must have additional power plants (usually fossil-based) on continual standby.  These balancing costs can be substantial, and ultimately, greater imbalancing costs lead to higher electricity prices for consumers. Ofgem, the UK’s electricity regulator, notes that balancing costs of around £850,000 per year adds £9 to the average UK electricity bill [5]. Worse still, getting the balance wrong leads to blackouts or power surges, further impacting energy costs.

Power system flexibility is therefore required – the capability to maintain continuous grid balance in the face of rapid and large swings in supply or demand. Without it, wholesale prices of renewable energy become depressed, particularly when wind and solar production exceeds the demand on sunny, windy days.

Renewable Energy Forecasting
To achieve power system flexibility, comprehensive forecasting of renewable systems is essential. Traditional forecasting approaches in the energy sector have mainly been statistical techniques, comprising of time-series methods or regressions. However, with the advances in big data, processing capacity and tools to manipulate these datasets, artificial intelligence-driven modelling is starting to take more precedence. This is due to its flexible nature and ability to handle non-linearity, which is inherent in many energy forecasting problems, especially those required at a granular time-scale. Figure 2 displays some of the inputs for solar generation forecasting using artificial intelligence-driven modelling.
Figure 2. System architecture of solar PV generation forecasting using artificial intelligence-driven modelling.
The Need for Weather Data
The power produced by a solar plant is strongly dependent on the global horizontal irradiance (GHI) – the total amount of shortwave radiation received from above by a horizontal surface, at an instantaneous point in time. GHI is measured by on-site pyranometers and expressed in Watt per square metre (Wm-2). GHI values are then adjusted as per the tilt and orientation of the solar panels – the global tilted irradiance, GTI (figure 3).
Figure 3. Global Horizontal Irradiance (GHI) and Global Tilted Irradiance (GTI).
GHI, and therefore solar PV output, has seasonal and diurnal cycles. GHI also dramatically fluctuates on much shorter timescales due to sharp changes in cloud cover and atmospheric conditions.

The power produced by wind turbines is dependent on the cube of wind speed. Turbines have a lower limit at which no power is generated, and an upper limit at which turbines are shut down to avoid damage and failure. Although less pronounced compared to solar, wind power may fluctuate on seasonal timescales, down to very short-term fluctuations due to localised wind speed changes.

All fluctuations in GHI and wind speed must be captured to high precisions – renewable energy forecasting is only good if the weather data feeding into the forecast models is good.

Different Weather Sources for Different Forecast Lead Time
The best source of weather data for forecasting of renewables depends upon the forecast lead time required, particularly for solar power forecasting (figure 4).
Figure 4. Conceptual plot of forecast lead time vs. forecast skill for different sources of weather data: ground observations, satellite imagery or NWP model output.
At sub-hourly lead times, the highest solar forecast skill is provided by methods based on on-site ground observations, such as from real-time infrared imagery.
But forecasts based on these local observations become less skilful as the forecast lead time increases, which is when satellite-based methods for computing GHI become useful. Geostationary satellite imagery (e.g. from Meteosat-10 over Europe and Africa) allows clouds further afield to be captured, and cloud motion vectors enables the future positions of clouds over the site of interest to be inferred.
Obviously, clouds evolve, so the use of satellite imagery to compute solar radiation is limited to around 7 hours ahead. After this, weather forecasts from numerical weather prediction (NWP) models become most effective for solar forecasting. NWP models incorporate our physical and dynamical understanding of the atmosphere to forecast the future state of weather using mathematical equations and huge computational power.

NWP Forecast Uncertainty
However, NWP forecasts are not perfect. It is impossible to predict with certainty for every point in space and time, but there is still plenty of room for improvement.

The main limit to their accuracy is the spatial resolution of the NWP model. The finer the spatial resolution, the smaller the grid cells that divide up the atmosphere, and therefore the more representative the predicted variables will be to every point in that cell. This itself is limited by computational power, despite the NWP models like the GFS and ECMWF running on the most super of supercomputers. Consequently, there’s a lot of information the models struggle to infinitely resolve. This includes all atmospheric phenomena that are smaller in size than the grid resolution, such as turbulence and small cloud features like cumulus clouds. Surface conditions are also not fully resolvable: is the surface cover vegetation or water or concrete? What is the topography of the surface?

Anything that cannot be resolved are parameterised in the model, which means that they are approximated using over-simplified assumptions. A cumulus cloud, which has an average width of about a 1km, must be parameterised, and hence may be poorly represented by the model. This provides a large source of uncertainty in solar forecasting; solar plant output can dramatically fall within seconds as a cloud moves over and blocks out the Sun’s radiation.

Good observational data is also essential – it’s a case of garbage in, garbage out. Where observational data is not available, interpolation is used to fill the grid with initial conditions. But any errors in the initial observational data will result in errors in the interpolated data too, and this may lead to large uncertainties creeping in. Chaos theory (the butterfly effect) means that the forecast output is highly sensitive to the initial and boundary conditions and so may have very different outcomes with just slight differences to the model inputs. Chaos will increase with time, so forecasts become increasingly more uncertain the further out they are made for.

High model generation frequency (how often the model runs) helps to eliminate this issue by re-running the model with the latest observational data. For example, the GFS runs every 6 hours, whilst the Met Office’s UKV model runs every 3 hours. As with the spatial resolution, model run frequency is limited by computer power.

Reducing Overall Costs
The savings due to good energy forecasting can be substantial - five-fold reductions in the costs of hourly imbalance penalties levied on generation have been seen in the past. The UK's Central Electricity Generating Board claimed that if perfect wind power prediction was possible, it would add 10% to the value of wind power on the system. 

Conclusion
With an increasing share of wind and solar in the generation mix across the globe, the overall generation variability is set to rise, and hence the challenges of balancing supply and demand will become greater. But while wind and solar power cannot be dispatched, they are schedulable, with the support of accurate weather forecasting. Ground instruments, satellite imagery and NWP models are all essential when forecasting renewable generation in real-time. As accurate forecasting of renewables becomes more prominent, renewable energy prices will fall, and this should lead to lower electricity bills for consumers.

References
[1]
Bloomberg New Energy Finance, “New Energy Outlook,” 2017.
[2]
International Energy Agency, “Renewables information 2017,” 2017.
[3]
Ofgem, “Annual Report 2015-2016,” 2016.
[4]
U.S. Energy Information Administration, “Annual Energy Outlook,” 2017.
[5]
Ofgem, “Balancing and Settlement Code (BSC) P333,” 2016.