Demand in the electricity market: analysis using big data

Abstract

The traditional business model of energy companies is changing in recent years. The introduction of smart meters has led to an exponential increase in the volume of data available, and their analysis can help find consumption patterns among electric customers to reduce costs and protect the environment. Power plants generate electricity to cover peak consumption at specific times. A set of techniques called “demand response” tries to solve this problem using artificial intelligence proposals. This document proposes a method for processing large volumes of data such as those generated by smart meters. Both for the preprocessing and for the optimization and realization of this analysis big data techniques are used. Specifically, a distributed version of the k-means algorithm and several indices of internal validation of clustering for big data in Spark. The source data correspond to the consumption of electric customers in Bogota, Colombia during the year 2018. The analysis carried out in this study about consumers helps their characterization. This greater knowledge about consumer habits and types of customers can enhance the work of utilities.