Machine Learning on 50 Million Smart Meters: Utility Powerhouse Extends C3 Platform Europe-wide 

source: shutterstock 60066028

In enterprise AI, C3 (formerly C3 IoT) is amassing an impressive and seemingly unmatched record, one that the company has extended with its latest win, the expansion of a five-year engagement with Enel, Europe’s largest power utility, to encompass nearly 50 million smart meters in homes and businesses.

This follows C3 contract wins last year with Royal Dutch Shell, the U.S. Air Force and 3M, along with partnerships with AWS, Google Cloud and Microsoft Azure. In the large utilities space, other customers include Con Edison, covering the New York metropolitan area, and Engie, one of the biggest utilities in France.

The new contract (dollar amount not disclosed) expands on C3’s existing, five-year engagement for Enel in Italy involving 32 million smart meters. C3 will provide the €74.6 billion utility with AI and smart grid analytics applications enabling Enel to deploy the Unified Virtual Data Lake, integrating data across its retail, distribution, trading, renewables and conventional generation businesses. The C3 AI Suite automates elements of data management, data science and AI application building, enabling the entirety of corporate data, regardless of data format or system – such as ERP, HR, financial and operational systems, including SAP Hana, Oracle, Siemens, PostGreSQL, MongoDB, and Cloudera – “to enable and deliver next-generation AI applications across Enel’s business,” C3 said.

“Leveraging the power of AI and IoT is key for Enel to achieve its digital transformation, while allowing tremendous benefits to be delivered to its customers and shareholders,” said Fabio Veronese, head of infrastructure and networks digital hub at Enel. “The collaboration with C3 is allowing us to harness innovative business processes enabled by big data analytics, unleashing a new era of operational efficiencies that strengthen our position as leader of the energy transition.”

Enel is one of the world’s largest utilities, operating in 45 countries with more than 2 million kilometers of power grid. The company runs the largest transmission network outside of China and has 43 gigawatts of renewable energy capacity.

“They’ve made big investments in digitalization: smart meters and sensors across their networks for full visibility to their generation capacity, their transmission of power, on a real time/near time basis,” said Ed Abbo, C3’s president and CTO.

Clearly, the results of C3’s “PoC” in Italy convinced Enel to expand C3 implementation Europe-wide. So the question becomes: how does C3 (the company and its AI Suite) do it?

Abbo told EnterpriseTech the C3 AI Suite incorporates three critical capabilities needed for enterprise-scale AI:

-          data management – ingestion, aggregation and cleansing of data from the hundreds, even thousands, of systems across the enterprise to derive a holistic view of company operations and customer behavior;

-          rapid testing and deployment of pre-built AI models (machine and deep learning, natural language processing, etc.) that accelerate the work of data scientists;

-          scalability, using public and private clouds and distributed computing techniques, spinning up and shutting down cloud resources as machine learning inferencing is needed on fluctuating volumes of incoming sensor data.

“Data science (automation) is one element and data management is another,” said Abbo. “C3 tackles both, and the third dimension is scaling the life cycle of these AI models, and unless you’ve got all three elements you’re still in the prototype world.”

That point about scale is key to C3 ability to tackle large engagements: the company has long claimed, credibly, that it has more sensors under management than anyone else. In the case of Enel, now C3’s largest engagement, there’s an estimated 100 million sensors attached to the utility’s 50 million smart meters in Europe transmitting millions of messages per second to algorithms in the C3 platform.

Another key to enterprise AI: implementation speed.

“The reason customers use C3 technology is because they can deploy applications faster with smaller teams,” Abbo said.

The C3 AI Suite, which industry analyst firm Forrester Research last year named the strongest current industrial IoT software platform offering, utilizes a model-driven architecture that represents AI enterprise and IoT application semantics in metadata, speeding up the process by requiring less code to be written, debugged and maintained.

“We benchmarked this, and it’s several orders of magnitude faster for less code and faster to actually design, develop and deploy AI and IoT applications than building it natively on cloud platforms,” Abbo said. “This is revolutionary in what it can do and entirely complementary with private and public cloud platforms, it leverages the native services but it’s a much higher productivity approach.”

Anything that simplifies this process is, of course, all to the good.

“The complexity in developing one of these systems is mind boggling,” said Abbo, “it requires writing a lot of code that integrates a plethora of platform software services, data access from a variety of sources, transformation of data, correlation of data, data cleansing, data aggregation, and then invoking machine learning services, authentication, authorization, encryption of data in motion and at rest, stream processing, batch processing, micro batch processing, persistence in relational stores, persistence in key value stores, persistence in network stores – there are about 100 different components that need to be stitched together using programming languages.”

Plethora, indeed.

The objective is to run a system that delivers a comprehensive, detailed, real/near time picture of the state of the entire company. This means aggregating and cleansing within a data lake an enormous, constantly growing streaming flow of multi-sourced data, and then contextualizing it.

“For most companies, their data is fragmented across thousands of systems,” Abbo explained. “The first piece is integrating and aggregating data from numerous systems so you can understand the state of your business. Then you have to keep that state current as new data comes in from sensors, from the ERP systems, and so forth. You’re keeping that information current and then triggering logic, like rules or machine learning inferences, to assist in decision making. Then those decisions are made, actions are taken, the information changes, so it’s a continual closed loop machine learning system.”

Enel’s objectives are multifold, including maximized usage of renewables and minimized overall energy consumption. In Italy, the C3 system aggregates readings of the country’s 32 million smart meters every 15 minutes, data for power consumption, temperature, voltage, along with notifications of equipment failures along Enel’s distribution circuit so that power can be re-routed.

“Bottom line: between digital smart meters and all their data from their supervisory control systems and fault path indicators and generators, the rate at which you need to process data is in excess of millions of messages per second,” Abbo said. “The initial projects were to process those data, which means not just collecting data at a very rapid rate but, importantly, to contextualize the data, which means we’re accessing information from their other systems. This might be historical billing information, it might be maintenance information, outage information on their circuit stem to the customer.  So we’re basically collecting the data at the rate at which it’s being generated and transmitted, contextualizing it by looking up the customer and other items, and then applying machine learning to those data for economic benefit.”

By “benefit” Abbo refers to operational and workforce efficiencies, along with better customer engagement. These include fraud detection and power theft (which saps an average of 3.5 percent of utility revenues, industry-wide), improvement of network reliability, identifying equipment likely to fail within the 30 days and better understanding how customers consume power, making recommendations for renewables, “all those ways you might assist the customer make smarter use of energy.”

In sum, Enel projects more than €2b in economic benefit from the C3 system over the next three years.

As would be expected given the multiple sources of energy in use and the drive to maximize renewables, better analysis of energy supply and demand is a critical success factor for Enel. This is in contrast to years past, when it was reasonably straightforward to forecast supply/demand intra-day power dispatch. “You could forecast the next day, line up your power supply from suppliers and then dispatch power to meet demand,” Abbo said.

But now, with the transitions to the smart grid, “you have factors that are less predictable – you have renewables where customers are putting solar on their rooftops, there’s battery storage of energy and the supply of wind and solar renewables is increasing, so now you’re transitioning to where demand is less certain because you have cloud(y weather) and wind and battery storage and other relevancies. It’s difficult to know how much power you’ll need at any given moment.”

C3 uses machine learning at scale to forecast the demand at every single customer point, intra-day, then aggregates that demand to forecast supply requirements intra-day, taking into account potential buying or selling power.

“The power grid is the most complex system ever built and it’s now becoming an order of magnitude more complex with the transition to using more renewables everywhere,” Abbo said. “It’s a fascinating use of machine learning to basically characterize the uncertainly in demand and supply.”