When you are on a mission to change the way that an entire country powers itself, innovation is the engine driving your business. That describes Vandebron, an online energy platform that allows consumers to pay for energy in a smarter way. 

As Dr. Giuseppe Procaccianti, Head of Big Data at Vandebron, explains, Vandebron can realize enormous value by accurately predicting how much renewable energy its partners will produce. However, by its nature, renewable energy is unpredictable. While this is a large-scale problem for a grid operator, it’s still problematic for a company like Vandebron because an energy imbalance means the company incurs costs.

 

For this reason, Vandebron decided to build an in-house wind forecasting pipeline that predicts the anticipated energy output 24 hours in advance since the Dutch energy market works on a day-ahead basis. The goal was to generate an energy plan for each wind producer showing how much they would produce every hour. To build the pipeline, Vandebron would need to ingest and process a few gigabytes of data every six hours. Ingesting and processing this much data in a short time frame meant Vandebron needed a high-performance distributed system to analyze and take action upon huge datasets.

With D2iQ, Vandebron went live in such a short time because of the packages available through the D2iQ Service Catalog, such as Apache NiFi and Jupyter which can quickly write Python code, test models, and explore data. “These are valuable tools enabling data scientists to move data, perform queries, quickly conduct analysis, and prepare reports. D2iQ gave us the right tools so our team could be productive quickly,” he continues.

Since going live with its wind forecasting pipeline, Vandebron has been incredibly happy with the results. “By leveraging the full power of D2iQ’s robust model processing technology, and optimized code, we are able to process huge datasets in just 3-4 minutes.” explains Procaccianti. 

Also thanks to D2iQ, Vandebron’s deployment process is easy and fast. “We’ve significantly shortened time to market by being able to quickly deploy new services to production. Before it would take weeks to install and run a service like Elastic Search or Spark. Now we’re operational in hours because we don’t have to take time reading documentation and manuals,” says Procaccianti.