Intro

As you may know TenneT BV, the TSO responsible for the Dutch Power grid, is moving away from EDINE message formats for market communication. Instead they are implementing web services to gradually replace all these EDI-based messages with their XML equivalents.

Energy One is rolling out support for the Dutch power grid and as such we are fully on board with this transition and implementing the required changes in our Market Communication ‘engine’. In this blog we want to give you a short update on what is changing and why; and of course you can count on us to help you out in case you’re active on the Dutch energy market.

What’s changing at TenneT?

The move away from EDINE to XML also entails switching over from the CPS (Central Postbus System) which is a secure mail system for exchanging the EDINE messages, to the MMC-Hub, which stands for Market-Market-Communication Hub. This HUB facilitates the message exchange between market participants and TenneT and between market participants amongst themselves. 

In 2017 TenneT started the preparation for this transition & eventually all EDINE messaging will be replaced by their XML equivalents. You can see the roadmap of ongoing projects in the screenshot below. There is a small error, with Nominations currently being deployed in 2021 (and not 2020 as shown on the slide).

Since this blog only summarises this process we refer everyone to a workshop from August 2021: https://www.youtube.com/watch?v=fCBM1SelBz8

So, what are web services & SOAP?

In short, a web service is a specialized data-exchange solution, and in this case the TenneT web services will be using the XML message format to exchange these data. On top of that they use SOAP (Simple Object Access Protocol) as a messaging protocol, which was designed for exchanging structured information across web services. See https://en.wikipedia.org/wiki/SOAP if you’d like to read more.

Basically, SOAP allows separate platforms to communicate using XML, over the internet (hence ‘web services’) without the need for cumbersome middleware/‘translators’ or integrations. SOAP has three major characteristics:

1. extensibility (security and WS-Addressing are among the extensions under development)
2. neutrality (SOAP can operate over any protocol such as HTTP(S), SMTP, TCP, UDP)
3. independence (SOAP allows for any programming model)

Application at TenneT

TenneT’s web services will send and receive messages over the internet, using SOAP over HTTPS. There will be ‘synchronous’ communication, where the web-service returns a response over an ‘open connection’ within a reasonable timeframe. In practice these timeframes are very short and almost instantaneous if everything is running as should be. An example is: sending a ‘nomination’ and quite quickly afterwards receiving an ‘acknowledgement’ from TenneT that the nomination has been received.

However there will be a 2^nd check needed that takes a bit longer, for example if the nominated amounts are not matching the reserved capacity. To do this the IT systems at TenneT have to check with other databases and cannot return a message ‘almost instantaneously’. Only after the checks have been done in the IT systems can the TSO reply. This is an example of asynchronous communication.

TenneT will set-up a range of synchronous and asynchronous web-services to handle all possible situations. We won’t delve much deeper into technical details, but all information can be received upon contacting TenneT.

Security of data exchange

TenneT will use commonplace and proven solutions such as encryption and certificates to ensure your data is safe and cannot be accessed or transformed by other users/parties. TenneT requires two way TLS1.2 security standards when communicating with market parties. The XML messages also need to contain information about the certificate used for signing the message.

How can we help?

Our business analysts and IT team are implementing these web-services in our ‘market communication engine’. We can provide you with the needed information about the required certificates, XML message formats, and much more. As a trusted service provider we are happy to help you have a smooth transition from EDI to the wonderful world of web-services.

On top of that adopting these modern web services opens up a wide range of possibilities for data exchange and IT optimization that could benefit all market participants on the Dutch power grid.

A short history of EDI

The beginning of EDI

To understand why TenneT and many other TSOs are moving away from EDIFACT-based message formats we have to take a small historical detour. Let’s start with the question raised in the 1960s-70s when the first computers appeared: If two systems want to communicate electronically, how do you manage this? To answer this question several international standards for Electronic Data Interchange (EDI) were born: https://en.wikipedia.org/wiki/Electronic_data_interchange

In short, EDI can be defined as the transfer of structured data, by agreed message standards, from one computer system to another without human intervention.

One widely adopted international standard for EDI is UN/EDIFACT (https://en.wikipedia.org/wiki/EDIFACT), providing:

• a set of syntax rules governing how to structure your data
• an interactive exchange protocol
• standard messages which allow multi-country and multi-industry exchange

Market regulators & TSOs across Europe adopted these formats. The Netherlands also adopted the EDIFACT standard and implemented the EDINE-format to exchange information messages. EDINE = EDI in de Nederlandse Energiemarkt – https://nl.wikipedia.org/wiki/EDINE
More info on EDINE https://www.tennet.eu/electricity-market/rules-and-procedures/edine-converter-nl/

EDI-based message formats and standards are used by over 300.000 companies and organisations worldwide to exchange data.

Some drawbacks appear over time

EDIFACT, ANSII X12, and other traditional EDI message formats have been designed with a focus on robustness and minimising ‘data footprint’. This makes sense as they were developed since the 1970-80s, when data flows had to be kept minimal because bandwidth and storage capacity were limited.

However, some drawbacks appeared over time. There are many articles explaining these in depth, but here’s a short summary:

• traditional EDI-standards require very specialised technical knowledge to implement & maintain as they have grown in complexity over time
• some modern software solutions no longer ‘recognize’ older EDIFACT-based messages, so you need ‘middleware’ = IT solutions acting as ‘translators’
• EDIFACT messages are hard to interpret for non-EDI specialised IT professionals and end-users. These messages are ‘machine-readable’ but not really ‘human-readable’ without the proper training and knowledge.

1998 – XML enters the scene

With the advent of the internet and the booming digital age of the 1990-2000s many of the original constraints that limited data exchange disappeared. Bandwidth, processing power, and storage capacity are increasing year over year.

During this boom a new open standard & message format was born in 1998: XML or eXstensible Markup Language (https://en.wikipedia.org/wiki/XML). The Markup language outlines the set of rules for encoding documents in a format that’s understandable and readable by both machines and humans. XML uses the internet as a means of communication.

XML as a format for data exchange has seen rapidly increasing adoption the past 23 years. Web-services using XML data formats are commonplace nowadays. As such they are also being adopted within the European Energy markets by TSOs and governing bodies such as ENTSO-E (More on CIM here: https://www.entsoe.eu/data/cim/).

Some key benefits of XML are:

• open-source: a range of low-cost tools and solutions are readily available, as well as training courses and knowledge bases
• scalability: allows for complex communication set-ups connecting thousands of parties, since it’s using internet technology there are practically no limits as to the amount of users & messages exchanged – as long as you account for adequate processing & storage capacity
• easy to adopt: XML uses human, not computer, language. XML is readable and understandable, even by novices, and no more difficult to code than HTML
• platform independent: XML is platform independent and programming language independent, thus it can be used on any system and supports the technology change when that happens

Much more can be found here https://www.javatpoint.com/features-and-advantages-of-xml & https://beginnersbook.com/2018/10/advantages-and-disadvantages-of-xml/