Open

Location

This example model can be found in the software under the category Cyber-Physical with the file name Cyber_Physical_Microgrid.ecf.

Description

This example is a demonstration of a cyber-physical simulation involving a microgrid subjected to cyber-attacks. An OPAL-RT Real-Time Simulator co-simulates a microgrid system, including its distributed energy resources (DERs), power converters, and loads modeled in HYPERSIM, and the underlying communication network in EXata CPS. The microgrid is a 25-kV medium voltage distribution network connected to a 120 kV sub-transmission system by means of a 15 MVA delta-wye transformer. The microgrid consists of 6 DERs, including two 1.5 MVA PV systems, three energy storage systems (ESS) (two of which are 500 kVA ESSs with 1.15 MWh capacity; one is a 1 MVA ESS with 5 MWh capacity), and one 10 MVA CHP unit.

The microgrid also has four aggregated loads: the first two are critical loads rated at 4 MW, the third is a 4 MW priority load, and the fourth is a 5 MW non-critical load. The microgrid model does not contain relay elements, and therefore there is no underlying protection scheme. A subsystem measurement component is programmed that produces P, Q, and V_rms measurements based on voltage and current measurements. Some details on the V_rms measurement are explained in the Annex. There is a measurement subsystem associated with each microgrid asset.

A simple rule-based microgrid controller is simulated on the same model. The controller receives measurements from the measurement subsystems. Using these measurements, the controller is programmed to send reference setpoints to some of the DERs in islanded mode for power balance, to shed loads, and to enable voltage support. However, the controller does not have a reconnect function. Users can connect an external controller and use it instead of the built-in internal controller. If an external controller needs to be connected, the user should also update the control devices at the inputs of the DERs and replace or retrofit them according to’s needs.

The microgrid DERs and the microgrid central controller (MGCC) are connected via IEC 61850 GOOSE messages to transmit measurements and commands. A list of the measurements and setpoints exchanged between the IEDs and the MGCC is in the Annex. The detailed GOOSE message parameters are also found in the Annex.

The figure below shows an overview of the microgrid controller schematic as designed in HYPERSIM.

Open

An optional LabVIEW panel is built for the example model and can be used for demonstration purposes. The LabVIEW panel runs on the host PC and uses TCP/IP to communicate with the simulator. The simulator acts as a server, and the host PC as a client. The LabVIEW front panel details the messages sent and received by the server, which are summarized in the Annex. 

In addition, a high-fidelity communication network emulator, SCALABLE Network’s EXata CPS software, simulates the underlying communication fabric of electrical grids and can test the cyber resilience of such systems. Using EXata CPS, command and measurement signals between the controller and microgrid can be subjected to cyber attacks, causing unexpected behavior within the system. The figure below shows a screenshot of the EXata CPS model schematic.

As can be seen, the IEDs are connected to the microgrid controller represented as node 1 through a common network switch, whereas each DER and load has an associated IED.

Setup

Setup Procedure

Please refer to EXata CPS | Installation and Licenses for a detailed guide on EXata CPS installation and licensing on the target and the host PC.

Setup Prerequisites

Hardware:

• OPAL-RT Simulator with at least 4 cores.

• 64-bit Windows-based host PC.

Software:

• Simulator:

  • OPAL-RTLinux 64-bit.

  • EXata CPS, including license for at least 12 communication nodes.

  • License for IEC 61850 GOOSE.

• Host PC:

  • If using HYPERSIM 2021.3 or later, EXata CPS GUI 1.1 or later version, including license for 12 communication nodes. If using HYPERSIM 2021.2 or earlier, EXata CPS GUI 1.0, including license for 12 communication nodes.

  • Latest version of MobaXterm, or an equivalent software to connect to the simulator using SSH.

  • (Optional) LabVIEW Run-Time Engine 2015 or later.

    • If use of the LabVIEW panel is desired, the download link is:
      https://www.ni.com/en-us/support/downloads/software-products/download.labview.html#306249
      
      

Simulation Procedure

HYPERSIM Simulation Workflow

• Open the HYPERSIM model, and specify the target in the simulation settings.

• In the simulation settings, advanced target settings, if the compiler is specified as opicc, please switch the compiler from opicc to opgcc. 

For help, consult the following KnowledgeBase article: https://www.opal-rt.com/support-knowledge-base/?article=AA-01265

• Starting with 2021.3 and EXataCPS 1.1, an EXata Network Interface must be specified. In order to make sure that your setup matches with the example model IO interface settings, open the IO interface; Then, in the EXata-CPS Configurator, double-check the EXata Network Interface name and make sure it matches the name of your simulators network interface which will be dedicated to EXata.

For help, you may connect to your simulator via MobaXterm (follow this KB: https://www.opal-rt.com/support-knowledge-base/?article=AA-01044) and then type-in "ifconfig". Typical names for a network interface on OPAL-RTLinux are "eno1" and "eno2".

• Start the simulation on HYPERSIM.

• Follow the EXata CPS Simulation Workflow EXata CPS Simulation Workflow.

• Follow the LabVIEW Panel Workflow (optional).

• Open ScopeView, then open the template “Cyber_Physical_Microgrid.svt”.

• If the model is running correctly, the user observes one or more of the events below occurring:

  • EXata CPS IEDs show that they are communicating as shown in the following screenshot (it may take a minute for the arrows to show):

1.  

   • ScopeView sent and received signals as part of the scenarios results should match. Specifically, in scenario 1, “Load 4 tripping command”, and in scenario 2, “Load 2 Measurement Packets”.

   • If the LabVIEW panel is used, the user should be able to see real-time updates in active power measurements at the DERs, voltage and frequency measurements at the PCC, and be able to send commands (Islanding, V Support, Inductive Load Fault) and see the corresponding outcome.

EXata CPS Simulation Workflow

• If this is the first time EXata CPS is run, or it is not yet configured or installed, refer to the HYPERSIM User Documentation > EXata CPS > EXata CPS | Installation and Licenses

• Depending on whether you have EXata CPS 1.1 or EXata CPS 8.1.3 installed on your windows host machine and on the simulation target you are using, you can follow these commands:

EXata CPS 1.1

• Open the EXata CPS 1.1 GUI as administrator.

• In EXata CPS, open the Cyber_Physical_Microgrid.config configuration file from the example model directory (...\Documents\HYPERSIM\Cyber_Physical_Microgrid\Cyber_Physical_Microgrid_hyp\EXataCPS_1_1\)

• Before running the example model for the first time, we must establish the communication between HYPERSIM and EXata CPS.

• You should see the following model:

Please refer to the HYPERSIM User Documentation > EXata CPS > EXata CPS | Communication Interface with HYPERSIM.

• Click Run Emulation on EXata CPS by clicking the run emulation button here: 

• Click Play. The Play button should be blue, as below:

EXata CPS 8.1.3

• Open the EXata CPS 8.1.3 GUI as administrator.

• In EXata CPS, open the Cyber_Physical_Microgrid.config configuration file from the example model directory (...\Documents\HYPERSIM\Cyber_Physical_Microgrid\Cyber_Physical_Microgrid_hyp\EXataCPS\)

• You should see the following model:

Please refer to the HYPERSIM User Documentation > EXata CPS > EXata CPS | Communication Interface with HYPERSIM.

• Click Run Emulation on EXata CPS by clicking the run emulation button here:

• Click Play. The Play button should be blue, as below:

LabVIEW Panel Workflow (Optional)

To use the associated LabVIEW panel:

• First, open the associated .exe file, and insert the IP address of the target here:

• Enable the LabVIEW_Enable as specified in the scenarios section. This lets HYPERSIM accept commands received from the panel.

Important Notes

• If the LabVIEW panel is not updating its measurements during execution, close LabVIEW and restart HYPERSIM.

• Since virtual Ethernet ports are specified (their default settings are specified in the Annex: Communication IO Configuration), EXataCPS must be running in order to make the microgrid controller functional. Otherwise, the microgrid controller is not able to send or receive any data. If the user wants to run the microgrid without EXataCPS and still use IEC61850, they can change all the virtual Ethernet port values to a non-virtual Ethernet port, such as eth1. In this case, it is recommended that eth1 is connected to an isolated local area network.

Simulation and Results

Scenario Introduction

This section summarizes the scenarios that can be played in this demo. Parameter En_S1_S2 needs to be changed to switch between the two scenarios.

This parameter can be found in the HYPERSIM model schematic here:

Note that the usage of the rest of the parameters, LabVIEW_Enable, V_Support, Inductive_Fault, and Internal_MGCC are optional, and are not part of the scenarios presented.

• LabVIEW_Enable allows the user to use the LabVIEW interface.

• V_Support enables voltage support from each ESS by enabling Q-v droop control.

• Inductive_Fault simulates an inductive fault on the microgrid.

• Finally, Internal_MGCC gives the user the option of bypassing the internal microgrid controller, in which case the user would set up the appropriate communications with their external controller.

The testing scenarios are summarized in the following table. The default mode of operation is Scenario 0, where the microgrid is in grid-connected mode and no cyberattacks are happening.

Scenario 1: Microgrid Islanding Without Delay Attack

In the first scenario, the microgrid goes to islanded mode after 1 second. Once the microgrid is islanded, the MGCC sends a shedding command to load 4 in order to balance the generation and load.

The following are the results, which do not include any cyber-attack:

As can be noticed from the figure, the microgrid is islanded at t = 1 second. Almost immediately after islanding, the MGCC sends a tripping command to load 4, as shown in the bottom right figure. The sent and received trip commands are overlapping, which indicates that they are synchronized. This also shows how EXata CPS does not add any significant processing delays, despite the signal being routed through EXata CPS. Fast load shedding helps the DERs respond to the frequency deviation allowing the microgrid frequency transient to be maintained within acceptable boundaries, dropping only to 58.5 Hz for a few milliseconds.

Users can also see from the Power Balance figure how the DERs primary response together with the shedding of load 4 allow the matching between supply and demand in islanded mode. However, the slight mismatch in the Power Balance figure is because the DERs are generating more than power than the demand due to microgrid distribution losses. Additionally, the microgrid RMS voltage is measured. Note that the voltage dip during islanding lasts for a very short time, until the load shedding occurs. Next, a cyber-attack on the shedding command is simulated.

Scenario 1: Microgrid Islanding With Delay Attack

A damaging cyber-security breach delays the shedding command to arrive at the IED located at load 4, causing a large frequency deviation. This delay attack is triggered using EXata CPS. The attack is already preconfigured as Delay_Attack.

The attack can be triggered in EXata CPS. However, the attack parameters look different between EXata CPS 1.1 and EXata CPS 8.1.3. Please refer to the relevant section in the documentation to view the attacks on EXata CPS. The results in ScopeView will be the same.

EXata CPS 1.1

To trigger the attack in EXata CPS 1.1, right-click node 1, which represents the microgrid controller, then Attacks > Modify Packets > Delay_Attack as follows:

The following Attack Editor appears:

The main things to be noted from the Attack Editor are the following:

1. Command Type is “Attack Command”

2. Attack Type is Modify Packets

3. Press “Launch” to start the attack.

EXata CPS 8.1.3

To trigger the attack in EXata CPS 8.1.3, right-click node 1, which represents the microgrid controller, then Attacks > Modify Packets > New:

Use the following parameters for Delay attack:

The main things to be noted from the Attack Editor are the following:

1. Command Type is “Attack Command”.

2. Attack Type is "Modify Packets".

3. Layer Type is 'MAC".

4. Destination MAC Address field has to have the MAC address of node 1 which is the RTAC controller in this example.

5. MODP Attack Type is "Flow Modification".

6. Number of Flow Modifications is 1 since we only need one delay attack.

7. Flow Modification Type is "Delay".

8. Random Delay Distribution is "Deterministic"

9. Press “Launch” to start the attack.

NOTE: If you create an attack using the Tools → Attack Template Editor option in EXata CPS 8.1.3, you can save it or if you make an attack during runtime and save it, you can use it later in your model.

ScopeView

Whether you run the attack in EXata CPS 1.1 or 8.1.3 , you can view the results in ScopeView. Open it and choose Import Template. Find the template file "Cyber_Physical_Microgrid.svt" file from the project folder. You may run the example on the ScopeView template and see the results on the Scenario 1 page. The results should look as follows:

The first thing to note from the results is how the trip command received is delayed by one second. Consequently, the frequency went down to 57.8 Hz before being regulated back to its nominal value. Previously without the attack, the frequency went down to only 58.6 Hz, for a very brief time.

On the other hand, a steeper and longer lasting voltage dip down to 23 kV was registered, with increased voltage oscillations. As a side note, there is no underlying protection system functionality in the model, which made these oscillations permissible. Additionally, from the power balance plot, the DERs were able to cover the supply-demand imbalance with their primary response. However, that did not help in the fast recovery of the frequency back to its nominal value, therefore a fast load shedding is indeed necessary.

After the results are captured, it is important to remember that the attack remains ongoing. Before proceeding to the second scenario, the user must stop the delay attack.

The delay attack is stopped as follows for both EXata CPS 1.1 and 8.1.3:

In the Attack Editor, accessed as specified previously, change the Command Type to “Stop Command”, Attack Type to “Modify Packets”, and click Launch.

EXata CPS outputs the following as verification in the EXata Output Window:

Scenario 2: Islanded Microgrid Without Data Multiply Attack

In the second scenario, the microgrid is operating in islanded steady-state operation. As can be seen in the results below, the frequency is well-regulated near the nominal 60 Hz value by the CHP unit.

• Additionally, the microgrid voltage at the PCC is stable and slightly lower than the utility nominal voltage at 24.4 kV.

• As well, the load 2 measurement packets as sent and received by the MGCC are overlapping. This is important to note as the cyber attack is targeting this datapoint.

• Finally, the load 3 active power measurement is steady near its 4 MW nominal value.

Scenario 2: Islanded Microgrid WIth Data Multiply Attack

A disruptive cyber attack during the microgrid steady-state operation manipulates the load 2 demand measurement sent to the MGCC. The MGCC is programmed to shed the priority load 3 if the mismatch between generation and load exceeds 3 MW. The attack is already preconfigured as Multiply_Attack.

The attack can be triggered in EXata CPS 1.1 and 8.1.3. The parameters of the attack are different between two versions. Please refer to the section which is related to the EXata CPS version you are using,

EXata CPS 1.1

To trigger the attack in EXata CPS 1.1, right-click node 13, which represents load 2, then Attacks > Modify Packets > Multiply_Attack as follows:

The following Attack Editor appears:

From the figure, the following must be noted:

1. Attack Type is Modify Packets, and the MODP Attack Type is Multiply

2. The MAC address is specified as 01-0C-CD-01-00-19, which is associated with the load 2 measurements.

3. Note the multiply value, number of bytes and the starting byte. From the figure, this means that we are multiplying bytes 113 and 114 by 2.

4. Press “Launch" to start the attack.

EXata CPS 8.1.3

To trigger the attack in EXata CPS 8.1.3, right-click node 13, which represents load 2, then Attacks > Modify Packets > New:

Change the parameters of the new Attack to the ones shown in the picture below. 

From the figure, the following must be noted:

1. Attack Type is Modify Packets, Layer Type is "MAC".

2. The MAC address is specified as "01-0C-CD-01-00-19", which is associated with the load 2 measurements.

3. MODP Attack Type is "Data Modification". Number of Data Modifications is 1 since we only need to initiate one attack.

4. Data Modification Type is "Multiply".

5. Multiply Type is Value, Value Type is "16 Bit Unsigned Integer". We are interested in changing two bytes of the payload, therefore we need to initiate the attack on 16 bits.

6. The start byte is 114 and multiply value is 2. 

7. Press “Launch" to start the attack.

NOTE: If you create an attack using the Tools → Attack Template Editor option in EXata CPS 8.1.3, you can save it or if you make an attack during runtime and save it, you can use it later in your model.

ScopeView

You may run the example ScopeView template and see the results on the Scenario 2 page.

The results should look as follows:

The first thing to observe from the results is how the load 2 measurements are manipulated before they are received by the MGCC. As a result, the MGCC periodically trips and reconnects load 3, which can be noticed from the load 3 active power measurement. Consequently, the microgrid suffers from deteriorating power quality, including high frequency and voltage oscillations.

This kind of cyber attack would indicate to microgrid control system designers to make their control more robust over time in order to reduce the risk of a major cyber security breach.

After the results are captured, do not forget to stop the attack, as specified in the first scenario, but with node 13.

Using the LabVIEW Panel (Optional)

The LabVIEW panel can be used for demonstration purposes. The LabVIEW panel screenshot is inserted below.

The LabVIEW panel receives breakers status of PCC, Load 3, Load 4 and the inductive load fault. The panel also receives the power generated or consumed by each DER or load, as shown next to each. Finally, the panel receives frequency and voltage measurement at the PCC on the microgrid side in order to watch these two important quantities during islanding, cyber-attacks, and other events.

The LabVIEW panel allows the user to island the microgrid, by clicking the far-right button on the top right of the panel. When the user clicks Islanding, the PCC breaker LED turns green as shown below. The user can activate voltage support by the ESSs. Finally, the user can activate an inductive load fault, which causes the load fault breaker LED to turn red, as it is open by default.

The user may launch cyber-attacks on EXata CPS, and then manipulate the microgrid with the available controls and watch the impact from the cyber-attack from the resulting measurements.

Additional Information

Communication I/O Configuration

Communication Points Between Microgrid Controller and IEDs

LabVIEW TCP/IP Interface

Messages sent to the LabVIEW Panel.

Messages received from the LabVIEW Panel.

V_rms Measurements

The voltage measurement associated with a measurement block calculates the true RMS of phase a of the voltage. The RMS value is calculated over a running average window of one cycle of the fundamental frequency. The output is multiplied by square root of 3. For quick validation, the RMS value calculated in HYPERSIM is compared to the ScopeView function Window_RMS_Value as shown below. The voltage measurement is made at the PCC on the microgrid side during islanding. As can be noticed, the two plots overlap.

References

[1] L. Zang, S. Li, L. Wihl, M. Kazemtabrizi, S. Q. Ali, J.-N. Paquin, and S. Labbé, “Cybersecurity Study of Power System Utilizing Advanced CPS Simulation Tools.”