GridAPPS-D Application Structure
Application Structure
The structure of a GridAPPS-D application can be broken into nine sections:
Querying for the power system model
Querying for measurement MRIDs
Querying for weather data (if needed)
Configuring parallel simulations (if needed)
App core algorithm & measurement processing
Subscribing to simulation output
Publishing equipment commands
Querying historical & timeseries data
Subscribing and publishing to logs
Each of these task sections within an application are explained below with sample code that can interact with a simulation running on the GridAPPS-D platform.
Connecting to GridAPPS-D Platform
Prior to running any of the API calls or other core application code, the application needs to establish a secure connection with the GridAPPS-D platform.
[ ]:
# Import GridAPPSD-Python Library:
from gridappsd import GridAPPSD
# When developing locally, paste Simulation ID into this variable
# When running inside docker, this is passed automatically by platform
viz_simulation_id = "242488458"
# Simulation running on IEEE 123 node model:
model_mrid = "_C1C3E687-6FFD-C753-582B-632A27E28507"
[ ]:
# Set environment variables - when developing, put environment variable in ~/.bashrc file or export in command line
# export GRIDAPPSD_USER=system
# export GRIDAPPSD_PASSWORD=manager
import os # Set username a
os.environ['GRIDAPPSD_USER'] = 'tutorial_user'
os.environ['GRIDAPPSD_PASSWORD'] = '12345!'
# Connect to GridAPPS-D Platform
gapps = GridAPPSD(viz_simulation_id)
assert gapps.connected
Querying for the Power System Model
The first portion of a GridAPPS-D application is series of queries to the PowerGrid Models API to obtain information about the power system model.
Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Blazegraph database and create a set of local variables that contain the information needed by the app to run its internal code.
An application will query for the various pieces of power system equipment relevant to its objective (e.g. a VVO app will be interested in regulators and capacitors, while a FLISR app will be interested in switches and reclosers present in the model). The query will typically include requests for information about the names, location, mRIDS, and electrical parameters for the various pieces of equipment needed by the application..
Model Query Information flow
The figure below shows the information flow involved in making a query for the power system model.
The query is sent using gapps.get_response(topic, message) on a queue channel (explained in API Communication Channels) with a response expected back from the platform within the specified timeout period.
Application passes query to GridAPPS-D Platform
First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in Using the PowerGrid Models API.
The application then passes the query through the PowerGrid Models API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the data managers, which obtain the desired information from the Blazegraph Database.
GridAPPS-D Platform responds to Application query
The data managers then publish the response from the Blazegraph Database to the appropriate queue channel. The PowerGrid Models API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object.
Model Query Sample App Code
Below is a sample query of how the application will use the PowerGrid Models API to query for the details associated for all the switches in the feeder.
[ ]:
from gridappsd import topics as t
message = {
"modelId": model_mrid,
"requestType": "QUERY_OBJECT_DICT",
"resultFormat": "JSON",
"objectType": "LoadBreakSwitch"
}
response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message)
switch_dict = response_obj["data"]
# Filter to get mRID for switch SW2:
for index in switch_dict:
if index["IdentifiedObject.name"] == 'sw2':
sw_mrid = index["IdentifiedObject.mRID"]
print(switch_dict[0]) # Print dictionary for first switch
print('mRID of sw2 is ',sw_mrid)
Querying for Measurement mRIDs
The next portion of a GridAPPS-D application is series of queries to the PowerGrid Models API to obtain information about the measurements associated with various pieces of equipment the application is interested in. Due to structure of the Common Information Model (introduced in Intro to Common Information Model), there exist a separate set of objects associated with the positive-neutral-voltage (PNV), volt-ampere (VA), and position measurements (POS) for each line, transformer, switch, etc.
Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Blazegraph Database and create a set of local variables that contain the unique mRIDS of each measurement needed by the app to run its internal code. In a subsequent step, the app will use these measurement mRIDs to subscribe to the live streaming data issued by the simulation.
Measurement Query Information Flow
The figure below shows the information flow involved in making a query for the power system model.
The query is sent using gapps.get_response(topic, message) on a queue channel (explained in API Communication Channels) with a response expected back from the platform within the specified timeout period.
The figure below shows the information flow involved in making a query for the power system model.
Application passes query to GridAPPS-D Platform
First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in Using PowerGrid Models API.
The application then passes the query through the PowerGrid Models API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the data managers, which obtain the desired information from the Blazegraph Database.
GridAPPS-D Platform responds to Application query
The data managers then publish the response from the Blazegraph Database to the appropriate queue channel. The PowerGrid Models API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object.
Measurement Query Sample App Code
Below is a sample query of how the application will use the PowerGrid Models API to query for the measurement mRIDs of all switches in the power system model
[ ]:
message = {
"modelId": model_mrid,
"requestType": "QUERY_OBJECT_MEASUREMENTS",
"resultFormat": "JSON",
"objectType": "LoadBreakSwitch"
}
response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message) # Pass query to PowerGrid Models API
measurements_obj = response_obj["data"]
global Pos_obj # Define global python dictionary of position measurements
Pos_obj = [k for k in measurements_obj if k['type'] == 'Pos'] # Filter measurements to just switch positions
print(Pos_obj[0]) # Print switch position measurement mRID for first switch
Querying for Weather Data
The next portion of a GridAPPS-D application is series of queries to the Timeseries API to obtain information about the weather data for the current time, including irradiation, temperature, etc. This information can be used for solar forecasting, load forecasting, etc.
Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Timeseries Influx Database and create a set of local variables that contain the weather data needed by the app to run its internal code.
Weather Query Information Flow
The figure below shows the information flow involved in making a query for the power system model.
The query is sent using gapps.get_response(topic, message) on the Timeseries queue channel (explained in API Communication Channels) with a response expected back from the platform within the specified timeout period.
Application passes query to GridAPPS-D Platform
First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in Using the Timeseries API.
The application then passes the query through the Timeseries API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Data Managers, which obtain the desired information from the Timeseries Influx Database.
GridAPPS-D Platform responds to Application query
The Data Managers then publish the response from the Timeseries Influx Database to the appropriate queue channel. The Timeseries API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object.
Weather Query Sample App Code
Below is a sample query to the Timeseries API requesting all weather data between a certain startTime and endTime (given in unix absolute time). The application can then use that weather data to feed its internal forecasting algorithm.
[ ]:
# Use queryFilter of "startTime" and "endTime"
message = {
"queryMeasurement":"weather",
"queryFilter":{"startTime":"1357048800000000",
"endTime":"1357048860000000"},
"responseFormat":"JSON"
}
response_obj = gapps.get_response(t.TIMESERIES, message) # Pass query to Timeseries API
weather_obj = response_obj["data"]
print(weather_obj[1]) # Print first line of weather data
Configuring a Parallel Simulation
Some applications may choose to run parallel simulations (similar to a digital twin), either within the GridAPPS-D platform or by exporting the model to OpenDSS, GridLAB-D, etc. This is accomplished through one or more queries to the Configuration File API to create a simulation configuration file and/or exported power system model.
The simulation configuration file contains all the necessary info to create a new simulation, including the power system model, date/time, and variations from the default basecase (i.e. re-dispatched DERs and switches that have been opened/closed).
The exported power system model is the entire model as a set of GLM or DSS that can be saved to an external file and then solved with a different power flow solver outside of the GridAPPS-D Platform.
Configuration Query Information Flow
The figure below shows the information flow involved in making a query for the power system model.
The query is sent using gapps.get_response(topic, message) on the Configuration File queue channel (explained in API Communication Channels) with a response expected back from the platform within the specified timeout period.
Application passes query to GridAPPS-D Platform
First, the application creates a query message for requesting information about the desired power system configuration in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in Using the Configuration File API
The application then passes the query through the Configuration File API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Configuration Manager.
GridAPPS-D Platform responds to Application query
The Configuration Manager obtains the CIM XML file for the desired power system model and then converts it to the desired output format with all of the requested changes to the model. The Configuration File API then returns the desired information back to the application as a JSON message (for Y-Bus or partial models) or export the files to the directory specified in the
Configuration Query Sample App Code
Below is a sample query showing how an application would make a query through the Configuration File API to change all loads to constant current loads, convert the power system model to a set of OpenDSS files, and export them to the directory /tmp/dsssimulation.
[ ]:
topic = "goss.gridappsd.process.request.config"
message = {
"configurationType": "DSS All",
"parameters": {
"directory": "/tmp/dsssimulation/",
"model_id": model_mrid,
"simulation_id": "12345678",
"simulation_name": "ieee123",
"simulation_start_time": "1518958800",
"simulation_duration": "60",
"simulation_broker_host": "localhost",
"simulation_broker_port": "61616",
"schedule_name": "ieeezipload",
"load_scaling_factor": "1.0",
"z_fraction": "0.0",
"i_fraction": "1.0",
"p_fraction": "0.0",
"solver_method": "NR" }
}
gapps.get_response(topic, message)
Processing Measurements & App Core Algorithm
The next portion of a GridAPPS-D application is the measurement processing and core algorithm section. This section is built as either a class or function definition with prescribed arguments. Each has its advantages and disadvantages:
The function-based approach is simpler and easier to implement. However, any parameters obtained from other APIs or methods to be used inside the function currently need to be defined as global variables.
The class-based approach is more complex, but also more powerful. It provides greater flexibility in creating additional methods, arguments, etc.
App Core Information Flow
This portion of the application does not communicate directly with the GridAPPS-D platform.
Instead, the next part of the GridAPPS-D application (Subscribing to Simulation Output) delivers the simulated SCADA measurement data to the core algorithm function / class definition. The core algorithm processes the data to extract the desired measurements and run its optimization / control agorithm.
No message from core algorithm to GridAPPS-D Platform
The core algorithm does not send any API messages to the platform
No response to core algorithm from GridAPPS-D Platform
The core algorithm receives its measurement data and other imputs from the subscription object defined next, rather than directly from the GridAPPS-D platform.
App Core Sample App Code
Below is a very simple core algorithm that determines the number of open switches in the model and prints the result for each simulation timestep. The syntax of the function / class definition is described in detail in
[ ]:
def demoSubscription1(header, message):
# Extract time and measurement values from message
timestamp = message["message"]["timestamp"]
meas_value = message["message"]["measurements"]
meas_mrid = list(meas_value.keys()) #obtain list of all mrid from message
# Filter to measurements with value of zero
open_switches = []
for index in Pos_obj:
if index["measid"] in meas_value:
mrid = index["measid"]
power = meas_value[mrid]
if power["value"] == 0:
open_switches.append(index["eqname"])
# Print message to command line
print("............")
print("Number of open switches at time", timestamp, ' is ', len(set(open_switches)))
Subscribing to Simulation Output
The next portion of a GridAPPS-D application is series of queries to the Timeseries API to obtain information about the weather data for the current time, including irradiation, temperature, etc. This information can be used for solar forecasting, load forecasting, etc.
Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Timeseries Influx Database and create a set of local variables that contain the weather data needed by the app to run its internal code.
Simulation Subscription Information Flow
The figure below shows the information flow involved in subscribing to the simulation output.
The subscription request is sent using gapps.subscribe(topic, class/function object) on the specific Simulation topic channel (explained in API Communication Channels). No immediate response is expected back from the platform. However, after the next simulation timestep, the Platform will continue to deliver a complete set of measurements back to the application for each timestep until the end of the simulation.
Application passes subscription request to GridAPPS-D Platform
The subscription request is perfromed by passing the app core algorithm function / class definition to the gapps.subscribe method. The application then passes the subscription request through the Simulation API to the topic channel for the particular simulation on the GOSS Message Bus. If the application is authorized to access simulation output, the subscription request is delivered to the Simulation Manager.
GridAPPS-D Platform delivers published simulation output to Application
Unlike the previous queries made to the various databases, the GridAPPS-D Platform does not provide any immediate response back to the application. Instead, the Simulation Manager will start delivering measurement data back to the application through the Simulation API at each subsequent timestep until the simulation ends or the application unsubscribes. The measurement data is then passed to the core algorithm class / function, where it is processed and used to run the app’s optimization / control algorithms.
Simulation Subscription Sample App Code
Below is an example of how an application subscribes to the GridAPPS-D simulation output using the function or class definition created as part of the Measurement Processing / App Core
[ ]:
from gridappsd.topics import simulation_output_topic
output_topic = simulation_output_topic(viz_simulation_id)
gapps.subscribe(output_topic, demoSubscription1)
Publishing Equipment Commands
The next portion of a GridAPPS-D App is publishing equipment control commands based on the optimization results or objectives of the app algorithm.
Depending on the preference of the developer, this portion can be a separate function definition, or included as part of the main class definition as part of the Measurement Processing / App Core class definition described earlier.
Equipment Command Information Flow
The figure below outlines information flow involved in publishing equipment commands to the simulation input.
Unlike the various queries to the databases in the app sections earlier, equipment control commands are passed to the GridAPPS-D API using the gapps.send(topic, message) method. No response is expected from the GridAPPS-D platform.
If the application desires to verify that the equipment control command was received and implemented, it needs to do so by 1) checking for changes in the associated measurements at the next timestep and/or 2) querying the Timeseries Database for historical simulation data associated with the equipment control command.
Application sends difference message to GridAPPS-D Platform
First, the application creates a difference message containing the current and desired future control point / state of the particular piece of power system equipment to be controlled. The difference message is a JSON string or equivalant Python dictionary object. The syntax of a difference message is explained in detail in Publishing Equipment Commands.
The application then passes the query through the Simulation API to the GridAPPS-D Platform, which publishes it on the topic channel for the particular simulation on the GOSS Message Bus. If the app is authenticated and authorized to control equipment, the difference message is delivered to the Simulation Manager. The Simulation Manager then passes the command to the simulation through the Co-Simulation Bridge (either FNCS or HELICS).
No response from GridAPPS-D Platform back to Application
The GridAPPS-D Platform does not provide any response back to the application after processing the difference message and implementing the new equipment control setpoint.
Equipment Command Sample App Code
Below is an example of an app code block
[ ]:
import time
from gridappsd import DifferenceBuilder
from gridappsd.topics import simulation_input_topic
input_topic = simulation_input_topic(viz_simulation_id)
my_open_diff = DifferenceBuilder(viz_simulation_id)
my_open_diff.add_difference(sw_mrid, "Switch.open", 1, 0) # Open switch given by sw_mrid
open_message = my_open_diff.get_message()
my_close_diff = DifferenceBuilder(viz_simulation_id)
my_close_diff.add_difference(sw_mrid, "Switch.open", 0, 1) # Close switch given by sw_mrid
close_message = my_close_diff.get_message()
while True:
time.sleep(5)
gapps.send(input_topic, open_message)
time.sleep(5)
gapps.send(input_topic, close_message)
Viewing Application Results in GridAPPS-D Viz
Return to the browser tab in which the GridAPPS-D Simulation is currently running. Switch sw5 will now be opening and closing every 5 seconds, with the downstream portion of the feeder being de-energized and reconnected with each switch operation.
The core application algorithm will also reflect this with the printed response alternating between two and three open switches every few timesteps.
Querying Historical & Timeseries Data
The next portion of a GridAPPS-D application is querying historical data from the current and/or previous simulations.
All simulation output and commands from the current and previous simulations are stored in the Timeseries Database, and can be queried to provide AI/ML training data, verify processing of equipment commands, or
Note that Timeseries Database data is cleared when the GridAPPS-D Platform is shut down with the ./stop.sh script. It is recommended to copy historical / training data to an external persistent directory using the docker cp command, as given in Docker Shortcuts.
Historical Data Query Information Flow
The figure below outlines the information flow involved in querying for historical and timeseries data.
The query is sent using the gapps.get_response(topic, message) method on the Timeseries queue channel with a response expected back from the GridAPPS-D platform within the specified timeout period.
Application passes query to GridAPPS-D Platform
First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in Querying Timeseries Data.
The application then passes the query through the Timeseries API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Data Managers, which obtain the desired information from the Timeseries Influx Database.
GridAPPS-D Platform responds to Application query
The Data Managers then publish the response from the Timeseries Influx Database to the appropriate queue channel. The Timeseries API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object.
Historical Data Query Sample App Code
[ ]:
import time
start_time = str(int(time.time())-10) # Start query from 10 sec ago
end_time = str(int(time.time()))
# Query for a particular set of measurments
message = {
"queryMeasurement": "simulation",
"queryFilter":{"simulation_id": simulation_id,
"startTime": start_time,
"endTime": end_time,
"measurement_mrid": pos_obj},
"responseFormat":"JSON"
}
gapps.get_response(t.TIMESERIES, message) # Pass API call
Subscribing and Publishing to Logs
The last portion of an application is subscribing and publishing to logs. This step is extremely useful for 1) informing end users of application behavior and 2) application debugging during development and demonstration.
The GridAPPS-D Logging API provides an extension of the standard Python logging library and enables applications to subscribe to real-time log messages from a simulation, query previously logged messages from the MySQL database, and publish messages to their either own log or their GridAPPS-D logs.
Logging Information Flow
The figure below shows the information flow involved in subscribing and publishing to logs.
Log Message Sample App Code
[ ]:
from gridappsd.topics import simulation_log_topic
log_topic = simulation_log_topic(viz_simulation_id)
def demoLogFunction(header, message):
timestamp = message["timestamp"]
log_message = message["logMessage"]
print("Log message received at timestamp ", timestamp, "which reads:")
print(log_message)
print("........................")
gapps.subscribe(log_topic, demoLogFunction)
