{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# GridAPPS-D Application Structure" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Application Structure\n", "\n", "The structure of a GridAPPS-D application can be broken into nine sections:\n", "\n", "* Querying for the power system model\n", "* Querying for measurement MRIDs\n", "* Querying for weather data (if needed)\n", "* Configuring parallel simulations (if needed)\n", "* App core algorithm & measurement processing\n", "* Subscribing to simulation output\n", "* Publishing equipment commands\n", "* Querying historical & timeseries data\n", "* Subscribing and publishing to logs\n", "\n", "Each of these task sections within an application are explained below with sample code that can interact with a simulation running on the GridAPPS-D platform." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connecting to GridAPPS-D Platform\n", "\n", "Prior to running any of the API calls or other core application code, the application needs to establish a secure connection with the GridAPPS-D platform." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Import GridAPPSD-Python Library:\n", "from gridappsd import GridAPPSD\n", "\n", "# When developing locally, paste Simulation ID into this variable\n", "# When running inside docker, this is passed automatically by platform\n", "viz_simulation_id = \"661579568\"\n", "\n", "# Simulation running on IEEE 123 node model:\n", "model_mrid = \"_C1C3E687-6FFD-C753-582B-632A27E28507\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Set environment variables - when developing, put environment variable in ~/.bashrc file or export in command line\n", "# export GRIDAPPSD_USER=system\n", "# export GRIDAPPSD_PASSWORD=manager\n", "import json\n", "import os # Set username a\n", "os.environ['GRIDAPPSD_USER'] = 'tutorial_user'\n", "os.environ['GRIDAPPSD_PASSWORD'] = '12345!'\n", "\n", "# Connect to GridAPPS-D Platform\n", "gapps = GridAPPSD(viz_simulation_id)\n", "assert gapps.connected" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying for the Power System Model\n", "\n", "The first portion of a GridAPPS-D application is series of queries to the PowerGrid Models API to obtain information about the power system model. \n", "\n", "Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Blazegraph database and create a set of local variables that contain the information needed by the app to run its internal code. \n", "\n", "An application will query for the various pieces of power system equipment relevant to its objective (e.g. a VVO app will be interested in regulators and capacitors, while a FLISR app will be interested in switches and reclosers present in the model). The query will typically include requests for information about the names, location, mRIDS, and electrical parameters for the various pieces of equipment needed by the application.. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Model Query Information flow\n", "\n", "The figure below shows the information flow involved in making a query for the power system model.\n", "\n", "The query is sent using `gapps.get_response(topic, message)` on a queue channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)) with a response expected back from the platform within the specified timeout period." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Query-for-power-system-model](images/2.4/01_query_model.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Application passes query to GridAPPS-D Platform__\n", "\n", "First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in [Using the PowerGrid Models API](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb). \n", "\n", "The application then passes the query through the PowerGrid Models API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the data managers, which obtain the desired information from the Blazegraph Database. \n", "\n", "__GridAPPS-D Platform responds to Application query__\n", "\n", "The data managers then publish the response from the Blazegraph Database to the appropriate queue channel. The PowerGrid Models API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### CIM-Graph Model Query\n", "It is recommended that users now use the CIMantic Graphs (`pip install cim-graph`) instead of the PowerGrid Models API.\n", "\n", "The CIM-graph library can be imported as shown below. It provides a labeled property graph / knowledge graph of the feeder model with all assets, electrical objects, and measurements" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from cimgraph.models import FeederModel\n", "from cimgraph.databases import GridappsdConnection, ConnectionParameters\n", "import cimgraph.data_profile.rc4_2021 as cim\n", "import cimgraph.utils as cimgraph_utils" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The network model can be obtained by connecting to the GridAPPS-D platform and then creating a new `FeederModel` object" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Establish connection to GridAPPS-D Platform\n", "params = ConnectionParameters(host = \"localhost\", port = \"61613\", cim_profile='rc4_2021', iec61970_301=7)\n", "cim_gapps = GridappsdConnection(params)\n", "\n", "feeder = cim.Feeder(mRID = model_mrid)\n", "global network\n", "network = FeederModel(connection=cim_gapps, container=feeder, distributed=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To obtain all attributes and associations of a given model dataclass, use the `.get_all_edges(cim.ClassName)` method. Shortcut methods are available in the `cimgraph.utils` library for common queries." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "network.get_all_edges(cim.LoadBreakSwitch)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "cimgraph_utils.get_all_switch_data(network)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PowerGrid Models API Query\n", "Legacy applications built on earlier versions of the GridAPPS-D platform use the PowerGrid Models API.\n", "Below is a sample query of how the application will use the PowerGrid Models API to query for the details associated for all the switches in the feeder. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd import topics as t\n", "\n", "message = {\n", " \"modelId\": model_mrid,\n", " \"requestType\": \"QUERY_OBJECT_DICT\",\n", " \"resultFormat\": \"JSON\",\n", " \"objectType\": \"LoadBreakSwitch\"\n", "}\n", "\n", "response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message)\n", "switch_dict = json.loads(response_obj[\"data\"])\n", "\n", "# Filter to get mRID for switch SW2:\n", "for index in switch_dict:\n", " if index[\"IdentifiedObject.name\"] == 'sw2':\n", " sw_mrid = index[\"IdentifiedObject.mRID\"]\n", "\n", "print(switch_dict[0]) # Print dictionary for first switch\n", "\n", "print('mRID of sw2 is ',sw_mrid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying for Measurement mRIDs\n", "\n", "The next portion of a GridAPPS-D application is series of queries to the PowerGrid Models API to obtain information about the measurements associated with various pieces of equipment the application is interested in. Due to structure of the Common Information Model (introduced in [Intro to Common Information Model](../overview/2.6-Common-Information-Model.ipynb)), there exist a separate set of objects associated with the positive-neutral-voltage (PNV), volt-ampere (VA), and position measurements (POS) for each line, transformer, switch, etc.\n", "\n", "Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Blazegraph Database and create a set of local variables that contain the unique mRIDS of each measurement needed by the app to run its internal code. In a subsequent step, the app will use these measurement mRIDs to subscribe to the live streaming data issued by the simulation.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Measurement Query Information Flow\n", "\n", "The figure below shows the information flow involved in making a query for the power system model. \n", "\n", "The query is sent using `gapps.get_response(topic, message)` on a queue channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)) with a response expected back from the platform within the specified timeout period." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Query-for-model-MRIDs](images/2.4/02_query_model_mrids.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The figure below shows the information flow involved in making a query for the power system model. \n", "\n", "__Application passes query to GridAPPS-D Platform__\n", "\n", "First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in [Using PowerGrid Models API](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb). \n", "\n", "The application then passes the query through the PowerGrid Models API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the data managers, which obtain the desired information from the Blazegraph Database. \n", "\n", "__GridAPPS-D Platform responds to Application query__\n", "\n", "The data managers then publish the response from the Blazegraph Database to the appropriate queue channel. The PowerGrid Models API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### CIM-Graph Measurement Query\n", "\n", "When using CIM-Graph, it is not necessary to keep a separate measurements dictionary or mapping to equipment objects. Instead, after calling `.get_all_edges(cim.ClassName)`, all measurements objects are already associated with all instances of that class type. To preview the list of measurement objects, simply print the `.Measurements` association of the object:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "for switch in network.graph.get(cim.LoadBreakSwitch, {}).values():\n", " print(f'switch {switch.name} has the following measurements:')\n", " print(switch.Measurements)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get more information about measurements besides their identifier, such as their type or associated equipment, run the `.get_all_edges()` method:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "network.get_all_edges(cim.Analog)\n", "network.get_all_edges(cim.Discrete)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Print the first three measurements only\n", "for meas in list(network.graph.get(cim.Analog, {}).values())[:3]:\n", " print(f'Measurement {meas.name} has type {meas.measurementType} and is associated with equipment with mRID {meas.PowerSystemResource.identifier}')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PowerGrid Models API Measurement Query\n", "Legacy applications built on earlier versions of the GridAPPS-D platform use the PowerGrid Models API.\n", "Below is a sample query of how the application will use the PowerGrid Models API to query for the measurement mRIDs of all switches in the power system model" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "message = {\n", " \"modelId\": model_mrid,\n", " \"requestType\": \"QUERY_OBJECT_MEASUREMENTS\",\n", " \"resultFormat\": \"JSON\",\n", " \"objectType\": \"LoadBreakSwitch\"\n", "}\n", "\n", "response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message) # Pass query to PowerGrid Models API\n", "measurements_obj = json.loads(response_obj[\"data\"])\n", "\n", "global Pos_obj # Define global python dictionary of position measurements\n", "Pos_obj = [k for k in measurements_obj if k['type'] == 'Pos'] # Filter measurements to just switch positions\n", "\n", "print(Pos_obj[0]) # Print switch position measurement mRID for first switch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying for Weather Data\n", "\n", "The next portion of a GridAPPS-D application is series of queries to the Timeseries API to obtain information about the weather data for the current time, including irradiation, temperature, etc. This information can be used for solar forecasting, load forecasting, etc.\n", "\n", "Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Timeseries Influx Database and create a set of local variables that contain the weather data needed by the app to run its internal code.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Weather Query Information Flow\n", "\n", "The figure below shows the information flow involved in making a query for the power system model. \n", "\n", "The query is sent using `gapps.get_response(topic, message)` on the Timeseries queue channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)) with a response expected back from the platform within the specified timeout period." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Query-for-weather](images/2.4/03_query_weather.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "__Application passes query to GridAPPS-D Platform__\n", "\n", "First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in [Using the Timeseries API](../api_usage/3.7-Using-the-Timeseries-API.ipynb). \n", "\n", "The application then passes the query through the Timeseries API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Data Managers, which obtain the desired information from the Timeseries Influx Database. \n", "\n", "__GridAPPS-D Platform responds to Application query__\n", "\n", "The Data Managers then publish the response from the Timeseries Influx Database to the appropriate queue channel. The Timeseries API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Weather Query Sample App Code\n", "\n", "Below is a sample query to the Timeseries API requesting all weather data between a certain startTime and endTime (given in unix absolute time). The application can then use that weather data to feed its internal forecasting algorithm." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Use queryFilter of \"startTime\" and \"endTime\"\n", "message = {\n", " \"queryMeasurement\":\"weather\",\n", " \"queryFilter\":{\"startTime\":\"1357048800000000\",\n", " \"endTime\":\"1357048860000000\"},\n", " \"responseFormat\":\"JSON\"\n", "}\n", "\n", "response_obj = gapps.get_response(t.TIMESERIES, message) # Pass query to Timeseries API \n", "weather_obj = json.loads(response_obj[\"data\"])\n", "\n", "print(weather_obj[1]) # Print first line of weather data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configuring a Parallel Simulation\n", "\n", "Some applications may choose to run parallel simulations (similar to a digital twin), either within the GridAPPS-D platform or by exporting the model to OpenDSS, GridLAB-D, etc. This is accomplished through one or more queries to the Configuration File API to create a simulation configuration file and/or exported power system model.\n", "\n", "The simulation configuration file contains all the necessary info to create a new simulation, including the power system model, date/time, and variations from the default basecase (i.e. re-dispatched DERs and switches that have been opened/closed).\n", "\n", "The exported power system model is the entire model as a set of GLM or DSS that can be saved to an external file and then solved with a different power flow solver outside of the GridAPPS-D Platform." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Configuration Query Information Flow\n", "\n", "The figure below shows the information flow involved in making a query for the power system model. \n", "\n", "The query is sent using `gapps.get_response(topic, message)` on the Configuration File queue channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)) with a response expected back from the platform within the specified timeout period." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![config-sim-export](images/2.4/04_config_sim_export.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Application passes query to GridAPPS-D Platform__\n", "\n", "First, the application creates a query message for requesting information about the desired power system configuration in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in [Using the Configuration File API](../api_usage/3.4-Using-the-Configuration-File-API.ipynb) \n", "\n", "The application then passes the query through the Configuration File API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Configuration Manager.\n", "\n", "__GridAPPS-D Platform responds to Application query__\n", "\n", "The Configuration Manager obtains the CIM XML file for the desired power system model and then converts it to the desired output format with all of the requested changes to the model. The Configuration File API then returns the desired information back to the application as a JSON message (for Y-Bus or partial models) or export the files to the directory specified in the " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Configuration Query Sample App Code\n", "\n", "Below is a sample query showing how an application would make a query through the Configuration File API to change all loads to constant current loads, convert the power system model to a set of OpenDSS files, and export them to the directory `/tmp/dsssimulation`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "topic = \"goss.gridappsd.process.request.config\"\n", "\n", "message = {\n", " \"configurationType\": \"DSS All\",\n", " \"parameters\": {\n", " \"directory\": \"/tmp/dsssimulation/\",\n", " \"model_id\": model_mrid,\n", " \"simulation_id\": \"12345678\",\n", " \"simulation_name\": \"ieee123\",\n", " \"simulation_start_time\": \"1518958800\",\n", " \"simulation_duration\": \"60\",\n", " \"simulation_broker_host\": \"localhost\",\n", " \"simulation_broker_port\": \"61616\",\n", " \"schedule_name\": \"ieeezipload\",\n", " \"load_scaling_factor\": \"1.0\",\n", " \"z_fraction\": \"0.0\",\n", " \"i_fraction\": \"1.0\",\n", " \"p_fraction\": \"0.0\",\n", " \"solver_method\": \"NR\" }\n", "}\n", "\n", "gapps.get_response(topic, message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Processing Measurements & App Core Algorithm\n", "\n", "The next portion of a GridAPPS-D application is the measurement processing and core algorithm section. This section is built as either a class or function definition with prescribed arguments. Each has its advantages and disadvantages:\n", "\n", "* The function-based approach is simpler and easier to implement. However, any parameters obtained from other APIs or methods to be used inside the function currently need to be defined as global variables. \n", "\n", "* The class-based approach is more complex, but also more powerful. It provides greater flexibility in creating additional methods, arguments, etc." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### App Core Information Flow\n", "\n", "This portion of the application does not communicate directly with the GridAPPS-D platform. \n", "\n", "Instead, the next part of the GridAPPS-D application ([Subscribing to Simulation Output](#Subscribing-to-Simulation-Output)) delivers the simulated SCADA measurement data to the core algorithm function / class definition. The core algorithm processes the data to extract the desired measurements and run its optimization / control agorithm." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![app-core-algorithm](images/2.4/05_app_core_algorithm.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__No message from core algorithm to GridAPPS-D Platform__\n", "\n", "The core algorithm does not send any API messages to the platform\n", "\n", "__No response to core algorithm from GridAPPS-D Platform__\n", "\n", "The core algorithm receives its measurement data and other imputs from the subscription object defined next, rather than directly from the GridAPPS-D platform." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### App Core Sample App Code\n", "\n", "Below is a very simple core algorithm that determines the number of open switches in the model and prints the result for each simulation timestep. The syntax of the function / class definition is described in detail in " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def demoSubscription1(headers, message):\n", " # Extract time and measurement values from message \n", " timestamp = message[\"message\"][\"timestamp\"]\n", " meas_value = message[\"message\"][\"measurements\"]\n", " \n", " # Filter to measurements with value of zero\n", " open_switches = []\n", " for switch in network.graph.get(cim.LoadBreakSwitch, {}).values():\n", " for meas in switch.Measurements:\n", " if meas.measurementType == 'Pos':\n", " power = meas_value[meas.mRID]\n", " if power[\"value\"] == 0:\n", " open_switches.append(switch.mRID)\n", " \n", " # Print message to command line\n", " print(\"............\")\n", " print(\"Number of open switches at time\", timestamp, ' is ', len(set(open_switches)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Subscribing to Simulation Output\n", "\n", "The next portion of a GridAPPS-D application is series of queries to the Timeseries API to obtain information about the weather data for the current time, including irradiation, temperature, etc. This information can be used for solar forecasting, load forecasting, etc.\n", "\n", "Because GridAPPS-D applications are designed to be portable across numerous power system models without any code modification, the application must query the Timeseries Influx Database and create a set of local variables that contain the weather data needed by the app to run its internal code.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simulation Subscription Information Flow\n", "\n", "The figure below shows the information flow involved in subscribing to the simulation output.\n", "\n", "The subscription request is sent using `gapps.subscribe(topic, class/function object)` on the specific Simulation topic channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)). No immediate response is expected back from the platform. However, after the next simulation timestep, the Platform will continue to deliver a complete set of measurements back to the application for each timestep until the end of the simulation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![subscribe-to-simulation](images/2.4/06_subscribe_to_sim.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Application passes subscription request to GridAPPS-D Platform__\n", "\n", "The subscription request is perfromed by passing the app core algorithm function / class definition to the `gapps.subscribe` method. The application then passes the subscription request through the Simulation API to the topic channel for the particular simulation on the GOSS Message Bus. If the application is authorized to access simulation output, the subscription request is delivered to the Simulation Manager.\n", "\n", "__GridAPPS-D Platform delivers published simulation output to Application__\n", "\n", "Unlike the previous queries made to the various databases, the GridAPPS-D Platform does not provide any immediate response back to the application. Instead, the Simulation Manager will start delivering measurement data back to the application through the Simulation API at each subsequent timestep until the simulation ends or the application unsubscribes. The measurement data is then passed to the core algorithm class / function, where it is processed and used to run the app's optimization / control algorithms. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simulation Subscription Sample App Code\n", "\n", "Below is an example of how an application subscribes to the GridAPPS-D simulation output using the function or class definition created as part of the [Measurement Processing / App Core](#6.-Processing-Measurements-&-App-Core-Algorithm)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.topics import simulation_output_topic\n", "\n", "output_topic = simulation_output_topic(viz_simulation_id)\n", "\n", "gapps.subscribe(output_topic, demoSubscription1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Publishing Equipment Commands\n", "\n", "The next portion of a GridAPPS-D App is publishing equipment control commands based on the optimization results or objectives of the app algorithm. \n", "\n", "Depending on the preference of the developer, this portion can be a separate function definition, or included as part of the main class definition as part of the [Measurement Processing / App Core](#Processing-Measurements-&-App-Core-Algorithm) class definition described earlier." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Equipment Command Information Flow\n", "\n", "The figure below outlines information flow involved in publishing equipment commands to the simulation input. \n", "\n", "Unlike the various queries to the databases in the app sections earlier, equipment control commands are passed to the GridAPPS-D API using the `gapps.send(topic, message)` method. No response is expected from the GridAPPS-D platform. \n", "\n", "If the application desires to verify that the equipment control command was received and implemented, it needs to do so by 1) checking for changes in the associated measurements at the next timestep and/or 2) querying the Timeseries Database for historical simulation data associated with the equipment control command." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![publish-commands](images/2.4/07_publish_commands.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Application sends difference message to GridAPPS-D Platform__\n", "\n", "First, the application creates a difference message containing the current and desired future control point / state of the particular piece of power system equipment to be controlled. The difference message is a JSON string or equivalant Python dictionary object. The syntax of a difference message is explained in detail in [Publishing Equipment Commands](../api_usage/3.6-Controlling-Simulation-API.ipynb#Format-of-a-Difference-Message).\n", "\n", "The application then passes the query through the Simulation API to the GridAPPS-D Platform, which publishes it on the topic channel for the particular simulation on the GOSS Message Bus. If the app is authenticated and authorized to control equipment, the difference message is delivered to the Simulation Manager. The Simulation Manager then passes the command to the simulation through the Co-Simulation Bridge (either FNCS or HELICS).\n", "\n", "__No response from GridAPPS-D Platform back to Application__\n", "\n", "The GridAPPS-D Platform does not provide any response back to the application after processing the difference message and implementing the new equipment control setpoint." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Equipment Command Sample App Code\n", "\n", "Below is an example of an app code block " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import time\n", "from gridappsd import DifferenceBuilder\n", "from gridappsd.topics import simulation_input_topic\n", "\n", "# Find switch named sw2\n", "for switch in network.graph.get(cim.LoadBreakSwitch,{}).values():\n", " if switch.name == 'sw2':\n", " sw2 = switch\n", " break\n", "\n", "input_topic = simulation_input_topic(viz_simulation_id)\n", "\n", "my_open_diff = DifferenceBuilder(viz_simulation_id)\n", "my_open_diff.add_difference(sw2.mRID, \"Switch.open\", 1, 0) # Open switch given by sw_mrid\n", "open_message = my_open_diff.get_message()\n", "\n", "my_close_diff = DifferenceBuilder(viz_simulation_id)\n", "my_close_diff.add_difference(sw2.mRID, \"Switch.open\", 0, 1) # Close switch given by sw_mrid\n", "close_message = my_close_diff.get_message()\n", "\n", "while True:\n", " time.sleep(5)\n", " gapps.send(input_topic, open_message)\n", " time.sleep(5)\n", " gapps.send(input_topic, close_message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Viewing Application Results in GridAPPS-D Viz\n", "\n", "Return to the browser tab in which the GridAPPS-D Simulation is currently running. Switch `sw5` will now be opening and closing every 5 seconds, with the downstream portion of the feeder being de-energized and reconnected with each switch operation. \n", "\n", "The core application algorithm will also reflect this with the printed response alternating between two and three open switches every few timesteps." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying Historical & Timeseries Data\n", "\n", "The next portion of a GridAPPS-D application is querying historical data from the current and/or previous simulations. \n", "\n", "All simulation output and commands from the current and previous simulations are stored in the Timeseries Database, and can be queried to provide AI/ML training data, verify processing of equipment commands, or \n", "\n", "Note that Timeseries Database data is cleared when the GridAPPS-D Platform is shut down with the ./stop.sh script. It is recommended to copy historical / training data to an external persistent directory using the `docker cp` command, as given in [Docker Shortcuts](../installation/1.6-Docker-Shortcuts.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Historical Data Query Information Flow\n", "\n", "The figure below outlines the information flow involved in querying for historical and timeseries data.\n", "\n", "The query is sent using the `gapps.get_response(topic, message)` method on the Timeseries queue channel with a response expected back from the GridAPPS-D platform within the specified timeout period." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![query-timeseries-data](images/2.4/08_query_timeseries.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "__Application passes query to GridAPPS-D Platform__\n", "\n", "First, the application creates a query message for requesting information about the desired power system components in the format of a JSON string or equivalant Python dictionary object. The syntax of this message is explained in detail in [Querying Timeseries Data](../api_usage/3.7-Using-the-Timeseries-API.ipynb). \n", "\n", "The application then passes the query through the Timeseries API to the GridAPPS-D Platform, which publishes it to a queue channel on the GOSS Message Bus. If the app is authenticated and authorized to pass queries, the query message is delivered to the Data Managers, which obtain the desired information from the Timeseries Influx Database. \n", "\n", "__GridAPPS-D Platform responds to Application query__\n", "\n", "The Data Managers then publish the response from the Timeseries Influx Database to the appropriate queue channel. The Timeseries API then returns the desired information back to the application as a JSON message or equivalant Python dictionary object." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Historical Data Query Sample App Code" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import time\n", "\n", "start_time = str(int(time.time())-20) # Start query from 10 sec ago\n", "end_time = str(int(time.time()))\n", "\n", "# Query for a particular set of measurments\n", "message = {\n", " \"queryMeasurement\": \"simulation\",\n", " \"queryFilter\":{\"simulation_id\": viz_simulation_id,\n", " \"startTime\": start_time,\n", " \"endTime\": end_time,\n", " \"measurement_mrid\": sw2.Measurements[1].mRID},\n", " \"responseFormat\":\"JSON\"\n", "} \n", "\n", "gapps.get_response(t.TIMESERIES, message) # Pass API call" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Subscribing and Publishing to Logs\n", "\n", "The last portion of an application is subscribing and publishing to logs. This step is extremely useful for 1) informing end users of application behavior and 2) application debugging during development and demonstration.\n", "\n", "The GridAPPS-D Logging API provides an extension of the standard Python logging library and enables applications to subscribe to real-time log messages from a simulation, query previously logged messages from the MySQL database, and publish messages to their either own log or their GridAPPS-D logs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Logging Information Flow\n", "\n", "The figure below shows the information flow involved in subscribing and publishing to logs. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![subscribe-publish-to-logs](images/2.4/09_subscribe_publish_logs.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Log Message Sample App Code" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.topics import simulation_log_topic\n", "log_topic = simulation_log_topic(viz_simulation_id)\n", "\n", "def demoLogFunction(header, message):\n", " timestamp = message[\"timestamp\"]\n", " log_message = message[\"logMessage\"]\n", " \n", " print(\"Log message received at timestamp \", timestamp, \"which reads:\")\n", " print(log_message)\n", " print(\"........................\")\n", "\n", "gapps.subscribe(log_topic, demoLogFunction)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![gridappsd-logo](../images/GridAPPS-D_narrow.png)" ] } ], "metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 4 }