{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# GridAPPS-D Python Library" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Intro to GridAPPSD-Python\n", "\n", "GridAPPSD-Python is a Python library that wraps API calls and passes them to the various GridAPPS-D APIs through the GOSS Message Bus.\n", "\n", "The library has numerous shortcuts to help you develop applications faster and interface them with other applications, services, and GridAPPS-D compatible software packages.\n", "\n", "The GridAPPSD-Python library requires a python version >= 3.6 and < 4 in order to work properly. (Note: no testing has been done with python 4 to date).\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The GridAPPSD-Python library can be installed using `pip install gridappsd-python`.\n", "\n", "For more information, see the GridAPPSD-Python [GitHub Repo](https://github.com/GRIDAPPSD/gridappsd-python) and [PyPi site](https://pypi.org/project/gridappsd-python/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connecting to GridAPPS-D Platform\n", "\n", "Before starting any development in the GridAPPS-D environment, it is necessary to establish a connection to the GridAPPS-D Platform.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specifying Environment Variables (Preferred)\n", "\n", "The preferred method for establishing a connection with the GridAPPS-D Platform is to define a set of environment variables that specify the connection address, port, username, and password. \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Specifying the Environment Variables in Python Script__\n", "\n", "This method is recommended for initial application development when running in a development environment, such as PyCharm or the Jupyter Notebook tutorials." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Establish connection to GridAPPS-D Platform:\n", "from gridappsd import GridAPPSD\n", "\n", "import os # Set username and password\n", "os.environ['GRIDAPPSD_USER'] = 'tutorial_user'\n", "os.environ['GRIDAPPSD_PASSWORD'] = '12345!'\n", "os.environ['GRIDAPPSD_ADDRESS'] = 'localhost'\n", "os.environ['GRIDAPPSD_PORT'] = '61613'\n", "\n", "# Connect to GridAPPS-D Platform\n", "gapps = GridAPPSD()\n", "assert gapps.connected" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Specifying the Environment Variable in ~/.bashrc Script__\n", "\n", "This method is recommended for more complete applications scripts where all the application scripts are called from a single ~/.bashrc script. In that script, the environment variables can be defined and then will be available to all scripts that need to connect the GridAPPS-D Platform." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "```\n", "# export allows all processes started by this shell to have access to the global variable\n", "\n", "# address where the gridappsd server is running - default localhost\n", "export GRIDAPPSD_ADDRESS=localhost\n", "\n", "# port to connect to on the gridappsd server (the stomp client port)\n", "export GRIDAPPSD_PORT=61613\n", "\n", "# username to connect to the gridappsd server\n", "export GRIDAPPSD_USER=app_user\n", "\n", "# password to connect to the gridappsd server\n", "export GRIDAPPSD_PASSWORD=1234App\n", "\n", "# Note these should be changed on the server in a cyber secure environment!\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specifying Connection Parameters Manually\n", "\n", "An older method of connecting to the GridAPPS-D Platform is manually specifying the connection parameters. This method is still supported, but may be deprecated in future releases. \n", "\n", "This method is less flexible and has an in-built portability issues associated with hard-coded platform passwords." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "gapps = GridAPPSD(\"('localhost', 61613)\", username='system', password='manager')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### GridAPPSD-utils Deprecated\n", "\n", "GridAPPS-D Platform releases prior to 2021 used a library called `utils` to establish a connection with the platform. This library has been deprecated and replaced with Java Token Authentication using the environment variable method shown above.\n", "\n", "The authentication method below will work with 2019-2020 versions of the GridAPPS-D Platform and GridAPPSD-Python, but not with any newer releases. \n", "\n", "```\n", "# DEPRECATED authentication method\n", "from gridappsd import GridAPPSD, utils\n", "gapps = GridAPPSD(address=utils.get_gridappsd_address(),\n", " username=utils.get_gridappsd_user(), password=utils.get_gridappsd_pass())\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`utils` -- __DEPRECATED__ A set of utilities to assist with common commands, inlcuding\n", "\n", "\n", "* `utils.validate_gridappsd_uri()` -- Checks if GridAPPS-D is hosted on the correct port\n", "\n", "* `utils.get_gridappsd_address()` -- Returns the platform address such that response can be passed directly to a socket or the STOMP library\n", "\n", "* `utils.get_gridappsd_user()` -- Returns the login username \n", "\n", "* `utils.get_gridappsd_pass()` -- Returns the login password\n", "\n", "* `utils.get_gridappsd_application_id()` -- Only applicable if the environment variable 'GRIDAPPSD_APPLICATION_ID' has been set\n", "\n", "* `utils.get_gridappsd_simulation_id()` -- Retrieves the simulation id from the environment.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__It is strongly recommended that applications that previously used this method replace any connection objects with environment variables to ensure compatibility with subsequent releases of the GRIDAPPS-D platform__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Passing API calls with GridAPPSD-Python\n", "\n", "There are three methods used in GridAPPSD-Python Library to pass API calls to the GridAPPS-D platform:\n", "\n", "* `.get_response(self, topic, message, timeout)` -- Pass a database query, response expected before timeout\n", "* `.subscribe(self, topic, callback)` -- Subscribe to a data stream\n", "* `.send(self, topic, message)` -- Send a command to a simulation, no response expected\n", "\n", "Each are explained in more detail below" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### .get_response(topic, message)\n", "\n", "This is the most commonly used method for passing API calls to the GridAPPS-D Platform. This method is used when a response is expected back from the GridAPPS-D platform within a particular timeout period. It is used for all database queries using\n", "\n", "\n", "* [PowerGrid Models API](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb) -- queries for model info, object mRIDs, measurement mRIDs\n", "\n", "* [Configuration File API](../api_usage/3.4-Using-the-Configuration-File-API.ipynb) -- queries to convert the model into other file format versions\n", "\n", "* [Timeseries API](../api_usage/3.7-Using-the-Timeseries-API.ipynb) -- queries for weather data and historical data from prior simulations\n", "\n", "\n", "The syntax used when calling this method is `gapps.get_response(topic, message)` or alternatively, `gapps.get_response(topic, message, timeout = 30)`, where \n", "\n", "* `topic` is the GridAPPS-D topic for the particular API (as described in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb).\n", "\n", "* `message` is the query message specifying what information the API should return\n", "\n", "* `timeout = ` is optional and gives the number of seconds given for the API to respond. Model conversion queries using the Configuration File API may take 30 - 60 seconds for very large models. Most other queries do not need a timeout specification.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### .subscribe(topic, message)\n", "\n", "This method is used for subscribing to the real-time data stream generated by the GridAPPS-D platform while running a simulation. It is used to subscribe to information published at each time step by the \n", "\n", "* [Simulation API](../api_usage/3.6-Controlling-Simulation-API.ipynb) -- simulated SCADA data and measurements created by the simulation\n", "\n", "* [Logging API](../api_usage/3.8-Using-the-Logging-API.ipynb) -- log messages published by the Platform, applications, and simulation\n", "\n", "The `.subscribe()`method is also used to subscribe to streaming data generated by some of the GridAPPS-D services. \n", "\n", "\n", "The syntax used when calling this method is `gapps.subscribe(topic, message)`, where\n", "\n", "* `topic` is the GridAPPS-D simulation output topic, log output topic, or service output topic for the particular real-time data stream the application needs to subscribe to, (as described in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb).\n", "\n", "* `message` is the subscription message. For simulation and log outputs, it is a method or class definition, as described in [Comparison of Subscription Approaches](../api_usage/3.6-Controlling-Simulation-API.ipynb#Comparison-of-Subscription-Approaches)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### .send(topic, message)\n", "\n", "This method is used for sending equipment command and simulation input messages to the GridAPPS-D platform while running a simulation. It is used to send difference messages to the [Simulation API](../api_usage/3.6-Controlling-Simulation-API.ipynb) and for other generic publishing needs, such as sending a command input to a GridAPPS-D Service.\n", "\n", "The syntax used when calling this method is `gapps.send(topic, message)`, where\n", "\n", "* `topic` is the simulation or service input topic(as described in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb).\n", "\n", "* `message` is the API call message to be published. The most commonly used simulation input message is a [Difference Message](../api_usage/3.6-Controlling-Simulation-API.ipynb#Format-of-a-Difference-Message) used to control equipment settings." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### .unsubscribe(conn_id)\n", "\n", "This method is used to unsubscribe from a simulation or service that was previously subscribed to using the `.subscribe` method. \n", "\n", "The syntax of this method is `gapps.unsubscribe(conn_id)`, where `conn_id` is the connection id obtained when previously subscribing using the `conn_id = gapps.subscribe(topic, message)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Importing Required Python Libraries\n", "\n", "A typical GridAPPS-D application will require several libraries to be imported from GridAPPSD-Python as well as from other Python libraries.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Required GridAPPS-D Libraries\n", "\n", "The GridAPPS-Python API contains several libraries, which are used to query for information, subscribe to measurements, and publish commands to the GOSS message bus. These inlcude\n", "\n", "\n", "* `GridAPPSD` -- This is the primary library that contains numerous methods for passing API calls, connecting to the GridAPPS-D platform, and other common tasks\n", "\n", "* `topics` -- This library contains methods for constructing the correct API channel strings\n", "\n", "* `Simulation` -- This library contains shortcut methods for subscribing and controlling simulations\n", "\n", "* `Logger` -- This library contains logging methods. It is recommended to invoke those methods using the `gapps.get_logger` method rather than importing the library\n", "\n", "* `GOSS` -- This library contains methods for passing API calls to the GOSS Message Bus. It is imported automatically when importing the `GridAPPSD` library\n", "\n", "* `Houses` -- This library populates a feeder with thermal house model loads. It is imported automatically when importing the `GridAPPS` library \n", "\n", "* `utils` -- Deprecated\n", "\n", "\n", "Each of the libraries can be imported using `from gridappsd import library_name`. For example," ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd import GridAPPSD" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd import topics as t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each of the libraries are discussed in detail in the next section." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other Required Python Libraries\n", "\n", "Below is a list of some of the additional libraries that you may need to import. \n", "\n", "You may not need all of these additional libraries, depending on the needs of your application\n", "\n", "* `argparse` -- This is the recommended command-line parsing module in Python.([Online Documentation](https://docs.python.org/3/howto/argparse.html))\n", "\n", "* `json` -- Encoder and decoder for JavaScript Object Notation (JSON). ([Online Documentation](https://docs.python.org/3/library/json.html))\n", "\n", "* `logging` -- This module defines classes and functions for event logging. ([Online Documentation](https://docs.python.org/3/library/logging.html)\n", "\n", "* `sys` -- Python module for system specific parameters. ([Online Documentation](https://docs.python.org/3/library/sys.html))\n", "\n", "* `time` -- Time access and conversions. ([Online Documentation](https://docs.python.org/3/library/time.html))\n", "\n", "* `pytz` -- Library to enable resolution of cross-platform time zones and ambiguous times. ([Online Documentation](https://pypi.org/project/pytz/)\n", "\n", "* `stomp` -- Python client for accessing messaging servers using the Simple Text Oriented Messaging Protocol (STOMP). ([Online Documentation](https://pypi.org/project/stomp.py/))\n", "\n", "* `os` -- Miscellaneous operating system interface. Needed to set environment variables for the GridAPPS-D connection if working from a single Python script or notebook. ([Online Documentation](https://docs.python.org/3/library/os.html))\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import argparse\n", "import json\n", "import logging\n", "import sys\n", "import time\n", "import pytz\n", "import stomp\n", "import os" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GridAPPSD-Python GridAPPSD Library\n", "\n", "This library contains the most commonly used methods needed for building GridAPPS-D applications and services.\n", "\n", "All of these methods are for the GridAPPS-D connection object defined using `gapps = GridAPPSD()`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get Methods\n", "\n", "\n", "This group of methods are used to get information and statuses about the GridAPPS-D platform and simulations:\n", "\n", "\n", "* `.get_application_status()` -- Returns the current status of an application\n", "\n", "* `.get_application_id()` -- Returns the unique ID of an application registered with the Platform\n", "\n", "* `.get_houses()` -- Returns houses populated in the feeder\n", "\n", "* `.get_logger()` -- Returns a log instance for interacting with logs within the Platform\n", "\n", "* `.get_platform_status()` -- Returns the current status of the Platform\n", "\n", "* `.get_service_status()` -- Returns the current status of a service\n", " \n", "* `.get_simulation_id()` -- Returns the simulation ID for the current GridAPPSD connection\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set / Send Methods\n", "\n", "This group of methods are used to set the status of applications and services:\n", "\n", "\n", "* `.set_application_status()` -- Set the status of an application\n", "\n", "* `.set_service_status()` -- Set the status of a service\n", "\n", "* `.set_simulation_id(simulation_id)` -- Set the simulation ID if none is defined\n", "\n", "* `.send_simulation_status(status, message, log_level)` -- Sets simulation + service status and writes to GridAPPS-D logs\n", "\n", "* `.send_status(status, message, log_level)` -- Sets application status and writes to GridAPPS-D logs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PowerGrid Models API Methods\n", "\n", "This group of methods run pre-built PowerGrid Models API queries for simpler query types:\n", "\n", "* `query_data(query, timeout)` -- [Run a generic SPARQL Query](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-using-a-Generic-SPARQL-Query)\n", "\n", "* `query_model(model_id, object_type, object_id)` -- [Query using full CIM100 prefix](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-using-a-SPARQL-filter)\n", "\n", "* `query_model_info()` -- [Query for dictionary of all models](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Dictionary-of-all-Models)\n", "\n", "* `query_model_names(model_id)` -- [Query for mRIDs of all models](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Dictionary-of-all-Models)\n", "\n", "* `query_object(object_id, model_id)` -- [Query for CIM attributes of an object](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-CIM-Attributes-of-an-Object)\n", "\n", "* `query_object_dictionary(model_id, object_type, object_id)` -- [Query for object dictionary](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Object-Dictionary)\n", "\n", "* `query_object_types(model_id)` -- [Query for CIM classes in a model](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Dictionary-of-all-Models)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GridAPPSD-Python Topics Library\n", "\n", "The GridAPPSD-Python topics library is used to obtain the correct [API Communication Channel](../api_usage/3.1-API-Communication-Channels.ipynb), which tells the GridAPPS-D platform to which database, application, or simulation a particular API call should be delivered. \n", "\n", "Static GridAPPS-D topics (such as those for the [PowerGrid Models API](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb), [Configuration File API](../api_usage/3.4-Using-the-Configuration-File-API.ipynb), and [Timeseries API](../api_usage/3.7-Using-the-Timeseries-API.ipynb)) can be imported using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd import topics as t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Dynamic GridAPPS-D topics (such as those for the [Simulation API](../api_usage/3.6-Controlling-Simulation-API.ipynb) and various GridAPPS-D services) can be imported using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.topics import simulation_output_topic" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.topics import simulation_input_topic" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.topics import simulation_log_topic" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each of the specific methods available in the `topics` library are discussed in detail in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GridAPPSD-Python Simulation Library\n", "\n", "The GridAPPSD-Python simulation library is used for starting, running, and controlling parallel digital twin simulations. For more details on specific usage, see\n", "\n", "\n", "* [Starting a Parallel Simulation](../api_usage/3.5-Creating-Running-Simulation-API.ipynb#Starting-the-Simulation)\n", "\n", "* [Pausing, Resuming, and Stopping Parallel Simulations](../api_usage/3.5-Creating-Running-Simulation-API.ipynb#Using-the-gridappsd.simulation-Python-Library)\n", "\n", "* [Subscribing to Parallel Simulations](../api_usage/3.6-Controlling-Simulation-API.ipynb#Subscribing-to-Parallel-Simulations)\n", "\n", "\n", "\n", "The Simulation library can be imported using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd.simulation import Simulation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Available methods in the `Simulation` library are\n", "\n", "* `.start_simulation()` -- Start the simulation\n", "\n", "\n", "* `.pause()` -- Pause the simulation\n", "\n", "\n", "* `.resume()` -- Resume the simulation\n", "\n", "\n", "* `.resume_pause_at(pause_time)` -- Resume the simulation, and then pause it in so many seconds\n", "\n", "\n", "* `.stop()` -- Stop the simulation\n", "\n", "\n", "* `.run_loop()` -- Loop the entire simulation until interrupted\n", "\n", "\n", "* `.simulation_id` -- Returns the Simulation ID of the simulation\n", "\n", "\n", "* `.add_ontimestep_callback(myfunction1)` -- Run the desired function on each timestep\n", "\n", "\n", "* `.add_onmesurement_callback(myfunction2)` -- Run the desired function when a measurement is received.\n", "\n", "\n", "* `.add_oncomplete_callback(myfunction3)` -- Run the desired function when simulation is finished\n", "\n", "\n", "* `.add_onstart_callback(myfunction4)` -- Run desired function when simulation is started\n", "\n", "\n", "__Note: method name `.add_onmesurement` is misspelled in the library definition!!__\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GridAPPSD-Python DifferenceBuilder\n", "\n", "`DifferenceBuilder` is a GridAPPSD-Python library that is used to create and correctly format difference messages that used to create equipment control commands. The usage of difference builder is given in [Using DifferenceBuilder](../api_usage/3.6-Controlling-Simulation-API.ipynb#Using-GridAPPSD-Python-DifferenceBuilder).\n", "\n", "The `DifferenceBuilder` library can be imported using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from gridappsd import DifferenceBuilder\n", "\n", "my_diff_build = DifferenceBuilder(simulation_id)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![gridappsd-logo](../images/GridAPPS-D_narrow.png)" ] } ], "metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }