{ "cells": [ { "cell_type": "markdown", "id": "39ece775", "metadata": {}, "source": [ "# Using the CIMantic Graphs API" ] }, { "cell_type": "markdown", "id": "cd7a22d7", "metadata": {}, "source": [ "This tutorial provides an introduction to usage of the CIMantic Graphs library (aka CIM-Graph). \n", "\n", "CIMantic Graphs is an open-source library for creating in-memory labeled property graphs for creating, parsing, and editing CIM power system models. It creates Python object instances in memory using a data profile exported from a specified CIM profile (e.g. IEC61970cim17v40 or GridAPPS-D RC4_2021)." ] }, { "cell_type": "markdown", "id": "ac891701", "metadata": {}, "source": [ "To install CIMantic Graphs clone the github repository or use pip install: `pip install cim-graph`" ] }, { "cell_type": "markdown", "id": "b0280e92", "metadata": {}, "source": [ "## Table of Contents:\n", "\n", "* [1. CIMantic Graphs Structure](#1-cimantic-graphs-structure)\n", "\n", "* [2. Importing CIMantic Graphs](#2-importing-cimantic-graphs)\n", "\n", "* [3. Specifying Connection Parameters](#3-specifying-connection-parameters)\n", "\n", "* [4. Connecting to the Data Source](#4-connecting-to-the-data-source)\n", "\n", "* [5. Creating a Container and Graph Model](#5-creating-a-container-and-graph-model)\n", "\n", "* [6. Automated Database Query Generation](#6-automated-database-queries)\n", "\n", "* [7. Traversing the Knowledge Graph](#7-traversing-the-knowledge-graph)" ] }, { "cell_type": "markdown", "id": "4860c0b8", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "17deb4d2", "metadata": {}, "source": [ "## CIMantic Graphs Structure\n", "\n", "CIMantic Graphs uses the layered architecture shown below:" ] }, { "cell_type": "markdown", "id": "853342b6", "metadata": {}, "source": [ "![cim-graph-structure](./images/3.9/cim_graph_structure.png)" ] }, { "cell_type": "markdown", "id": "6bca6c88", "metadata": {}, "source": [ "### Database Layer\n", "CIMantic Graphs currently supports the following databases and interfaces:\n", "\n", "* Blazegraph Database\n", "* GraphDB Database\n", "* Neo4J Database\n", "* MySQL Database (in progress)\n", "* GridAPPS-D Platform\n", "* AVEVA PI Asset Framework (in progress)\n", "* RDFLib File Parser\n", "\n", "\n", "* XML Flat Files\n", "* JSON-LD Flat Files (in progress)\n", "* CSV Flat Files (in progress)\n", "\n", "The library uses a unified syntax for all upper-level calls and routines. The databases can be swapped interchangeably by changing the `ConnectionParameters` created during application startup and no other changes to any other application syntax or methods" ] }, { "cell_type": "markdown", "id": "2f713cc3", "metadata": {}, "source": [ "### Data Profile Layer\n", "\n", "CIMantic Graphs is able to support any standard or custom CIM profile. The CIM profile needs to be exported as an XSD data profile / schema. CIMantic Graphs is then able to ingest the data profile and convert all UML classes and attributes to python dataclasses, which power all of the routines and unified API syntax" ] }, { "cell_type": "markdown", "id": "48561db2", "metadata": {}, "source": [ "### API Layer\n", "\n", "CIMantic Graphs offers a breakthrough in terms of ease-of-use through a unified API with two core methods.\n", "\n", "__Access to labeled property graph objects:__\n", "\n", "* `network.graph[cim.ClassName]`: This offers access to a catalog of CIM object instances stored in memory and sorted by class type and mRID forming the named property graph.\n", "\n", "__Universal database query method:__\n", "\n", "* `network.get_all_edges(cim.ClassName)`: This is a universal query method that gets all attributes and all objects one edge away from instances of the specified class. This method works for all CIM classes, CIM profiles, serialization formats, and supported databases." ] }, { "cell_type": "markdown", "id": "f2baa96a", "metadata": {}, "source": [ "### Knowledge Graph Layer\n", "\n", "CIMantic Graphs offers three core knowledge graph classes for handing various kinds of power system models:\n", "\n", "* `BusBranchModel`: Transmission bus-branch models commonly used for planning and power flow studies\n", "\n", "* `NodeBreakerModel`: Transmission node-breaker models commonly used inside energy management systems\n", "\n", "* `FeederModel`: Distribution feeder models with support for single-phase unbalanced networks used in North America.\n", "\n", "Centralized or distributed representations of the power system network model can be used. Centralized models use a single labeled property graph for the network. Distributed models use nested `DistributedArea` class instances to represent the grouping of equipment inside a Substation, VoltageLevel, Bay, Feeder, and switch-delimited topological area inside a combined T+D model." ] }, { "cell_type": "markdown", "id": "7e50e57f", "metadata": {}, "source": [ "### Application Layer\n", "\n", "T+D applications are able to access all of the power system objects through knowledge graph, without any need to connect to the database or perform any custom i/o operations. " ] }, { "cell_type": "markdown", "id": "c82d5ab2", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "225b514c", "metadata": {}, "source": [ "## Importing CIMantic Graphs\n", "\n", "CIMantic Graphs contains several modules within the core library:\n", "\n", "* `data_profiles`: This package contains full CIM profiles exported through [CIMTool](https://github.com/CIMug-org/CIMTool/releases) and provides direct access to CIM dataclasses, their attributes, and usage documentation.\n", "\n", "* `databases`: This package contains database i/o connections and backend query handling for multiple databases (e.g. Blazegraph, GraphDB, Neo4J, MySQL, etc.).\n", "\n", "* `models`: This package contains knowledge graph models for transmission node-breaker, transmission bus-branch, and distribution feeder models.\n", "\n", "* `queries`: This package contains generic queries that are built at runtime for any CIM profile and CIM class.\n", "\n", "* `utils`: This package contains shortcut methods for common routines, such as writing out new files." ] }, { "cell_type": "markdown", "id": "ece42a4f", "metadata": {}, "source": [ "### Importing the CIM Profile\n", "The first step in using CIMantic Graphs is to import the python data profile for desired CIM profile. The data profile provides the ability to invoke CIM classes directly based on their name.\n", "\n" ] }, { "cell_type": "markdown", "id": "84c8e710", "metadata": {}, "source": [ "__Method 1:__ Use `importlib`:" ] }, { "cell_type": "code", "execution_count": 1, "id": "ac2d5fee", "metadata": {}, "outputs": [], "source": [ "import importlib\n", "cim_profile = 'rc4_2021'\n", "cim = importlib.import_module('cimgraph.data_profile.' + cim_profile)" ] }, { "cell_type": "markdown", "id": "a11bf413", "metadata": {}, "source": [ "__Method 2:__ Directly import the desired CIM profile:" ] }, { "cell_type": "code", "execution_count": 2, "id": "f72f6871", "metadata": {}, "outputs": [], "source": [ "import cimgraph.data_profile.rc4_2021 as cim" ] }, { "cell_type": "markdown", "id": "14beafa1", "metadata": {}, "source": [ "### Invoking CIM classes" ] }, { "cell_type": "markdown", "id": "748182ce", "metadata": {}, "source": [ "After importing the data profile, it is possible to create an instance of a class or view the attributes of any class. " ] }, { "cell_type": "markdown", "id": "9ef22b58", "metadata": {}, "source": [ "__Example 1:__ Create a new breaker with a name and mRID. The `uuid` library can be used to generate the unique identifier used for all CIM objects." ] }, { "cell_type": "code", "execution_count": 3, "id": "c80ca66c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"@id\": \"7e72cb38-275c-4369-9c3b-e8b5a42a8703\",\n", " \"@type\": \"Breaker\",\n", " \"name\": \"breaker_01\",\n", " \"open\": \"True\"\n", "}\n" ] } ], "source": [ "breaker = cim.Breaker(name = \"breaker_01\")\n", "breaker.open = True\n", "breaker.pprint()" ] }, { "cell_type": "markdown", "id": "f196508e", "metadata": {}, "source": [ "__Example 2:__ Associate two CIM objects based on their associations" ] }, { "cell_type": "code", "execution_count": 4, "id": "d9cc48f3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"@id\": \"7e72cb38-275c-4369-9c3b-e8b5a42a8703\",\n", " \"@type\": \"Breaker\",\n", " \"name\": \"breaker_01\",\n", " \"EquipmentContainer\": {\n", " \"@id\": \"5d58b92a-5734-4ae7-8f3c-1357e662cf16\",\n", " \"@type\": \"Substation\"\n", " },\n", " \"open\": \"True\"\n", "}\n", "{\n", " \"@id\": \"5d58b92a-5734-4ae7-8f3c-1357e662cf16\",\n", " \"@type\": \"Substation\",\n", " \"name\": \"sub_1\",\n", " \"Equipments\": [\n", " {\n", " \"@id\": \"7e72cb38-275c-4369-9c3b-e8b5a42a8703\",\n", " \"@type\": \"Breaker\"\n", " }\n", " ]\n", "}\n" ] } ], "source": [ "substation = cim.Substation(name = \"sub_1\")\n", "breaker.EquipmentContainer = substation\n", "substation.Equipments.append(breaker)\n", "breaker.pprint()\n", "substation.pprint()" ] }, { "cell_type": "markdown", "id": "28f96a79", "metadata": {}, "source": [ "__Example 3:__ View documentation of the ACLineSegment class" ] }, { "cell_type": "code", "execution_count": 5, "id": "a01ec261", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " A wire or combination of wires, with consistent electrical characteristics,\n", " building a single electrical system, used to carry alternating current\n", " between points in the power system.\n", " For symmetrical, transposed 3ph lines, it is sufficient to use attributes\n", " of the line segment, which describe impedances and admittances for the\n", " entire length of the segment. Additionally impedances can be computed by\n", " using length and associated per length impedances.\n", " The BaseVoltage at the two ends of ACLineSegments in a Line shall have\n", " the same BaseVoltage.nominalVoltage. However, boundary lines may have slightly\n", " different BaseVoltage.nominalVoltages and variation is allowed. Larger\n", " voltage difference in general requires use of an equivalent branch.\n", " \n" ] } ], "source": [ "print(cim.ACLineSegment.__doc__)" ] }, { "cell_type": "markdown", "id": "7f4eec9b", "metadata": {}, "source": [ "-----" ] }, { "cell_type": "markdown", "id": "4dbc75d2", "metadata": {}, "source": [ "## Specifying Connection Parameters\n", "\n", "The next step in using any of the CIMantic Graphs library classes to establish the connection parameters for reading or writing the CIM model. The `ConnectionParameters` class can be imported through:" ] }, { "cell_type": "code", "execution_count": 6, "id": "9a90825b", "metadata": {}, "outputs": [], "source": [ "from cimgraph import ConnectionParameters" ] }, { "cell_type": "markdown", "id": "3c2c59f6", "metadata": {}, "source": [ "The parameters can be defined using an instance specifying the required paramters for the specific connection interface (database) to be used." ] }, { "cell_type": "code", "execution_count": 7, "id": "26a1fe08", "metadata": {}, "outputs": [], "source": [ "params = ConnectionParameters(url = \"http://localhost:8889/bigdata/namespace/kb/sparql\",\n", " cim_profile='rc4_2021', iec61970_301=7) # Blazegraph params" ] }, { "cell_type": "markdown", "id": "20e65385", "metadata": {}, "source": [ "The required and optional arguments for the `ConnectionParameters` class are described in detail in [ConnectionParameters Arguments](link)" ] }, { "cell_type": "markdown", "id": "435aa6c0", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "36445b93", "metadata": {}, "source": [ "## Connecting to the Data Source\n", "\n", "The next step is to connect to the database or file source that will be used for CIM model. A complete list of connection types currently supported are described in [Supported Databases](link)" ] }, { "cell_type": "code", "execution_count": 8, "id": "60f50f57", "metadata": {}, "outputs": [], "source": [ "from cimgraph.databases.blazegraph import BlazegraphConnection\n", "blazegraph = BlazegraphConnection(params)" ] }, { "cell_type": "markdown", "id": "7d1b5e5d", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "2655ebc7", "metadata": {}, "source": [ "## Creating a Container and Graph Model\n", "\n", "CIMantic Graphs uses an EquipmentContainer class as the starting point for building a knowledge graph of the power system model. The supported classes are `BusBranchModel`, `NodeBreakerModel`, and `FeederModel`." ] }, { "cell_type": "code", "execution_count": 9, "id": "9222efed", "metadata": {}, "outputs": [], "source": [ "from cimgraph.models import FeederModel" ] }, { "cell_type": "markdown", "id": "8874aa74", "metadata": {}, "source": [ "The power system network is then created by passing the database connection, container object, and a boolean flag whether a centralized or distributed model should be built." ] }, { "cell_type": "code", "execution_count": 10, "id": "d3038906", "metadata": {}, "outputs": [], "source": [ "feeder_mrid = \"_49AD8E07-3BF9-A4E2-CB8F-C3722F837B62\" # 13 bus\n", "feeder = cim.Feeder(mRID=feeder_mrid)\n", "network = FeederModel(connection=blazegraph, container=feeder, distributed=False)" ] }, { "cell_type": "markdown", "id": "f9a085e4", "metadata": {}, "source": [ "The network is populated by default with all ConductingEquipment, ConnectivityNode, and Terminal class instances. The knowledge graph is indexed by the class type and then the device mRID.\n", "\n", "\n", "To view instances of a particular class, use the `.pprint(cim.ClassName)` method. Use of the default python print method is not recommended due to foward-reverse relationships resulting in an infinite print loop." ] }, { "cell_type": "code", "execution_count": 11, "id": "5f5a1c4c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"52de9189-20dc-4c73-bdee-e960fe1f9493\": {\n", " \"identifier\": \"52de9189-20dc-4c73-bdee-e960fe1f9493\",\n", " \"mRID\": \"52de9189-20dc-4c73-bdee-e960fe1f9493\",\n", " \"Terminals\": \"['1d81c7fe-e88f-41e3-a900-476ca6476ccd', '2847e06b-c8ed-41e6-b515-c61c9e8eb4b4']\"\n", " }\n", "}\n" ] } ], "source": [ "network.pprint(cim.Breaker)" ] }, { "cell_type": "markdown", "id": "15ae92ce", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "90acff5d", "metadata": {}, "source": [ "## Automated Database Queries\n", "CIMantic Graphs contains automatic query generation routines for all classes and all supported databases using the `.get_all_edges(cim.ClassName)` method. This query obtains all attributes for all objects of that class type and expands the knowledge graph by one edge with default instances of all associated objects." ] }, { "cell_type": "code", "execution_count": 12, "id": "a13183df", "metadata": {}, "outputs": [], "source": [ "network.get_all_edges(cim.Breaker)\n", "network.get_all_edges(cim.Terminal)\n", "network.get_all_edges(cim.ConnectivityNode)" ] }, { "cell_type": "code", "execution_count": 13, "id": "8446ad26", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"52de9189-20dc-4c73-bdee-e960fe1f9493\": {\n", " \"identifier\": \"52de9189-20dc-4c73-bdee-e960fe1f9493\",\n", " \"mRID\": \"_52DE9189-20DC-4C73-BDEE-E960FE1F9493\",\n", " \"name\": \"brkr1\",\n", " \"Location\": \"085bbe1f-ff95-4260-a9d2-8d2f1a8ea9a3\",\n", " \"Measurements\": \"['40ac2899-1d2a-469f-a14a-1e14ea29d172', '7c6f94c1-1419-4582-ab2d-a0b11772c26b', '8e7f04ee-a032-4128-838e-a3442a1c3026', 'ab18a8e1-f023-4f9e-bf02-c75bf05164df', 'b393e719-0981-4498-9d96-07f1be7de009', 'f11a9ad9-5b68-483b-b52f-dd4af07bb77d', '0c48c74a-ceee-4c99-bd73-28079561ca7a', '3c60208a-8ef8-483c-828b-30ee42d402dc', '43f80e34-281e-4baa-8aba-d931a9a3ab87', '9f5cb9c4-71d6-4f2b-9dc1-26c7e9f84410', 'aec42b89-f3c0-47e9-b21a-82736b2a1149', 'baccfd49-ec98-4380-8be9-d242dc8611f3']\",\n", " \"EquipmentContainer\": \"_49AD8E07-3BF9-A4E2-CB8F-C3722F837B62\",\n", " \"normalOpen\": \"False\",\n", " \"open\": \"False\",\n", " \"retained\": \"True\",\n", " \"breakingCapacity\": \"400.0\",\n", " \"ratedCurrent\": \"400.0\",\n", " \"BaseVoltage\": \"2a158e0c-cd01-4a50-aeba-59d761fcf15d\",\n", " \"Terminals\": \"['_1D81C7FE-E88F-41E3-A900-476CA6476CCD', '_2847E06B-C8ED-41E6-B515-C61C9E8EB4B4']\"\n", " }\n", "}\n" ] } ], "source": [ "network.pprint(cim.Breaker)" ] }, { "cell_type": "markdown", "id": "3fc53077", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "id": "5b5edc63", "metadata": {}, "source": [ "## Traversing the Knowledge Graph\n", "CIMantic Graphs associates CIM classes by creating direct references between in-memory object instances based on the naming and hierarchy of the underlying information model. " ] }, { "cell_type": "markdown", "id": "f7ca504a", "metadata": {}, "source": [ "To view the attributes of particular object instance, directly invoke the attribute from the UML class diagram." ] }, { "cell_type": "code", "execution_count": 14, "id": "491e5fb1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "False\n" ] } ], "source": [ "from uuid import UUID\n", "breaker = network.graph[cim.Breaker][UUID(\"52de9189-20dc-4c73-bdee-e960fe1f9493\")]\n", "print(breaker.normalOpen)" ] }, { "cell_type": "markdown", "id": "bd732b34", "metadata": {}, "source": [ "To traverse the knowledge graph, no custom queries are required. Instead, directly invoke the UML association names that serves as references between objects in the graph:" ] }, { "cell_type": "code", "execution_count": 15, "id": "b94a9be6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "650\n" ] } ], "source": [ "print(breaker.Terminals[0].ConnectivityNode.name)" ] }, { "cell_type": "markdown", "id": "e8d9b0e9", "metadata": {}, "source": [ "No separate queries or mapping are required for measurment objects. Call the `.get_all_edges` method for each measurement class, and then obtain the measurements from the equipment object" ] }, { "cell_type": "code", "execution_count": 16, "id": "a0599da8", "metadata": {}, "outputs": [], "source": [ "network.get_all_edges(cim.Analog)\n", "network.get_all_edges(cim.Discrete)" ] }, { "cell_type": "code", "execution_count": 17, "id": "dd8c642d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name: Breaker_brkr1_Voltage phase: PhaseCode.A bus: 650\n", "name: Breaker_brkr1_Voltage phase: PhaseCode.B bus: brkr\n", "name: Breaker_brkr1_Voltage phase: PhaseCode.B bus: 650\n", "name: Breaker_brkr1_Voltage phase: PhaseCode.C bus: 650\n", "name: Breaker_brkr1_Voltage phase: PhaseCode.C bus: brkr\n", "name: Breaker_brkr1_Voltage phase: PhaseCode.A bus: brkr\n", "name: Breaker_brkr1_Current phase: PhaseCode.A bus: 650\n", "name: Breaker_brkr1_Current phase: PhaseCode.B bus: 650\n", "name: Breaker_brkr1_Current phase: PhaseCode.C bus: 650\n", "name: Breaker_brkr1_State phase: PhaseCode.A bus: 650\n", "name: Breaker_brkr1_State phase: PhaseCode.C bus: 650\n", "name: Breaker_brkr1_State phase: PhaseCode.B bus: 650\n" ] } ], "source": [ "for meas in breaker.Measurements:\n", " p = meas.phases\n", " print(\"name:\", meas.name, \" phase:\" , meas.phases, \" bus:\", meas.Terminal.ConnectivityNode.name)" ] }, { "cell_type": "markdown", "id": "4c52863d", "metadata": {}, "source": [ "----" ] }, { "cell_type": "markdown", "id": "f21e446d", "metadata": {}, "source": [ "![GridAPPS-D-narrow.png](../images/GridAPPS-D_narrow.png)" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 5 }