Rowell Dionicio

Get Techie With It

devasc

Parsing XML with Python Minidom

By 1 Comment

A core skill for a DevNet associate is being knowledgeable of how to parse XML into a Python structure.

Data can be retrieved from devices using different data structures such as XML, YAML, or JSON.

In this lab, I’ll look at parsing XML into a useable structure within Python.

First step is to use a Python script to send an HTTP request to our device so we can obtain data which will be returned in XML format.

Cisco provides us with a sandbox to test with using this Python script through their Coding 201 Parsing XML lab. I’ve had to modify it a little to ensure I can ignore the certificate verification. And I had to select another Sandbox due to authorization issues.

from urllib.request import Request, urlopen
import ssl 

req = Request('https://msesandbox.cisco.com/api/contextaware/v1/maps/info/DevNetCampus/DevNetBuilding/DevNetZone')
req.add_header('Authorization', 'Basic bGVhcm5pbmc6bGVhcm5pbmc==')

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

r = urlopen(req, context=ctx)
rString = r.read().decode("utf-8")

print(rString)
r.close()

The following is the output from our UGLY request:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><Floor objectVersion="19" name="DevNetZone" isOutdoor="false" floorNumber="1" floorRefId="723413320329068590"><Dimension length="81.9" width="307.0" height="16.5" offsetX="0.0" offsetY="0.0" unit="FEET"/><Image imageName="domain_0_1421088463647.png"/><GPSMarker name="GPS_Marker_17"><GeoCoordinate latitude="36.125859" longitude="-97.066969" unit="DEGREES"/><MapCoordinate x="0.6" y="0.6" unit="FEET"/></GPSMarker><GPSMarker name="GPS_Marker_18"><GeoCoordinate latitude="36.125859" longitude="-97.06595" unit="DEGREES"/><MapCoordinate x="299.77" y="0.6" unit="FEET"/></GPSMarker><GPSMarker name="GPS_Marker_19"><GeoCoordinate latitude="36.125641" longitude="-97.066969" unit="DEGREES"/><MapCoordinate x="0.6" y="80.09" unit="FEET"/></GPSMarker><AccessPoint name="T1-3" radioMacAddress="00:2b:01:00:04:00" ethMacAddress="00:2b:01:00:04:f0" ipAddress="10.10.20.243" numOfSlots="2" apMode="LOCAL"><MapCoordinate x="155.28" y="57.57" unit="FEET"/><ApInterface band="IEEE_802_11_B" slotNumber="0" channelAssignment="1" channelNumber="1" txPowerLevel="1" antennaPattern="Internal-1140-2.4GHz" antennaAngle="1.57" 
**TRUNCATED**

As I’ve described in a previous post, XML is just another data structure. It’s commonly used in other network equipment such as Juniper and Palo Alto Networks.

In order to parse XML data we will need to import a library. There are two libraries we can use, but in this post we’ll look at Minidom:

Parsing XML with Minidom

Minidom is a Python library called Minimal DOM. It’s a minimal implementation of the Document Object Model interface.

To import the library just add the following to the top of the script:

import xml.dom.minidom

Now we modify our script so we can parse the returned XML data.

We’ll add the following lines:

xmlparse = xml.dom.minidom.parseString(rString)
prettyxml = xmlparse.toprettyxml()
print(prettyxml)

The parse() function in xml.dom.minidom.parseString(rString) to parse out the XML data and assign it to xmlparse.

Next, we use a DOM function to pass xmlparse into toprettyxml to make a pretty-printed version of the XML output we just saw.

The following output now looks more readable. We can see the root node, nests, etc.:

<?xml version="1.0" ?>
<Floor floorNumber="1" floorRefId="723413320329068590" isOutdoor="false" name="DevNetZone" objectVersion="19">
    <Dimension height="16.5" length="81.9" offsetX="0.0" offsetY="0.0" unit="FEET" width="307.0"/>
    <Image imageName="domain_0_1421088463647.png"/>
    <GPSMarker name="GPS_Marker_17">
        <GeoCoordinate latitude="36.125859" longitude="-97.066969" unit="DEGREES"/>
        <MapCoordinate unit="FEET" x="0.6" y="0.6"/>
    </GPSMarker>
    <GPSMarker name="GPS_Marker_18">
        <GeoCoordinate latitude="36.125859" longitude="-97.06595" unit="DEGREES"/>
        <MapCoordinate unit="FEET" x="299.77" y="0.6"/>
    </GPSMarker>
    <GPSMarker name="GPS_Marker_19">
        <GeoCoordinate latitude="36.125641" longitude="-97.066969" unit="DEGREES"/>
        <MapCoordinate unit="FEET" x="0.6" y="80.09"/>
    </GPSMarker>
    <AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:04:f0" ipAddress="10.10.20.243" name="T1-3" numOfSlots="2" radioMacAddress="00:2b:01:00:04:00">
        <MapCoordinate unit="FEET" x="155.28" y="57.57"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="0" antennaPattern="Internal-1140-2.4GHz" band="IEEE_802_11_B" channelAssignment="1" channelNumber="1" slotNumber="0" txPowerLevel="1"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="11" antennaPattern="Internal-1140-5.0GHz" band="IEEE_802_11_A" channelAssignment="1" channelNumber="64" slotNumber="1" txPowerLevel="5"/>
    </AccessPoint>
    <AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:05:f0" ipAddress="10.10.20.244" name="T1-4" numOfSlots="2" radioMacAddress="00:2b:01:00:05:00">
        <MapCoordinate unit="FEET" x="213.6" y="12.6"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="0" antennaPattern="Internal-1140-2.4GHz" band="IEEE_802_11_B" channelAssignment="1" channelNumber="1" slotNumber="0" txPowerLevel="1"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="11" antennaPattern="Internal-1140-5.0GHz" band="IEEE_802_11_A" channelAssignment="1" channelNumber="64" slotNumber="1" txPowerLevel="5"/>
    </AccessPoint>
    <AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:06:f0" ipAddress="10.10.20.245" name="T1-5" numOfSlots="2" radioMacAddress="00:2b:01:00:06:00">
        <MapCoordinate unit="FEET" x="253.7" y="58.48"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="0" antennaPattern="Internal-1140-2.4GHz" band="IEEE_802_11_B" channelAssignment="1" channelNumber="1" slotNumber="0" txPowerLevel="1"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="11" antennaPattern="Internal-1140-5.0GHz" band="IEEE_802_11_A" channelAssignment="1" channelNumber="64" slotNumber="1" txPowerLevel="5"/>
    </AccessPoint>
    <AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:03:f0" ipAddress="10.10.20.242" name="T1-2" numOfSlots="2" radioMacAddress="00:2b:01:00:03:00">
        <MapCoordinate unit="FEET" x="98.1" y="11.7"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="0" antennaPattern="Internal-1140-2.4GHz" band="IEEE_802_11_B" channelAssignment="1" channelNumber="1" slotNumber="0" txPowerLevel="1"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="11" antennaPattern="Internal-1140-5.0GHz" band="IEEE_802_11_A" channelAssignment="1" channelNumber="64" slotNumber="1" txPowerLevel="5"/>
    </AccessPoint>
    <AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:02:f0" ipAddress="10.10.20.241" name="T1-1" numOfSlots="2" radioMacAddress="00:2b:01:00:02:00">
        <MapCoordinate unit="FEET" x="43.9" y="57.88"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="0" antennaPattern="Internal-1140-2.4GHz" band="IEEE_802_11_B" channelAssignment="1" channelNumber="1" slotNumber="0" txPowerLevel="1"/>
        <ApInterface antennaAngle="1.57" antennaElevAngle="0.0" antennaGain="11" antennaPattern="Internal-1140-5.0GHz" band="IEEE_802_11_A" channelAssignment="1" channelNumber="64" slotNumber="1" txPowerLevel="5"/>
    </AccessPoint>
    <LocationFilterRegion regionType="OUTSIDE">
        <MapCoordinate unit="FEET" x="0.0" y="0.0"/>
        <MapCoordinate unit="FEET" x="307.0" y="0.0"/>
        <MapCoordinate unit="FEET" x="307.0" y="81.9"/>
        <MapCoordinate unit="FEET" x="0.0" y="81.9"/>
    </LocationFilterRegion>
</Floor>

Now that we have our request we need to identify what we’re looking for. I want to get information about access points.

I do see a sub-object, AccessPoint, which contains attributes of the access point element:

<AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:03:f0" ipAddress="10.10.20.242" name="T1-2" numOfSlots="2" radioMacAddress="00:2b:01:00:03:00"></AccessPoint>

Let’s grab information on each the access point’s elements of name, ethMacAddress, and ipAddress.

Time to parse through the XML data and get only what we need.

access_points = xmlparse.getElementsByTagName('AccessPoint')
for access_point in access_points:
    ap_name = access_point.getAttribute('name')
    ap_mac = access_point.getAttribute('ethMacAddress')
    ap_ip = access_point.getAttribute('ipAddress')
    print(access_point.tagName + ': ' + ap_name + '\t mac: ' + ap_mac + '\t ip: '+ ap_ip)

What does this do?

Let’s analyze what each line does 🤔

access_points = xmlparse.getElementsByTagName('AccessPoint') – With minidom it is possible to walk through each child node tree. That’s what we’re doing here with xmlparse.getElementsByTagName(‘AccessPoint’). We’re going to find each child of the name AccessPoint.

Next we’ll get into a for loop to cycle through any of the child nodes we’re looking for. In this case, AccessPoint.

Within the for loop there are three variables: ap_name, ap_mac, and ap_ip.

We’re going to use a Minidom element object, getAttribute, to return the value of the attribute named.

access_point.getAttribute(‘name’) – Through each child node with a tag of AccessPoint we want it to return the Name of that access point and assign that value in ap_name.

access_point.GetAttribute(‘ethMacAddress’) – We’re going to return the MAC Address of the access point under the tag of ethMacAddress, if it exists, and assign it to ap_mac.

access_point.getAttribute(‘ipAddress’) – The next attribute I want to collect is the IP address. If returned, it will be assigned to ap_ip.

Next, I want to visualize that information on the screen. With the print statement, print(access_point.tagName + ': ' + ap_name + '\t mac: ' + ap_mac + '\t ip: '+ ap_ip), we’re going to display some of the attributes we found.

The output is much more appealing now:

% python parsing_xml_with_python.py
AccessPoint: T1-3     mac: 00:2b:01:00:04:f0  ip: 10.10.20.243
AccessPoint: T1-4     mac: 00:2b:01:00:05:f0  ip: 10.10.20.244
AccessPoint: T1-5     mac: 00:2b:01:00:06:f0  ip: 10.10.20.245
AccessPoint: T1-2     mac: 00:2b:01:00:03:f0  ip: 10.10.20.242
AccessPoint: T1-1     mac: 00:2b:01:00:02:f0  ip: 10.10.20.241

Final Script

from urllib.request import Request, urlopen
import ssl
import xml.dom.minidom

req = Request('https://msesandbox.cisco.com/api/contextaware/v1/maps/info/DevNetCampus/DevNetBuilding/DevNetZone')
req.add_header('Authorization', 'Basic bGVhcm5pbmc6bGVhcm5pbmc==')

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

r = urlopen(req, context=ctx)
rString = r.read().decode("utf-8")

#print(rString)

xmlparse = xml.dom.minidom.parseString(rString)
prettyxml = xmlparse.toprettyxml()
#print(prettyxml)

access_points = xmlparse.getElementsByTagName('AccessPoint')
for access_point in access_points:
	ap_name = access_point.getAttribute('name')
	ap_mac = access_point.getAttribute('ethMacAddress')
	ap_ip = access_point.getAttribute('ipAddress')
	print(access_point.tagName + ': ' + ap_name + '\t mac: ' + ap_mac + '\t ip: '+ ap_ip)

r.close()

Final Thoughts

I just went through parsing XML data using Python’s Minidom library. I thought it was straightforward with Python’s documentation clearly defining not just Minidom but also the Element Objects.

The Element Object of Element.getAttribute helps us narrow down all the XML data to just the data we need (parsing).

Now, I’d like to try to figure out how I can do this against my Cisco C9800-CL

What is YANG – For Network Engineers

By Leave a Comment

YANG – Yet Another Next Generation. Network engineers are used to hearing Next Generation. For example, Next Generation Firewalls and Next Generation Wireless.

But in this case, I’m talking about a data modeling language. It’s used with something like NETCONF and RESTCONF. YANG is used to describe networking data models and defines a structure.

YANG was developed by the IETF. Read RFC 6020 to get all the details of how it works. API requests use NETCONF or RESTCONF to make configuration changes and review the configuration.

Building blocks of YANG

  • module – top of the hierarchy of nodes
  • containers – related nodes
  • lists – identifies nodes
  • leaf – individual attributes of a node
  • type – every leaf has an associated type

Here’s an example:

  • Employee
    • Employee ID
    • SSN
    • Gender
    • Address
    • Birthday

A data model could be about a network device or service. A network device will have containers and leafs about its interfaces, statistics, VLANs, etc. A YANG model on a service may be ACLS, VRFs, BGP, etc.

To explore the YANG models pyang can be used. Pyang will validate and convert YANG modules to different formats.

I’m going to be using using my DevNet lab files to look into pyang further.

We’ll explore a file on my desktop, ietf-interfaces.yang. If you’re going through the DevNet classes then you will also learn about YANG. Some of the examples are taken from DevNet.

YANG Model

To view the the YANG model, I’m going to issue a command in iTerm2, pyang, to display the tree.

I was able to get a copy of the YANG model from the DevNet labs. In one lab I used git to download a whole folder of YANG models. On my computer I use pyang to view details of those YANG models.

Taking a look at ietf-interfaces.yang, there are two containers, created called interfaces and interfaces-state.

ietf-interfacesis a module of the container. Inside a container is a list. Within the list is the leaf, which are attributes of the individual node items.

You may notice there are rw and ro in front of the name.

rw = read/write
ro = read-only

In the example below, ietf-interfaces is the module. interfaces will be the container, consisting of interface leaf with attributes such as the interface name, description, and type.

To the right of the leaf describes what type of attribute it is.

$ pyang -f tree ietf-interfaces.yang
module: ietf-interfaces
  +--rw interfaces
  |  +--rw interface* [name]
  |     +--rw name                        string
  |     +--rw description?                string
  |     +--rw type                        identityref
  |     +--rw enabled?                    boolean
  |     +--rw link-up-down-trap-enable?   enumeration {if-mib}?
  |     +--ro admin-status                enumeration {if-mib}?
  |     +--ro oper-status                 enumeration
  |     +--ro last-change?                yang:date-and-time
  |     +--ro if-index                    int32 {if-mib}?
  |     +--ro phys-address?               yang:phys-address
  |     +--ro higher-layer-if*            interface-ref
  |     +--ro lower-layer-if*             interface-ref
  |     +--ro speed?                      yang:gauge64
  |     +--ro statistics
  |        +--ro discontinuity-time    yang:date-and-time
  |        +--ro in-octets?            yang:counter64
  |        +--ro in-unicast-pkts?      yang:counter64
  |        +--ro in-broadcast-pkts?    yang:counter64
  |        +--ro in-multicast-pkts?    yang:counter64
  |        +--ro in-discards?          yang:counter32
  |        +--ro in-errors?            yang:counter32
  |        +--ro in-unknown-protos?    yang:counter32
  |        +--ro out-octets?           yang:counter64
  |        +--ro out-unicast-pkts?     yang:counter64
  |        +--ro out-broadcast-pkts?   yang:counter64
  |        +--ro out-multicast-pkts?   yang:counter64
  |        +--ro out-discards?         yang:counter32
  |        +--ro out-errors?           yang:counter32
  x--ro interfaces-state
     x--ro interface* [name]
        x--ro name               string
        x--ro type               identityref
        x--ro admin-status       enumeration {if-mib}?
        x--ro oper-status        enumeration
        x--ro last-change?       yang:date-and-time
        x--ro if-index           int32 {if-mib}?
        x--ro phys-address?      yang:phys-address
        x--ro higher-layer-if*   interface-state-ref
        x--ro lower-layer-if*    interface-state-ref
        x--ro speed?             yang:gauge64
        x--ro statistics
           x--ro discontinuity-time    yang:date-and-time
           x--ro in-octets?            yang:counter64
           x--ro in-unicast-pkts?      yang:counter64
           x--ro in-broadcast-pkts?    yang:counter64
           x--ro in-multicast-pkts?    yang:counter64
           x--ro in-discards?          yang:counter32
           x--ro in-errors?            yang:counter32
           x--ro in-unknown-protos?    yang:counter32
           x--ro out-octets?           yang:counter64
           x--ro out-unicast-pkts?     yang:counter64
           x--ro out-broadcast-pkts?   yang:counter64
           x--ro out-multicast-pkts?   yang:counter64
           x--ro out-discards?         yang:counter32
           x--ro out-errors?           yang:counter32

YAML Basics for DevNet

By 1 Comment

Taking a look at the Cisco DevNet Associate exam topics (DEVASC 200-901), one of the first objectives is to compare different data formats. Those data formats you should be able to compare are XML, JSON, and YAML.

To start, I decided to take a look at YAML and what it has to offer from a very basic overview. I’m very new when it comes to DevOps or NetDevOps but this will become my notes of sorts as I look into the DevNet Associate certification.

Studying for the Cisco DevNet Associate Certification?

Let me know what your coding experience is in the comments below.

YAML is a human-readable structured data format which stands for Stands for Yet Another Markup Language or now YAML Ain’t Markup Language, depending on where you look 😉 The official documentation page states the latter.

Goals of YAML

  • Data serialization standard for programming languages.
  • Human-readable structured data format.
  • Portable between programming languages.
  • Consistent model.
  • Expressive and extensible.
  • Easy to implement and use.

A quick overview of YAML

  • Writable and readable.
  • Case sensitive
  • Files end in .yaml or .yml
  • Spaces are used instead of tabs (because various tools treat tabs differently)
  • Indentation is used for structure
  • Colons separate key-value pairs
  • Dashes create bullet lists
  • The start of a YAML file begins with three dashes

Data structures:

  • Mappings (hashes/dictionaries)
  • Sequences (arrays/lists)
  • Scalars (strings/numbers)

Working with Blocks
Block collections use indentation for scope and begin each entry on its new line. Block sequences use a dash and space to indicate each entry. Mappings use a colon and space to mark a key-value pair.

To create a comment, you must use a hash tag.

Sequence
A sequence entry will be denoted by a dash -.

 - netsw-idf-01
 - netsw-idf-02
 - netsw-idf-03

Scalar
A scalar is just a single value. They can be strings, numbers, Boolean, or null values.

# scalars can be quoted
“GigabitEthernet1/0/1”

# scalars can also be unquoted
GigabitEthernet1/0/2

Mapping Scalars to Sequences
Also creates an array.

routers:
    - rtr-idf-01
    - rtr-idf-02
switches:
    - netsw-idf-01
    - netsw-idf-02

Sample Inventory in YAML

—--
sites:
    - name: san-jose # This is a comment. Begin block 1.
        hosts:
            - hostname: r1
              host: 192.168.1.1
              os: cisco_ios
            - hostname: sw1
              host: 192.168.1.5
              os: cisco_ios
            - hostname: sw2
              host: 192.168.1.6
              os: junos

    - name: san-diego # This is a comment. Begin block 2.
        hosts:
            - hostname: r2
              host: 192.168.2.1
              os: cisco_ios
            - hostname: sw1
              host: 192.168.2.5
              os: cisco_ios

Have we created a list or a dictionary? List items which is a dictionary.

Here’s a sample YAML script used to configure a VLAN on a Cisco switch.

---
- hosts: cisco
  connection: network_cli
  tasks:
    - nxos_vlan:
      vlan_id: 10
      name: STORAGE