The Era of Network-Modeling

Posted On Mar-12

It’s 2016. In contrast to the Networks of yester-years, Enterprise Networks can no longer be summarized as just a couple of rows of cabinets in a datacenter.

Today, when we say large-scale enterprises, we’re referring to a few hundred rows , multiple datacenters, several clouds, routers providing connectivity between different vrfs, and wan conections to a few different nodesites. At these scales, the need to represent the network in its entirety as a data-model, quickly becomes obvious.

State vs. Intent

At its simplest form, models are nothing more than a unified representation of the state of all protocols and neighborships in the Network. The idea is to keep this ‘network-state’ always updated.

Then you’d subscribe to events that are triggered when the above state-model updates. To each event, you’d react by verifying that the latest network state aligns with the expected-state. So this necessiates the maintenance of a separate ‘expected-state’ topology. Vendors , in 2016, have decided to name the expected state, the ‘Intent’, and the active-state the ‘State’.

The ‘intent’ and the ‘state’ together make up the core of Network-Modeling.

Monitor the State, Provision the Intent

In a stable Network, the state and intent are converged. From the perspective of Monitoring, the State is always the source-of-truth. Anything that should be monitored pertaining to the Network, is made part of the State-collection – be it OSPF adjacencies, BGP Summary, LLDP neighbors, Muticast State, just about anything.

Monitoring scripts then start working on this state, and diff-ing it against the Intent. Any divergence between the two indicates instability in the Network.

Similarly, Provisioning tools make changes only to the Intent. They don’t directly touch Network devices. The modified Intent then propagates out to the Network – and while doing so, automatically picks the best transport mechanism for propagation – falls back from most favorable to least favorable transport ( Think of it as cascading down from Netconf XML RPCs, thru YANG/Tail-F models, down to plain old SSH-CLI commands.) The idea is that all of that dirty transport-work is abstracted from the engineer, who only deals with writing tools to interact with the Intent.

What does abstraction provide?

Until recently, Provisioning tools interacted directly with the Network. But there are several gaps with that approach. For starters, the lack of any abstraction between provisioning and the actual propagation of candidate config out to the network, makes it nearly impossible to predict what the impact of config change might have on the Network topology as a whole. Maintaining a separate ‘Intent’ lets you run algorithms to try and predict this.

Also, any modification made to the intent, would be reflected in the following state-collection. So this forms the basis of a feedback-system, to check if changes have propagated out to the network successfully, with the intended results. All of these are critical in automating functions involved in managing an Enterprise Network.

The future of Network Models

Provisioning used to be through the CLI. And Monitoring was synonymous with SNMP. Welcome to 2016 – CLI is plain painful. SNMP’s gotta go. And we’re starting to run agents directly on Network equipment to facilitate both Provisioning and Monitoring, in smart and efficient ways.

Cisco has made onePK available publicly and NXAPI improves accessibility and provisioning of Nexus devices . Arista’s EOS comes with excellent EEM-functionality, along with eAPI that can do wonderful things. With the newest EOS release, they even stream such events to remote workstations, which can then react to sysdb changes. Juniper lets you run agents on Junos as well, and subscribe to events.

Think about it for a second – You dont have to ‘poll’ for MAC table changes, or LANZ messages or counters anymore. You just subscribe to these events, and only react when you recieve an event. Provisioning takes place through RPCs with well defined YANG models. Perhaps OpenConfig will gain more steam. Eventually, Networks will provision themselves, Monitor themselves, and hopefully heal themselves (atleast in part). This is the way we all envisioned Network Automation to be!

fastsnmpy

Posted On Jan-17

Superfast parallel snmpbulkwalk

Fastsnmpy is a module that leverages python-extensions that come with net-snmp and provide highly parallelized faster-methods to walk oid-trees on devices.

In addition, it provides a method to bulkwalk mib-trees..

BulkWalk methods are missing from native python-bindings for net-snmp. By creating a wrapper around the GetBulk method instead, and maintaining state while traversing the oid-tree, fastsnmpy provides a clever solution to bulkwalk oids much faster

FastSNMPy provides the following:

  • snmpwalk(): Native python-bindings distributed with net-snmp, combined with fastsnmpy’s ability to parallelize snmpwalk operations.
  • snmpbulkwalk(): Ability to snmpbulkwalk devices, which makes it several magnitudes faster than net-snmp’s implementation of snmpwalk.
    • By leveraging the getbulk method, this module provides a quick snmpbulkwalk utility.
  • PROCESS-POOLS: By passing in a ‘workers=n’ attribute to the above methods, fastsnmpy can instantiate a process-pool to parallelize the snmpwalk and snmpbulkwalk methods, resulting in several devices being walked at the same time, effectively using all cores on a multicore machine.
  • One-Line, and Two-Line scripts that enable you to discover/walk all devices in a whole datacenter

Quick example – Running in interactive mode

Python 2.7.10 (default)

<blockquote>
  >> import netsnmp
  >> from fastsnmpy import SnmpSession
 
  >> hosts =['c7200-2','c7200-1','c2600-1','c2600-2']
  >> oids = ['ifDescr', 'ifIndex', 'ifName', 'ifDescr']
  >> newsession = SnmpSession ( targets = hosts, oidlist = oids, community='oznet' )
 
  >> results = newsession.snmpbulkwalk(workers=15)
  >> len(results)
  171031
  >>

Note: To use the module in scripts, please see the example.py included with the package.

Benchmarks

(1) Walking 30 nodes for ifDescr using snmpwalk():

time ./fastsnmpy.py
real    0m18.63s
user    0m1.07s
sys     0m0.38s

(2) Walking 30 nodes for ifDescr using snmpbulkwalk():

time ./fastsnmpy.py
real    0m9.17s
user    0m0.48s
sys     0m0.11s

(3) Walking 30 nodes for ifDescr using snmpwalk(workers=10):

time ./fastsnmpy.py
real    0m2.27s
user    0m2.87s
sys     0m0.66s

(4) Walking 30 nodes for ifDescr using snmpbulkwalk(workers=10):

time ./fastsnmpy.py
real    0m0.90s
user    0m2.44s
sys     0m0.40s

As you can see, fastsnmpy’s bulkwalk mode is almost 20 times faster than using python’s native snmp bindings for walking

Latest-version: Fastsnmpy2-1.2.1

  Download Here

Get from Git

Or fork from the git-repo GitHub-FastSNMPy