Acquisition, Aggregation, and Intelligence – A Hierarchical Visibility Architecture
The purpose of a network visibility architecture is to provide a method of access for security, analysis, and performance tools to access the data traversing your network infrastructure. These appliances are incedibly valuable to your organization, however they are only as effective as the data they are seeing. This post is intended to provide a framework to design a visibility architecture that can support the tools you currently have as well as scale to support the tools of the future.
There are five key considerations that a visibility architecture should address. Along with these considerations, your design should adhere to a well defined, repeatable approach that can be translated across all environments with minimal architecture changes. Adhering to these principles allows for a supportable architecture that is easy to maintain and cost effective to deploy.
The five key considerations:
- Provide shared access
- Wire-speed, reliable transport
The Three Tiers – Acquisition, Aggregation and Intelligence
A good way to visualize the design of your visibility architecture is a hierarchical, three tiered approach. Those tiers are acquisition, aggregation, and intelligence. Each layer defines a separate role within the visibility infrastructure and contributes individually to the five key considerations above. Let’s dive into each role to get a better understanding of it’s place in our architecture.
The acquisition tier could be compared to the access layer in standard network design. This tier provides the connectivity or ‘hooks’ into the network. It’s this point where production network traffic becomes out of band and is actually pulled off the wire for consumption by the remaining layers of your visibility network, and eventually your network tools. The three most common methods of data acquisition are:
- Capture agents
These are the most basic building blocks of your visibility architecture. Without these data inputs, you simply don’t have the required visibility into your network. Conversely, without the supporting and organizational layers below, you simply cannot scale these traffic sources.
The main purpose of the aggregation layer is to prepare data collected in the acquisition layer for consumption by the intelligence layer. This layer should handle input consolidation, source port tagging, and possibly basic filtering. The key focus of this layer is to reduce “cost-per-port” within the visibility network.
Within the aggregation layer, a strong emphasis is placed on input consolidation. Input consolidation is the process of aggregating several input sources (SPANs, taps, etc.) from the acquisition layer into a consolidated output for consumption by the intelligence layer. This process typically happens on lower cost matrix switches with the end goal being a reduced count of ports being used by the higher cost switches that compose the intelligence layer. A typical input consolidation strategy would be consolidating several 1Gb mirror port sources into a single 10Gb output. Another scenario would be aggregating tap ports for a switch uplink into a pair of 10Gb output ports. This is usually achieved via output port over-subscription.
The intelligence layer is the point of our visibility strategy which ties all of the components together. The end result of this tier is to provide data as output to your packet capture appliances, application performance analysis systems, and security tools. At this point, the data should be copied off our network, minimally conditioned, and consolidated into streams that are easy to work with. Within this tier, these core tenants of the acquisition network are taking place:
- Network to tool port mapping
- Masking and slicing
- Time stamping
- Data output
Again, it’s expected that these roles are requiring a higher level of processing from your hardware. This will result in an increase in hardware costs, which is why we try to defer the expense of hardware purchases by doing a consolidation of the traffic within the aggregation layer. Simply put, we’re trying to use cheap matrix switches to do dumb tasks, and save your expensive hardware for the more complicated tasks.
Tying it All Together
In the end, the purpose of a visibility architecture is to provide access to your business data for performance, troubleshooting, and security analysis purposes. Your architecture should be planned to fit within your organizational needs. Considerations such as budget, supportability, and scalability are key in planning for your network design. These design guidelines are written as a framework to assist in the development of a successful visibility architecture.