Insider Surveillance reviews Netcope, a maker of FPGA interface platforms for wire speed packet capture and filtering of 100G networks.
Before getting in line to purchase ISS solutions capable of intercepting traffic on 100G Ethernet networks, it is wise to begin with a sanity check. Just how prevalent are 100G networks? Is the trend real or merely hype?
The short answer is that 100G is indeed real and growing, driven by insatiable demand for bandwidth. At the same time, network architectures are amorphous. Even as networks evolve toward 100G and beyond, many smaller carriers may remain perfectly satisfied with 10G or even 1G to meet customer requirements. Law enforcement and government agencies need to be prepared to conduct intercept and monitoring on a wide of array of networks. Speed and volume of data across networks are issues that aren’t going away. That includes 100G monitoring – the metier of Czech-based Netcope Technologies.
World Class FPGA Adaptors
Netcope claims to be the first company in the world to develop an “FPGA Adaptor” to support, manage or monitor Ethernet traffic at 100G wire speed. Field Programmable Gate Arrays or “FPGAs” are integrated circuits that can be configured by the customer after manufacturing. FPGAs are noteworthy for their ability to capture unstructured data – video, test, images, etc. – at significantly greater speeds, lower energy, logic and cost than conventional ASIC chips. FPGAs are commonly referred to as hardware accelerators.
Netcope makes several products that build on the FPGA Adaptor concept. Each houses one or more FPGAs that capture and filter “all incoming network traffic” at 100G wire speed with zero packet loss. The data may then be fed to analytics solutions such as Deep Packet Inspection, Flow Monitoring or both.
Filtering at 100G is an important asset to such packet analysis capabilities because it intelligently eliminates high volume apps like Netflix and Youtube, which by some estimates consume some 65 percent of data streaming on high-speed networks.
Netcope FPGA Adaptors discard these distractions to ensure that analytic engines such as DPI can focus on data relevant to investigations. Without FPGA in the middle, DPI bogs down and wastes precious time on the irrelevant, hampering access to real-time intelligence. Indeed, without such Adapters filtering data in hardware platforms prior to analysis by software, real timemonitoring of 100G networks without packet loss would not be feasible.
Netcope comes at these capabilities through a distinguished heritage. The company is one of two spin-offs of INVEA-TECH, an enterprise itself inspired and eventually spun off by academic researchers at the University of Brno who helped advance the science of flow monitoring – a solution that gained momentum as data flows on high-speed neworks began to overwhelm conventional packet analysis such as DPI.
As INVEA-TECH grew, it became clear that the company’s technology had a split focus on addressing the problem of packet analysis in 10G, 40G and 100G network environments. One half of that focus was software: flow monitoring analysis based on representative sampling of specific flows. The other was the use of FPGAs to parse and filter the data before it was handed off to CPUs to conduct flow monitoring or DPI.
In 2015 INVEA-TECH decided to split into two separate entities, each dedicated to its specific expertise in managing/tracking streaming data on high-speed networks. The new Flowmon Networks took the software component of packet analysis, and its fraternal twin Netcope Technologies the hardware side.
Both companies continue to work closely together, though independent and free to go their own way. They maintain a close relationship and still co-conduct research with the university. Each contributes its part to solutions used in lawful intercept.
Top Cop Tool – Netcope Session Filter
Netcope offers multiple FPGA-based platforms or “Adaptors” that might be applied to any number of applications: real-time financial trading, cybersecurity or traffic monitoring for signs of criminal and terrorist activities in progress on high-speed networks.
Here we will examine one such solution as used for lawful intercept: the Netcope Session Filter (NSF). The NSF, like other Netcope hardware, uses FPGAs made by Xilinx, typically the Virtex-7.
Whichever adaptor is used, the principle at work is the same: the FPGA board offloads traffic direct from the network to pre-process it before the data is handed off to CPU space dedicated to either DPI or Flow Analysis.
Taking a cue from Flowmon’s approach, NSF views network traffic as being comprised of separate flows numbering in the thousands – each with uniquely identifiable features. Communications in a flow typically share traits such as commonality of IP addresses, or similarities at the application layer.
Filtering traffic as IP flows in hardware first enables the system to single out specific flows of interest to the agent for further processing and analysis. This is more efficient that conducting sending each packet straight to DPI or other processing. NSF discards traffic of no interest – e.g., Netflix – and focuses on flows most likely to contain evidence.
The user can program NSF to look for flows with specific characteristics based on statistical data common to each type of flow: IP addresses, ports, Level 4 protocol, type of app indicated at Level 7, beginning and ending timestamps, or the number of transferred packets and bytes.
Depending on the requirements of the next phase of analysis – by software – Netcope’s filter can forward designated IP flows in whatever format is required for further analysis: headers only, whole packets, or packets shortened to a predetermined length.
Still, even when traffic is qualified and “downsized” on Netcope hardware, isn’t it still a great deal of volume to transfer into CPU processing? It could be. Netcope’s solution is bifurcation, using a single Xilinx FPGA to manage sustained traffic flows, full duplex in 100G Ethernet.
Measuring Market Demand
According to research by IHS Markit, shipment of 100G ports to operators doubled in 2015 and increased another 30 percent to some US $8.8 billion in 2016. Demand was driven by carriers currently running at 10G who decided to skip an incremental rise to 40G and go straight to 100G.
100G growth is expected to rise steadily through 2020 to roughly US $13 billion, again at the expense of 40G.
Not surprisingly, demand for 1G has peaked as carriers look for faster alternatives. However, the largest sector is – and through 2020 shall remain – 10G, which will grow from a healthy US 19.0 billion marketplace today to over US $27 billion by 2020, more than double the expected investment in 100G network that same year.
Who is investing in 100G? Not everyone. Using just the U.S. as an example, hundreds of smaller CSPs serving rural or remote areas in the United States are perfectly satisfied with 1G and aren’t even making the switch to 10G as yet, let alone 40G or 100G. Large network operators in populous nations, on the other hand, most certainly are making the move to 100G.
For LEAs that operate in mainstream population centers, owning ISS solutions that can filter target data from 100G networks is a must-do. As Netcope correctly observes, with the demand for high definition video for entertainment purposes, 100G is the new network standard. Whether to manage the network or protect it from malicious activity, service providers will require real time DPI. But sheer data volume alone will make real time DPI via software totally infeasible. The same challenges apply to the monitoring of 100G by law enforcement.
Netcope is in the right place at the right time with proven solutions. When in the market for wire speed packet capture, Netcope Technologies should be on the short list.
Insider Surveillance rating for Netcope Technologies: 4.5 Stars