Onboarding IE3400 as a Virtual Edge Node

Elisity supports the onboarding of Cisco Catalyst IE3400 as a Virtual Edge Node for policy enforcement. This document details how to onboard an IE3400 into Elisity Cloud Control Center.

 

Elisity announced support for the Cisco Catalyst IE3400 series switch and IEM3400 switch modules in Q4 of 2022. Due to platform limitations on the IE3400 the deployment model is unique and requires a Virtual Edge Identity Agent to be hosted locally by the IE3400. This agent assists in metadata collection, efficient packaging and transmission to the Elisity Virtual Edge for further processing and forwarding to Cloud Control Center. 

Catalyst IE3400 can either be onboarded with a Virtual Edge hosted as a VM on a hypervisor, or with a Virtual Edge hosted on a switch that supports application hosting (ie. Catalyst 9400). To learn more about our Virtual Edge deployment models read this article. 

The following diagram depicts the supported deployment architectures mentioned above. 

 

The following chart describes the terminology used in this document

Cloud Control Center	Elisity's cloud native and cloud delivered control, policy and management plane.
Virtual Edge	The Elisity software running as a docker container on either a hypervisor such as VMware ESXi or on a switch that supports Application Hosting.
Virtual Edge Node	An access switch onboarded to a Virtual Edge to be leveraged as an enforcement point in the network.
Virtual Edge Identity Agent	Lightweight Elisity application that runs on the IE3400 application hosting container space and assists in metadata collection, efficient packaging and transmission to the Elisity Virtual Edge

 

Onboarding Catalyst IE3400 as a Virtual Edge Node

Before we get started, we need to familiarize ourselves with how the Virtual Edge Identity Agent hosted on the IE3400 communicates with the switch itself as well as with the Virtual Edge. 

The Virtual Edge Identity Agent hosted by the IE3400 has two interfaces: guest-interface 0 and guest-interface 1. The first guest interface is used solely to receive identity metadata from the switch, while the second guest interface is used to communicate back to the Virtual Edge. 

The IE3400 must have a dedicated Identity VLAN and VLAN interface for communication with the Virtual Edge Identity Agent. This VLAN cannot be used for any other purpose on the switch and must not be carried on any interface other than the AppGigEthernet interface. This VLAN and VLAN interface communicates with the Virtual Edge Identity Agent's guest-interface 0 and must be on the same network.

There must also be another VLAN and gateway (local or remote) available that provides a routed or switched path to the Virtual Edge. This must be in the same network as guest-interface 1 on the Virtual Edge Identity Agent.

Lastly, the IE3400 leverages a virtual interface called AppGigabitEthernet to trunk both VLANs to the container space where the Virtual Edge Identity Agent is hosted and must be configured as a trunk. The AppGigabitEthernet interface is numbered based on whether or not an additional IEM module is installed or not. Typically AppGigabitEthernet1/1 is the correct interface when no IEM module is installed. 

The following diagram depicts the details explained above.

Step 1: Make sure the IE3400 you wish to onboard with the Virtual Edge has the following commands configured.

iox
ip http authentication local
ip http secure-server
restconf

Step 2:  You should either have a user account with privilege 15 configured or TACACS login configured to provide privilege 15 level access. This is needed for the Virtual Edge to authenticate with the IE3400. Execute the following command under global configuration mode if a local account is being used and is not already configured:

switch(config)# username <username> privilege 15 secret 0 <password>

Add the following commands to your IE3400 configuration if using TACACS

switch(config)# aaa authentication login HTTP_AUTH group <group name> local
switch(config)# ip http authentication aaa login-authentication HTTP_AUTH

Step 3: Copy the Virtual Edge Identity Agent .tar file to the IE3400 internal flash using SCP, FTP or whatever method you prefer. 

You can start onboarding Virtual Edge Nodes in two ways.

Method 1: Select your Virtual Edge and Add a VEN
Go to the Virtual Edge dashboard in Cloud Control Center, select the Virtual Edge you would like to use as the parent for the Virtual Edge Node you are about to onboard. 

After clicking on the VE, you can click on Add Virtual Edge Node and select Add Single Virtual Edge Node.

 

Method 2: Onboard VENs Directly from the Virtual Edge Node panel.

Select the Virtual Edge Node tab in the bottom menu, and select Add Virtual Edge Node then select Add Single Virtual Edge Node. Note that adding a VEN from this screen still requires you to select a Virtual Edge as a parent.

If creating a Virtual Edge Node from this screen, you will need to select the parent Virtual Edge. You can search for a VE, sort by site label, and apply custom filters to find the exact Virtual Edge you would like to onboard nodes to. 

After selecting a Virtual Edge, we need to fill out the details for the Virtual Edge Node. Here is a summary of both the required and optional fields.

The following chart provides details about each field in the VEN onboarding workflow.

Switch Management IP	This is the management IP of the switch you wish to onboard as a Virtual Edge Node for policy enforcement. This can be an IP as long as it is reachable by the previously deployed Virtual Edge container. This field is mandatory.
Description	This allows a user-defined description to be configured for the VEN. This field is optional.
**Enable Enhanced Endpoint Discovery	Selecting this option enables the active collection of identifying data for endpoints discovered behind a VEN, gleaned from access switch telemetry. This feature actively tracks assets for updates in identifying data. (Recommended) This setting is autoconfigured per switchport if enabled during this onboarding. You can choose to enable autoconfiguration or manually configure after onboarding by leaving this box unchecked. The logic for this autoconfiguration is discussed later in this article.
**Enable Flow Telemetry	Selecting this option enables the collection of flow data and network traffic analytics that are sent to Cloud Control Center. (Recommended) This setting is autoconfigured per switchport if enabled during this onboarding. You can choose to enable autoconfiguration or manually configure after onboarding by leaving this box unchecked. The logic for this autoconfiguration is discussed later in this article.
**Enable Passive Endpoint Discovery	Selecting this option enables the passive collection of identifying data using data plane telemetry about endpoints discovered behind a VEN. This is a global setting per VEN. (Recommended) You can choose to enable this setting later by leaving this box unchecked.
Site Label	Site labels can be applied to Virtual Edge Nodes for policy distribution and for analytics purposes. Site labels are used to assign Virtual Edges and Virtual Edge Nodes to Policy Sets. If this field is left blank, the site label from the parent Virtual Edge is inherited, if it exists.
Switch Admin Username	If not using global admin credentials, this is the admin username of the switch you wish to onboard as a Virtual Edge Node for policy enforcement. This can either be local or TACACS/RADIUS. Privilege 15 is required. This field is mandatory.  Username should be alphanumerical and may contain only permitted special characters (_, +, \\\\, /, -).'}
Switch Admin Password	If not using global admin credentials, this is the admin password of the switch you wish to onboard as a Virtual Edge Node for policy enforcement. This can either be local or TACACS/RADIUS. Privilege 15 is required. This field is mandatory. Password cannot contain whitespaces

 

**NOTE: If you would prefer to manually configure switchport configurations, or enable autoconfiguration at a later time, leave these settings disabled.

 

After filling out all the required fields, click Add. The Virtual Edge Node onboarding process will begin immediately.

 

Step 6: Create the Identity VLAN and VLAN interface. 

switch(config)# vlan 10
switch(config-vlan)# name Identity_VLAN

switch(config)# interface vlan 10
switch(config)# description Identity_SVI
switch(config)# ip address 192.168.10.1 255.255.255.0

 

Step 7: Usually the second VLAN that provides connectivity to the Virtual Edge for the Virtual Edge Identity Agent already exists. If it does not, go ahead and create this VLAN and VLAN interface as a gateway (if locally routed) and make sure it can provide connectivity to the Virtual Edge. 

 

switch(config)# vlan 11
switch(config-vlan)# name Gateway_VLAN

*** If locally routed ***

switch(config)# interface vlan 11
switch(config)# description Gateway_SVI
switch(config)# ip address 192.168.11.1 255.255.255.0

 

Step 8: Configure the appropriate AppGigEthernet interface as a trunk to carry both the Identity VLAN as well as the gateway VLAN.

 

switch(config)# interface AppGigEthernet1/1
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk allowed vlan 10,11

 

Step 9: Use the following example, chart and diagram to configure the IE3400 to host the Virtual Edge Identity Agent. You will need to assign an IP to guest-interface 0 and guest-interface 1 based on the VLAN and networks previously configured. Make sure to change all of the IP addresses in the example below to those that are relevant for your deployment. 

Identity VLAN	Dedicated Identity VLAN for communication with the Virtual Edge Identity Agent. This VLAN cannot be used for any other purpose on the switch and must not be carried on any interface other than the AppGigEthernet interface. This VLAN communicates with the Virtual Edge Identity Agent's guest-interface 0 and must be on the same network.
Gateway VLAN	A gateway VLAN available that provides a routed or switched path to the Virtual Edge. This must be in the same network as guest-interface 1 on the Virtual Edge Identity Agent.
Identity VLAN IP	IP address in the Identity VLAN subnet assigned to guest-interface 0 of the Virtual Edge Identity Agent. This IP is used for communication between the IE3400 and the Virtual Edge Identity Agent. The IE3400 should also have a VLAN interface configured with an IP address in the same VLAN and subnet.
Gateway VLAN IP	IP address in the Gateway VLAN subnet assigned to guest-interface 1 of the Virtual Edge Identity Agent. This IP is used for communication between the Virtual Edge Identity Agent and the Virtual Edge. The IE3400 may also have a VLAN interface configured with an IP address in the same VLAN and subnet if it is being used as a local default gateway. If the default gateway for this VLAN is remote (upstream), no IP address in this VLAN is required on the IE3400.
Gateway VLAN Default Gateway	IP address of the Gateway VLAN default gateway. This may be local to the IE3400 or remote (upstream). The only requirement is that this IP be layer 2 reachable by the Virtual Edge Identity Agent and provide a routed path to the Virtual Edge.
Management IP	The IP address of the IE3400 that was defined during onboarding into Cloud Control Center in Step 5.
Virtual Edge IP	The IP address of the Virtual Edge being used to onboard the IE3400.

 

app-hosting appid VE-AGENT 
app-vnic AppGigabitEthernet trunk 
vlan <Identity VLAN> guest-interface 0 
guest-ipaddress <Identity VLAN IP> netmask <Subnet Mask>
vlan <Gateway VLAN> guest-interface 1 
guest-ipaddress <Gateway VLAN IP> netmask <Subnet Mask>
app-default-gateway <Gateway VLAN Default Gateway> guest-interface 1
app-resource docker 
run-opts 1 "--entrypoint /etc/init.d/edge"  
run-opts 2 --cap-add=NET_ADMIN  
run-opts 3 "--ulimit nofile=90000:90000"  
run-opts 4 "--env EDGE_TYPE=VE-AGENT --env EDGE_UPLINK_IP=<Gateway VLAN IP> 
--env EDGE_AGENT_SRC=<Management IP>  --env EDGE_AGENT_DST=<Virtual Edge IP>" 
run-opts 5 "--hostname VE-AGENT"  
app-resource profile custom  
cpu 1400  
memory 768  
vcpu 2  
name-server0 8.8.8.8
persist-disk 1024 
start 

*** Example Config ***

app-hosting appid VE-AGENT 
 app-vnic AppGigabitEthernet trunk 
  vlan 10 guest-interface 0 
   guest-ipaddress 192.168.10.2 netmask 255.255.255.0
  vlan 11 guest-interface 1 
   guest-ipaddress 192.168.11.2 netmask 255.255.255.0
 app-default-gateway 192.168.11.1 guest-interface 1
 app-resource docker 
  run-opts 1 "--entrypoint /etc/init.d/edge"  
  run-opts 2 --cap-add=NET_ADMIN  
  run-opts 3 "--ulimit nofile=90000:90000"  
  run-opts 4 "--env EDGE_TYPE=VE-AGENT --env EDGE_UPLINK_IP=192.168.11.2 
     --env EDGE_AGENT_SRC=192.168.11.1 --env EDGE_AGENT_DST=10.1.1.1"
  run-opts 5 "--hostname VE-AGENT"  
 app-resource profile custom  
  cpu 1400  
  memory 768  
  vcpu 2  
  persist-disk 1024
 name-server0 8.8.8.8  
 start 

 

Step 10: Make sure the SD card is inserted and formatted by running the following command: 

switch# format sdflash: ext4 

 

Step 11: The .tar file was previously copied to IE3400 internal flash in step 3. Install the Virtual Edge Identity Agent application on the IE3400 using the following command. 

switch# app-hosting install appid VE-AGENT package flash:<tar file name> 

 

Step 12: Verify that the Virtual Edge Identity Agent is running using the following command.

switch# show app-hosting list
App id                                   State
---------------------------------------------------------
VE-AGENT                                 RUNNING

 

Step 13: Next we need to configure the IE3400 switch to enable ERSPAN export of identity metadata to the Virtual Edge Identity Agent we just deployed in the previous steps. Use the following configuration example. In an upcoming release this configuration will be automated. 

 

monitor session 1 source interface <monitored interface range> 
monitor session 1 destination remote vlan <Identity VLAN ID>
monitor session 1 destination format-erspan <Virtual Edge Identity Agent Identity VLAN IP>

monitor session 1 source interface GigabitEthernet1/1 - 10 rx 
monitor session 1 destination remote vlan 10                  
monitor session 1 destination format-erspan 192.168.10.2    

*** If you want to exclude interfaces (such as Gig1/6) from the list follow this example ***

monitor session 1 source interface GigabitEthernet1/1 - 5, GigabitEthernet 1/7 - 10 rx
 

 

Checking the Status of a VEN Onboarding

In the top right of your Cloud Control Center dashboard, you will see a notification icon. After beginning the VEN onboarding, a blue dot will indicate that the status of your VEN onboarding has an update. 

 

Clicking on this icon will reveal the status of your VEN onboarding. As each step of the onboarding is completed successfully, that item is marked with a green check mark and a "Success" status.

If any errors are encounter during onboarding, a red error indicator will appear on that item, with a brief description of the issue. In this case, we can surmise that the reason this onboarding has failed is because the switch is unreachable. We need to then check for errors and confirm that our Virtual Edge Node can reach both CCC and our VE.

Once the onboarding is complete, your VEN will show green in Cloud Control Center and information about the switch is now visible such as hostname, switch model, number of discovered devices, and more. 

You are now ready to review port configurations for this VEN in the next step.

 

Port Configurations on Virtual Edge Nodes

Port configurations for endpoint discovery and analytics can be manually configured or automated based on the following logic.

Elisity offers the ability to automate the configuration process for switch ports to selectively enable or disable the collection of device and analytics data. This automation is enabled during Virtual Edge Node onboarding where administrators have the option to enable Enhanced Endpoint Discovery, Flow Telemetry, and Passive Endpoint Discovery upon onboarding. This automation is designed to enhance network security and operational efficiency by focusing on relevant data collection and minimizing unnecessary endpoint discovery and telemetry on specific ports. This prevents discovery and analytics of devices that are not in the scope of an organizations microsegmentation efforts, such as upstream networking equipment or daisy chained access switch designs.  

 

Endpoint Discovery and Telemetry Mechanisms

Enhanced Endpoint Discovery
Enhanced Endpoint Discovery is key for identifying and managing devices on your network. However, Elisity now provides the ability to automatically exclude certain port types (as listed above) from this discovery process to optimize network performance and security. This is configured at the port level on each VEN.

Passive Endpoint Discovery
Passive Endpoint Discovery remains a global configuration within Elisity and is not subject to automatic enablement or disablement based on port types. This ensures consistent passive monitoring across the network. This is a global setting per VEN, but only collects data for devices discovered through one of the other mechanisms.

Flow Telemetry
Flow Telemetry, Elisity's equivalent of NetFlow, provides valuable insights into your network's traffic patterns. With automatic configuration, Flow Telemetry will be disabled on the specified port types but can still be enabled or disabled manually per switchport, or globally on a per-VEN basis as required.

 

Criteria for Automatic Configuration

Elisity automatically disables Active Endpoint Discovery and Flow Telemetry on the following types of switch ports:

100Gig and 40Gig Interfaces

Due to their high capacity, these interfaces are typically used for backbone connections or high-traffic areas where endpoint discovery and flow telemetry is typically not desired. Commonly these interfaces are used as uplinks to other switching infrastructure. 

AppGig Interfaces

These application-specific interfaces are excluded from automatic discovery to prioritize more critical network traffic and devices. These interfaces are not typically in scope for policy enforcement and discovery.

VLAN Interfaces & Port Channel Members

Switch Virtual Interfaces (SVIs) or interfaces that are a member of a Port Channel are also excluded from discovery to avoid redundancy and focus on individual device connectivity.

 

Utilizing CDP/LLDP for Uplink Detection
Elisity leverages CDP (Cisco Discovery Protocol) and LLDP (Link Layer Discovery Protocol) to identify switch and router uplinks. The system periodically checks (every 5 minutes) LLDP/CDP neighbors to update configurations based on network topology changes, ensuring accurate and up-to-date discovery and telemetry data. If a switch or router is identified on the other end of a switchport, that port will have Endpoint Discovery and Telemetry disabled.

 

If a switchport meets any of these criteria, the Endpoint Discovery and Telemetry Mechanisms are excluded on those ports.

Modifying Port Configurations for a VEN

After onboarding, you can review the port configurations in Cloud Control Center for each port and modify them according to your network design and the scope of your microsegmentation efforts. If you chose to leave these options disabled during onboarding, now is the time to either enable autoconfiguration or manually configure each switchport for each setting.

To do this, select your VEN, navigate to Port Configurations, and click Edit Port Configuration. This will take you to the port configuration editor for all three settings, regardless of which port configuration setting you are currently viewing.

If you have not yet configured any port configurations, simply click on Add Port Configuration as seen below.

The port configuration editor is very straightforward. For each setting, you can globally enable or disable for the selected VEN. Just below the global setting, you can choose automatic or manual port configurations for the selected setting. For manual configuration, you can select specific ports or select all ports by clicking the top check box. After selecting ports, use the arrows between the two columns to move ports into the Disabled Ports or Enabled Ports tables. 

Selecting Automatic Configuration will overwrite any manually configured ports, and will disable the ability to select switchports for each table as this process will be handled according to the logic defined earlier in this article.

After reviewing these port configs and making any adjustments, click submit and your configurations will be immediately pushed to the VEN. Within 24 hours you should begin to see discovery data and analytics.

Decommissioning and Deleting a Virtual Edge Node

Decommissioning a VEN takes the enforcement point out of service by removing the configurations from the switch, but retains the configuration in Cloud Control Center so that you can easily put the VEN back in service with a single click.

 

Open the details view of your Virtual Edge Node and then select Decommission in the top right. The Virtual Edge Node status will say Decommissioned. 

You can also decommission from the main VEN dashboard by clicking the three dots to the right and selecting Decommission Virtual Edge Node. 

If you want to decommission multiple VENs simultaneously, select the VENs using the check boxes on the left and click Bulk Actions. Here you can perform various bulk actions such as restart Restonf, Decommission, and Delete. 

In any case, you will be presented with a confirmation request to finalize the decommission action with warnings or errors where applicable.

After decommissioning, the Activity Panel will show the status of the decommission process. The Activity Panel is accessible through the notification icon in the top right corner of Cloud Control Center. 

After completing the decommission process for the VENs, you can then delete them from Cloud Control Center, or leave as decommissioned for easily recommissioning at a later time. Clicking the more options button under the actions panel to the right of the VENs will show the delete or recommission options for each VEN. These options are also available in the bulk actions menu as seen earlier. Deleting a VEN requires no further action.

 

Recommissioning a VEN will also provide a status feedback in the activity panel for tracking the step by step recommissioning process.