Nexus 3016 Interface Numbering

In a new deployment that I’m working on, I recently came across Cisco’s Nexus 3016 for the first time. ¬†It’s also the first time I’ve worked with 40G, so that’s exciting too ūüôā

The Nexus 3016 comes with 16x 40 Gbps QSFP+ ports…but the 40G ports can be split out into 4x 10G ports. ¬†Given that not a whole lot of devices have 40G connectivity today, this functionality will be needed in most deployments.

The 3016 has several different port modes that can be configured on the device. ¬†These are six different combinations of 10G and 40G ports that you can allocate. ¬†This configuration is done at the device level – you can’t configure each port individually to be 10G or 40G.

These are the port modes that are currently supported on the devices we’re running:

  • 0x10g +¬†16x40g
  • 8x10g + 14x40g
  • 16x10g + 12x40g
  • 32x10g + 8x40g
  • 48x10g + 4x40g
  • 64x10g + 0x40g

Here is Cisco’s documentation on the port modes.

It’s somewhat of a pain to change these port modes once you’ve set them. ¬†It’s not something you can trivially do on the fly…you’ll have to reload the switch and Cisco recommends clearing the configuration. ¬†From a couple of tests I did, clearing the configuration isn’t 100% necessary, but you should expect that your interface configurations will be messed up afterwards. ¬†After changing the port mode and reloading, some port configurations ended up being configured on different ports. ¬†Depending on the scenario, that could definitely be bad. ¬†So doing a “write erase” is probably the safest option.

One aspect that I couldn’t find any information on is what the interface naming looks like for the 10G ports. ¬†I had to fire up the 3016 to take a look at this.

First, the 10G ports are allocated from port e1/1 going up. ¬†So, if you’ve assigned 4x physical ports for 10G connectivity (16x 10G connections total), ports e1/1 – e1/4 will be used for 10G connections. ¬†Ports e1/5 through e1/16 will be 40G.

The 10G ports appear as individual ports under a module, where the module is the original number of the physical port that it uses. ¬†This is probably easier to demonstrate in an example. This is what the interfaces look like if you’re using the 8x10g+14x40g port mode.

Interface Name Type Physical Port
e1 / 1 / 1 10G port 1
e1 / 1 / 2 10G port 1
e1 / 1 / 3 10G port 1
e1 / 1 / 4 10G port 1
e1 / 2 / 1 10G port 2
e1 / 2 / 2 10G port 2
e1 / 2 / 3 10G port 2
e1 / 2 / 4 10G port 2
e1 / 3 40G port 3
e1 / 4 40G port 4
e1 / 5 40G port 5
e1 / 6 40G port 6
e1 / 16 40G port 16

I’d be curious to hear about other folks who have used the 3016’s. What have your experiences been?

Comments are closed.