In a new deployment that I’m working on, I recently came across Cisco’s Nexus 3016 for the first time.  It’s also the first time I’ve worked with 40G, so that’s exciting too :)

The Nexus 3016 comes with 16x 40 Gbps QSFP+ ports…but the 40G ports can be split out into 4x 10G ports.  Given that not a whole lot of devices have 40G connectivity today, this functionality will be needed in most deployments.

The 3016 has several different port modes that can be configured on the device.  These are six different combinations of 10G and 40G ports that you can allocate.  This configuration is done at the device level - you can’t configure each port individually to be 10G or 40G.

These are the port modes that are currently supported on the devices we’re running:

  • 0x10g + 16x40g
  • 8x10g + 14x40g
  • 16x10g + 12x40g
  • 32x10g + 8x40g
  • 48x10g + 4x40g
  • 64x10g + 0x40g

Here is Cisco’s documentation on the port modes.

It’s somewhat of a pain to change these port modes once you’ve set them.  It’s not something you can trivially do on the fly…you’ll have to reload the switch and Cisco recommends clearing the configuration.  From a couple of tests I did, clearing the configuration isn’t 100% necessary, but you should expect that your interface configurations will be messed up afterwards.  After changing the port mode and reloading, some port configurations ended up being configured on different ports.  Depending on the scenario, that could definitely be bad.  So doing a “write erase” is probably the safest option.

One aspect that I couldn’t find any information on is what the interface naming looks like for the 10G ports.  I had to fire up the 3016 to take a look at this.

First, the 10G ports are allocated from port e1/1 going up.  So, if you’ve assigned 4x physical ports for 10G connectivity (16x 10G connections total), ports e1/1 - e1/4 will be used for 10G connections.  Ports e1/5 through e1/16 will be 40G.

The 10G ports appear as individual ports under a module, where the module is the original number of the physical port that it uses.  This is probably easier to demonstrate in an example. This is what the interfaces look like if you’re using the 8x10g+14x40g port mode.