FCoE uses a Fabric Provided MAC Address (FPMA) for SAN traffic. The ENode still gets a 24-bit FCID. FPMA is built by concatenating a 24-bit FCoE MAC address prefix (FC-MAP), ranging from 0x0E-FC-00 to 0x0E-FC-FF, to the 24-bit FCID, as shown in Figure 9-13. Being able to build a unique MAC address for the ENode directly from its FCID saves the switch from having to maintain a table that associates FCID and MAC addresses.
Figure 9-13 Fabric-Provided MAC Address (FPMA)
The FC-MAP range was introduced so that different values can be assigned to different SANs. For example, SAN A would be associated with 0x0EFC00 and SAN B with 0x0EFC01. This additional configuration ensures the uniqueness of the produced FPMA in the whole network. FC-MAPs are different for different SANs, FCIDs are uniquely assigned within a SAN, and the resulting FC-MAP and FCID are unique across the different SANs in the entire network.
Figure 9-14 shows a Fibre Channel frame traversing native Fibre Channel SAN and FCoE SAN. For simplicity, only the header information is displayed in the figure. The FCoE forwarding in Figure 9-14 can be summarized in the following steps:
Figure 9-14 FCoE Forwarding
1. The Fibre Channel N_port on the storage array sends out the FC frame to the VN port on the CNA adapter with Source FCID (S_ID) as 7.1.1 and Destination FCID (D_ID) as 1.1.1. The frame is switched by the first Fibre Channel Switch.
2. Because the destination ID (D_ID) is not in FC domain 7 of the receiving switch, the switch forwards the frame to the port associated with the shortest path to the destination, using the FSPF algorithm. The Fibre Channel switch with domain ID 7 sends the frame to the FCoE switch with domain ID 3.
3. The FCoE switch with domain 3 receives the frame. Because the destination ID (D_ID) is not in the FC domain 3 of the receiving switch, the switch forwards the frame to the port associated with the shortest path to the destination, using the FSPF algorithm. In this case, the frame will be transmitted to the FCoE-enabled Ethernet fabric. The FC frame is encapsulated inside an Ethernet frame with source MAC address (MAC A) of VE_port of switch with FC domain 3 and destination MAC address (MAC B) of VE_port of switch with FC domain 1.
4. When the frame arrives at the FCoE switch with the FC domain 1, the frame is decapsulated, and the switch determines that the FC frame destination is within its domain. The FC frame is reencapsulated with a source MAC address of MAC B and a destination MAC address of MAC C. The frame is then transmitted out to the appropriate port where MAC C and FCID 1.1.1 are connected.
5. When the HBA adapter with MAC C receives the frame, it decapsulates the frame and accepts the FC frame with FCID 1.1.1
FCoE Initialization Protocol (FIP)
FCoE Initialization Protocol (FIP) is the FCoE control protocol responsible for establishing and maintaining Fibre Channel virtual links between pairs of FCoE devices (ENodes or FCFs). During the virtual link establishment phase, FIP first discovers FCoE VLANs and remote virtual FC interfaces; then it performs virtual link initialization functions (fabric login [FLOGI] and fabric discovery [FDISC], or exchange link parameters [ELP]) similar to their native Fibre Channel equivalents. After the virtual link is established, Fibre Channel payloads can be exchanged on the virtual link, and FIP remains in the background to perform virtual link maintenance functions; it continuously verifies reachability between the two virtual FC interfaces on the Ethernet network, and it offers primitives to delete the virtual link in response to administrative actions to that effect.
FIP aims to establish virtual FC links between VN_Ports and VF_Ports (ENode to FCF), as well as between pairs of VE_Ports (FCF to FCF), because these are the only legal combinations supported by native Fibre Channel fabrics. This section focuses on FIP in the context of virtual FC links between VN_Ports and VF_Ports.
Cisco NX-OS supports the T11-compliant FIP on Cisco Nexus devices. FIP is used to perform device discovery, initialization, and link maintenance. FIP performs the following protocol steps:
FIP VLAN discovery: FIP discovers the FCoE VLAN that will be used by all other FIP protocols and the FCoE encapsulation for Fibre Channel payloads on the established virtual link. FIP VLAN discovery occurs in the native VLAN used by the initiator or target to exchange Ethernet traffic. The FIP VLAN discovery protocol is the only FIP protocol running on the native VLAN; all other FIP protocols run on the discovered FCoE VLANs.
FIP discovery: When an FCoE device is connected to the fabric, it sends out a Discovery Solicitation message. A Fibre Channel Forwarder (FCF) or a switch responds to the message with a Solicited Advertisement that provides an FCF MAC address to use for subsequent logins.
FCoE virtual link instantiation: FIP defines the encapsulation of FLOGI, fabric discovery (FDISC), logout (LOGO), and exchange link parameter (ELP) frames, along with the corresponding reply frames. The FCoE devices use these messages to perform a fabric login.
FCoE virtual link maintenance: FIP periodically sends maintenance messages between the switch and the CNA to ensure the connection is still valid. This is referred to as the FCoE Keepalive (FKA).