Fibre Channel (FC) is a high-speed data transfer protocol providing in-order, lossless delivery of raw block data, primarily used to connect computer data storage to servers. The lossless delivery of a raw data block is achieved based on a credit mechanism known as buffer-to- buffer credits, which we discuss later in this chapter. Fibre Channel typically runs on optical fiber cables within and between data centers but can also run on copper cabling. We will discuss copper cabling options in the next chapter when we dig deep into the FCoE protocol. Fibre Channel networks form a switched fabric because they operate in unison as one big switch. Here we set the stage for the Fibre Channel switched fabric initialization process after discussing various Fibre Channel topologies and port types.
Fibre Channel Topologies
It is common practice in SAN environments to build two separate, redundant physical fabrics (Fabric A and Fabric B) in case a single physical fabric fails. In the design of SANs, most environments fall into three types of topologies within a physical fabric: single-tier (collapsed-core), two-tier (core-edge design), and three-tier (edge-core-edge design).
Collapsed-core topology: Within the single-tier design, servers are connected to the core switches. Storage devices are also connected to one or more core switches, as shown in Figure 8-8. Core switches provide storage services. It has single management per fabric and is mostly deployed for small SAN environments.
Figure 8-8 Collapsed-Core Topology
The main advantage of this topology is the degree of scalability offered at a very efficient port usage. The collapsed-core design aims to offer very high port density while eliminating a separate physical layer of switches and their associated ISLs.
The only disadvantage of the collapsed-core topology is its scale limit relative to the core-edge topology. While the collapsed-core topology can scale quite large, the core-edge topology should be used for the largest of fabrics. However, to continually scale the collapsed-core design, you could convert the core to a core-edge design and add another layer of switches.
Core-edge topology: Within the two-tier design, servers connect to the edge switches, and storage devices connect to one or more core switches, as shown in Figure 8-9. This allows the core switch to provide storage services to one or more edge switches, thus servicing more servers in the fabric. The inter-switch links (ISLs) will have to be designed so that the overall fabric maintains both the fan-out ratio of servers to storage and the overall end-to-end oversubscription ratio. High availability is achieved using two physically separate, but identical, redundant SAN fabrics.
Figure 8-9 Core-Edge Topology
In the design of a core-edge topology, a major trade-off is made between three key characteristics of the design. The first trade-off is the overall effective port density that can be used to connect hosts or storage devices. For a given number of switches in a core-edge design, the higher the effective port density, typically the higher ISL oversubscription from the edge layer to the core layer. The second characteristic is the oversubscription of the design. Oversubscription is a natural part of any network topology because the nature of a SAN is to “fan out” the connectivity and I/O resources of storage devices. However, the higher the oversubscription of the design, the more likely congestion may occur, thereby impacting a wide scope of applications and their I/O patterns. The third characteristic that ties these other two together is cost. The basic principle suggests the higher the oversubscription for a given effective port density, the lower the overall cost of the solution.