|
||
|
|
|
              |
|
 |
 |
|||||||||||
|
 |
|
|||||||||
By William Van Winkle |
|||||||||||
[ iSCSI ] Fibre Channel remains firmly entrenched in the enterprise. But the majority of resellers and VARs deal with SMBs below this level—a region where FC is often cost prohibitive. Fibre entails adding a dedicated storage network on top of whatever other networking technologies are already in place. In contrast, Internet SCSI, or iSCSI, is a networking protocol that allows the SCSI command set to run over conventional TCP/IP networks, meaning Ethernet in most cases. The wiring behind iSCSI is all conventional Ethernet; the real work is done with software that creates iSCSI "initiators" and "targets" in order to facilitate block-level communication. With this in place, the pertinent hardware simply looks like SCSI devices to the systems involved. The Internet Engineering Task Force (IETF) only ratified iSCSI in 2003, so the technology is still fairly fresh and sparsely deployed. ISCSI is gaining steam quickly, in part because of its lower learning barrier and the increasing deployment of Gigabit Ethernet throughout the industry. Strapped to 10/100 Mbps Ethernet, iSCSI thoughput would be unbearably slow for corporate applications. With Gigabit and especially 10 Gigabit gaining ground, though, the prospects for iSCSI look much brighter, and that's before considering that iSCSI bandwidth can be expanded by aggregating Ethernet links. Naturally, you don't get 2 Gbps by aggregating two Gigabit lines. There are overhead costs and stability issues that need mitigating as you combine connections. The overhead issue caused by TCP/IP processing in particular is critical with iSCSI. Every time a data packet arrives at the network card, the NIC interrupts the host CPU, which shunts the packet into the network buffer, then into system memory for processing. With this done, the CPU can return to its regular work until the next packet arrives. Obviously, the more packets there are to process, the heavier the load on the CPU. "ISCSI has a lot more protocol overhead than Fibre Channel," says Tom Treadway, CTO for block storage and RAID at Adaptec. "I've seen a lot of people throw out the figure that 1 GHz of CPU power is needed to keep a 1 Gbit line busy. We've done some measurements, and it's certainly in that ballpark. The way iSCSI was implemented, there's just a ton of TCP/IP overhead. Like when you look at our iSCSI target, about half of our processing went to just the TCP/IP overhead. ISCSI itself isn't that big of a thing. It's just a way of putting commands out on the wire. But that TCP/IP overhead is the bottleneck." There are a few primary ways to tackle TCP/IP overhead today. The first is a brute force attack. Just throw enough processor power at the problem and in theory it will eventually go away. It's too bad, then, that CPU resources and TCP throughput don't ramp in a linear fashion. In an ACM Queue article called "TCP Offload to the Rescue," Andy Currid outlines a "relative cost" for TCP throughput and CPU utilization that boils down to this: Relative Cost = (CPU utilization % * CPU speed in MHz) / Throughput in Mbps This equation predates the rise of multi-core processors, but the central idea remains. "In all cases," Currid writes, "the relative CPU cost per megabit of throughput was higher for the faster CPU." Some sources indicate that a filled Gigabit Ethernet connection with all of its TCP processing handled in software will redline a 2.4 GHz Pentium 4. Intel sought to tackle this problem a couple of years ago with its silicon-based I/O Acceleration Technology (I/OAT). In a server platform such as Bensley (Intel's 5000-series Xeon platform), I/OAT plants special engines in the processor(s), chipset, network controller, and firmware to perform TCP/IP optimization and data crunching. In essence, these hardware elements are either eliminating or taking over much of the work normally performed by the CPU in software. With I/OAT, Bensley servers show over 30% faster data exchange between system hardware and applications and up to 40% less CPU utilization while realizing up to 100% improvement in data throughput. A similar approach is to employ a TCP Offline Engine (TOE). Like other contemporary secondary processors, such as audio or graphics processor units, a TOE generally appears on a network card and offloads TCP/IP processing tasks into dedicated circuitry, sometimes called an Internet Protocol Processor (IPP). The IPP handles the TCP traffic and routes it to system memory so the CPU doesn't have to. Alacritech (www.alacritech.com) owns a large chunk of the TOE intellectual property, as was evidenced in 2005 when the company advantageously settled a patent dispute against Microsoft and its "Chimney" implementation of TCP/IP offloading in order to accelerate Windows server operating systems. Nowadays, Alacritech refers to its TOE products as Scalable Network Accelerators (SNAs).
Exactly how much offloading is necessary depends as much on the applications running as the hardware involved. In some cases, the benefits of a TOE may be minimal. In July of 2003, storage library vendor Spectra Logic published a white paper comparing IP SAN versus FC SAN using a 20GB Microsoft Exchange Server database. A dual-Pentium 4 server ran to either a Fibre or IP switch then out to Spectra Logic's library device. Using a single data stream (1 Gbps), FC showed 19.32 MBps against iSCSI's 21.14 MBps. With four streams, FC scored 35.35 MBps against iSCSI's 32.72 MBps. From these numbers, you not only get a sense of the relativity between stream counts and throughput but also an awareness that Fibre and iSCSI are surprisingly close on performance under such circumstances, even without the help of a TOE. What isn't close is the infrastructure cost. According to Spectra Logic, the iSCSI solution costs 63% to 74% less than Fibre, and that was back in 2003 before Gigabit Ethernet costs plummeted. Getting involved in iSCSI requires some basic knowledge about how the protocol moves data, much of which derives from legacy SCSI architecture. The client/server model used in SCSI outlines initiator and target entities. An initiator is the client-type device that "initiates" device service and task management requests from an application client to a device server and task manager. These latter two functions are found in the target. Said more simply, an iSCSI initiator connects a client to some service being offered up by the target—storage in this case. The target coordinates the requests for storage resources from one or more initiators, acting as a bridge between the network and the storage pool. The end result is that iSCSI targets look like local devices to the client rather than being network shares. This gets around the problem of many applications not wanting to interact with shared network storage. Obviously, iSCSI networks may not always behave like local storage because overall performance is at the mercy of network conditions. Large amounts of traffic can impact throughput to the iSCSI storage system. Building a dedicated IP network for storage is one way to avert such risks. (Shelling out for Fibre Channel is another.) If a dedicated network isn't an option, you might choose to implement a virtual LAN (VLAN) to segregate a certain amount of bandwidth from concurrent traffic. Or you could push for a tiered approach where the most critical storage resources are put on FC and the second tier goes with iSCSI. The question of FC vs. iSCSI will probably be settled when 10 Gigabit Ethernet (10 GbE) becomes standard in the high-end business world. Until then, FC still has many fans.
Deploying iSCSI is in some ways easier to do than explain. For instance, you can have a hardware or software initiator. A TOE-enabled NIC can be a hardware initiator. Similarly, there are iSCSI host bus adapters (HBAs) with TOE functionality but lacking the Gigabit port. If you feel your processor setup is more than adequate for the TCP crunching involved, a software initiator with an ordinary integrated Gigabit port may be enough. Whether talking about hardware or software initiators, the iSCSI driver in the host is what packages the SCSI commands and exports them via TCP/IP. There are now software initiators for OSes ranging from AIX to Max OS X to Windows 2000 and later. On the Windows platform, the initiator of choice is often Microsoft iSCSI Software Initiator for Windows, in version 2.03 as of this writing and free to download from Microsoft. Make sure that your initiator supports multipath I/O (MPIO), a technology that allows not only for multiple NICs to connect to storage resources for fault tolerance but also for aggregating the bandwidth of those network ports. Microsoft's initiator supports MPIO, but be aware that Windows Server is much better about handling multiple NICs than Windows XP.
Like initiators, iSCSI targets can also be hardware- or software-based. You may find parallel SCSI or FC interfaces bridged with iSCSI target software, an iSCSI-enabled controller within the storage enclosure, or an external iSCSI bridge. Adaptec's SnapServer and NetApp/StoreVault's S500 are two examples of native iSCSI targets. One of the most popular software targets was String Bean Software's WinTarget. Microsoft bought String Bean last year and now includes WinTarget as a feature with Windows Storage Server 2003 R2 and Windows Unified Data Storage Server (WUDSS). You'll also find several open source iSCSI targets for Linux available on SourceForge (sourceforge.net). The more advanced your iSCSI solution gets, the more attention you'll have to pay to additional network components. For example, for larger iSCSI deployments a managed Level 2 Ethernet switch is highly recommended. Anticipate taking the time to learn the switch's many options capable of accelerating or hindering iSCSI storage traffic. But once you have the basic components and software elements in place, getting an iSCSI solution running is comparatively simple. You'll likely have an environment such as Microsoft's WUDSS or Adaptec's OnTarget 3.0, in which you can view storage resources through a semi-intuitive GUI. OnTarget, for instance, knows how to configure both PCI RAID cards and iSCSI boxes, offers central management so you can have one server and see an entire network of DAS islands and iSCSI storage pools, and lets you configure all of it through one tool...provided each piece is sitting behind a suitable controller. You have a single utility for both iSCSI and DAS resources, which means that iSCSI is just as easy as DAS to set up. From there, you're down to mounting volumes, setting up shares, and, if applicable, configuring snapshot and backup settings. At this point, iSCSI may sound like the obvious choice over Fibre Channel, and for most SMBs it probably is. The cost savings and lower learning cycle inherent in iSCSI are too persuasive. However, some organizations will still find elements of FC worth adopting or continuing. For instance, Fibre is often bootable because the server's controller card has an onboard BIOS. ISCSI is harder to make bootable, especially with an integrated NIC, because the motherboard's BIOS doesn't know how to boot via the iSCSI protocol. New standards are emerging to address this, but today the issue is still largely unresolved.
"I'd almost bet you that if 100 SANs are sold in the next couple of hours, 70% of them will be Fibre," says Don Young, president of custom server reseller Solutions. "That's just my own guess. Now, most SMBs are fine with iSCSI because they're not pumping that much data in a limited time window. But SAN buyers typically have a lot of storage, and that might mean a lot of customers trying to access it. And if that's the case, someone better start doing the math to see if you've got enough bandwidth to access all that data. That's why people pay a couple thousand dollars for a Fibre port, so they don't have to worry about that. It all comes down to how much access to the data you really need." Keeping in mind that Terian is building a remote storage services business (see sidebar), Terian itself is an example of a small business currently using iSCSI but verging on making an upgrade jump into Fibre. "A typical iSCSI connection is a Gigabit port," says Young. "You can aggregate them, but Intel only lets you aggregate to a point. There's only so many of them on a box. Or I can drop in a Fibre card. I've got 4 Gig now, and I can aggregate those. I can aggregate 4, 8, 16 Fibre ports together and not worry about bandwidth. The disk or processor may be a little bit of a bottleneck, but how much data are you pushing through that pipe in the back? Say I've got 10TB. At certain times of the night, I'm throwing hundreds of megs per second through this pipe, and it all has to get through in a strict time window before start of business. With a big enough pipe, I can handle everybody within a few hours. If I don't, it may take eight to ten hours, and that's too big of a window." While Young's point is worth noting, Terian may well be one of the exceptions to the rule. Most SMBs probably only have SANs with two or three servers and an enclosure or two of drives. At this level, Fibre Channel is clearly overkill. As we've seen, iSCSI is a solid option, but some customers may be tempted to try and cut corners on their iSCSI solutions. They may save a few hundred dollars on less processing power, not bother with a TOE product, and so on, and that's when the performance drop becomes a factor. [ SAS ] Another option is just now starting to come online: Serial Attached SCSI (SAS). Yes, as we saw last month, SAS is the enterprise-level drive and controller alternative to SATA. The key here is having a suitable SAS switch...and it's too bad that no one has any for sale yet. Fortunately, you can expect to see debut SKUs from LSI Logic, Rancho Technology, and others in the second half of this year. LSI made headlines way back in March of 2006 showing designs for a 1U, nine-port (four SAS lanes per port) switch able to connect up to 10 servers to one or more SAS storage arrays. With the ability to cascade multiple switches, creates zones for different storage domains, and run external SAS cable lengths up to eight meters, you can see how this opens up the potential for a SAS storage fabric rather than just a DAS or NAS interconnect. Once again, SAS volumes accessed through these switches look like local client drives. As smaller organizations progress beyond direct-attached storage and the limitations of NAS, a SAS-based SAN may soon emerge as the option of choice for both speed and cost savings. Another way to decide on a SAN architecture may be something as mundane as cable lengths. If you want a solution that's going to reside in one to several racks in very close proximity, SAS may be your best bet. If you're running room-to-room, Fibre Channel makes more sense. From building to building and beyond, there's no question: iSCSI. ...more |
|||||||||||
 Back to top
|
|||||||||||
Copyright © 2007 RAM Magazine. All rights reserved.
Do not duplicate or redistribute in any form. |
|||||||||||
