Tag Archives: fibre channel

Brocade Accredited Server Specialist – BASP

http://www.brocade.com/education/certification-accreditation/accredited_server_connectivity/curriculum.page

I’m currently preparing for yet another accreditation, the previous one I took was the BADCS,

The BASP (Brocade Accredited Server Specialist) appear to focus on the server side. Things like:

  • how to install drivers
  • HBA management tools
  • describe features
  • how to run diagnostics

This accrediation has the most questions of all the current ones, but it has the same amount of time allotted (one hour) so this exam will have a lot less time available for each question.

The curriculum for this accreditation are also free, they are called Introduction to HBA and Introduction to CNA. There’s also some docs about the 1860 Fibre Adapter. They can be found on Brocade’s Saba/training website under my.brocade.com.

 

// Update 20140422: This accrediation has been replaced with something else. See the current list here: http://www.brocade.com/education/certification-accreditation/index.page?

Brocade Accredited Data Center Specialist – BADCS

 

Time to study for another one :) Working my way towards the “Data Center Track”. To complete it it would be enough for me to complete 5 accreditations.

This one has a pretty cool name – BADCS!

http://www.brocade.com/education/certification-accreditation/accredited-data-center-specialist/prerequisites.page

I haven’t tried one of these Accredited exams before, but as far as I can tell:

  • Cheap: only 20$ USD
  • The exam is web based, no need to find a test center, you can do it exactly when you want to.
  • Accreditations do not expire
  • You don’t _have_ to take the course in the prerequisites before taking the exam, but it is recommended :)

Also, for this Accreditation the pre-requisite is the FC-101 course on brocade’s SABA page – and it’s free!

– The BADCS exam consists of 38 questions and lasts 60 minutes
– To pass this exam you must get a score of 71% or better 

So that’s about 27 correct out of 38 questions.

The objectives are on this page.

The only part I was initially not entirely sure about is the “Given a scenario, describe when portlog dumps are required”. The objectives indicate that a Fibre Channel theory knowledge is necessary, so the FC-101 course seems like a very good idea to study. I doubt many people remember specific FC mechanisms/theory if they don’t work with these occasionally. Like the well-known addresses – who remembers the address of the name-server or controller? =)

My general tip for the BADCS: Learn the material of the FC-101 course. Really. Learn. it.

You may be tricked into thinking that Brocade’s accrediations are easy because you can do them from home.

Storage FC HBA Transfer Size Tuning

HP just published an advisory describing how to tune some parameters for Emulex, Qlogic and Brocade Fibre Channel HBAs: c02518189. It sounds like these are new, but these changes have been around for at least 6 months in all three vendors’ HBAs.

Emulex

“Emulex driver version 2.42.002 or later, along with OneCommand Manager version 5.1.53.2 or later,”

Use HBAnywhere to change these.

Examples to tune the server or port level transfer size:

  • 128 kbytes, set the LimTransferSize = 2 and ExtTransferSize = 0 (default)
  • 512 kbytes, set the LimTransferSize = 0 (default) and ExtTransferSize = 0 (default)
  • 1 Mbytes, set the LimTransferSize = 0 (default) and ExtTransferSize = 1

Qlogic

This is part of the Qlogic SANSurfer utility.

  • c:\>qlfc -tsize /fc
  • c:\>qlfc -tsize /fc /set 128
  • c:\>qlfc -tsize /fc /set default

Brocade

  • bcu drvconf –key bfa_max_xfer_len –val 64
  • bcu drvconf –key bfa_max_xfer_len –val 128

BCFP – good threads in the brocade forum

How to set up two FCIP tunnels between two B7500: http://community.brocade.com/message/15261

Connecting two sites via FCR: http://community.brocade.com/message/16828#16828

Set up FCIP between two B7800: http://community.brocade.com/message/14216#14216

Why is there an IP in ipaddrshow on FC interfaces: http://community.brocade.com/message/3978#3978

 

 

 

 

BCFP – FCIP – Fibre Channel over TCP/IP

Still studying for Brocade’s BCFP Exam.
This post is to try to put light on some of the terms/technologies you’ll be surrounded by when learning about FCIP.

Guides you should see are the “Fabric OS FCIP Administrator’s Guide” and you should probably start with the material for BCFP – part 4 (theory) and 5 (administration).

Basically the FC frames will be encapsulated in packets over TCP/IP, making the TCP/IP part  invisible/irrelevant to the SAN fabric and the FC frames invisible/irrelevant to the TCP/IP. Except of course for the FC routers that bridge the networks. It is possible to run FCR over FCIP as well via the VEX ports (virtual EX_port). Extension. This means that it’s using TCP flow control, no BB credits.

Terminology

Tunnel (VE_port) – are seen as VE_ports in the fabric.

Circuits (GbE ports) are inside a tunnel (VE_port)
Is a logical connection between two IP addresses.

Metric 0 – active (you can have several links at metric 0)
Metric 1 – standby

FCIP tunnels support max two hops.

Multi-Gigabit Circuits

On the FX8-24:

2x 10GbE
1x 10Gbe and 10x 1GbE
10x 1GbE
Not,  all ports at the same time.

FCIP Trunking

Basically adding more circuits to a tunnel, not recommended to set up several tunnels (limited anyway) but because ISL trunking is not supported on VE_ports.

FICON timeout: 1s
FC timeout: 4s
Consider altering these depending on your setup/latencies.

Virtual Fabric considerations

Define several logical switches inside a physical.
You can with FOS 7.0.0 have a VE_port (the GbE ports) defined in the base/default switch and then share it with other logical switches, giving you the possibility to extend/route the fabrics over a shared trunk while they are still isolated. You cannot mix dedicated (in an LS) and a shared (in default) in the same FCIP tunnel.

QoS

Not enforced if there is no contention (there is free bandwidth)

VC0 (or F_frames – fabric frames) – always first.
QoS_High: >50% : : 6
QoS_Medium: >30% : 3
QoS_low: >20% : 1

DSCP (6 bits of priorities – 64 )
L2CoS (3 bits of priorities- 8 )
Priority is set in the TOS – in the header.

Compression

(four different ones, hardware, software, mix, auto)

10GbE

“lossless” failover only in FOS 7.0.0. (brocade chipset did not share ports)
You cannot use both 10GbE and get 20GbE. You can have them active/standby or use both A/A and get 5Gbps on each.
Disabling port != failover testing.  Can/will cause disruptions.

Crossport

Crossports are addresses and routes that belong to the other 10GbE (XGE) port’s DP or VE group.

The crossport for xge0 is xge1 and for xge1, the crossport is xge0. To use crossports, the port must be configured in 10 Gbps mode.

The crossport is the non-local XGE port for a VE_Port group. In other words, for
VE ports 12 through 21, xge1 is the local XGE port and xge0 is the crossport. For VE ports 22
through 31, xge0 is the local XGE port and xge1 is the crossport.

SACK

(selective acknoledgement – prevent that each lost packet requires an ack, bundles up several lost packets into one, default is ON)

Adaptive Rate Limiting

Configure minimum and maximum rates on an FCIP circuit.
Let’s say you have one FCIP router with two circuits going to two independent IP-routers, these two share a link to another site. The idea is that then you can use ARL to configure minimum half of the shared link on each of the circuits from the FCIP router to the IP router, and a max of the whole one. So if one goes down, you’re not stuck with half and you’re not oversubscribing. There, easy to explain in words :d


Hardware

FX8-24

2 x 10GbE ports, 12 x 1GbE and 12 x FC8
Link to hardware page on Brocade.

7800

6 x 1GbE ports, 16 x FC8
Link to hardware page on Brocade.

Steps

  1. What settings are you going to have on the ports/links/tunnels?
  2. Configure hw ports (media type, mode etc)
  3. Disable VE_ports (Virtual FC E_ports) with the tunnel (portdisable)
  4. Create ip intf for each phy Ethernet port that’s going to be used (portcfg ipif)
  5. Config IP route for each port to specify an IP Gateway (not required; portcfg iproute)
  6. Verify IP network between the two IP interfaces that will form the tunnel. (portcmd –ping slot/port)
  7. Create an FCIP tunnel (circuit 0; portcfg fciptunnel; portcfg fcipcircuit)
  8. Config FCIP Features (SACK, compression, etc)
  9. Verify config, enable VE_ports, verify that it’s working
  10. Add more circuits to the tunnel(s).

BCFP – Fibre Channel Routing – FCR – FC-FC Routing

FCR; Fibre Channel Routing; FC-FC Routing; etc; etc.

This has many names. FC-FC Routing service provides FCR (fibre channel routing).
Basically what it does is that it lets you zone devices in two separate fabrics without merging them.
These two separate fabrics are called ‘edge fabrics’ in Brocade lingo, they are otherwise known as SAN islands.
The edge fabric is connected to a backbone fabric (an FC router or at least an EX_Port).

Integrated Routing – is a licensed feature that lets you run FCR on a port that is in a normal port in a normal switch or port blade (so not in a dedicated router switch or router blade).

There’s a few things required to set up FCR:

  1. Verify that you have the proper setup (required licenses/hardware)
  2. Assign backbone FIDs (switchdisable; fosConfig –disable fcr; fcrconfigure; fosconfig –enable; switchenable)
  3. Configure FCIP tunnel (not required but: portcfg fciptunnel 8/ge0 create 2 1.1.1.1 1.1.2.1 0 -v 100 -p 3 -P 7 . Remote IP first, tunnel ID, vlan, Classes for layer2 control and data traffic)
  4. Configure IFLs – inter fabric links – links between edge and backbone fabrics (portcfgvexport, portcfgexport 7 -a 1 -f 30 . port 7, enable, fabric id 30)
  5. Modify cost on the EX_ports (not required; portdisable; portcfgexport 7 -a 1; fcrrouterportcost 7 10000; for default, set it to 0; fcrRouteShow also shows cost)
  6. Connect cables (if you do it before and they are configured as E_port you may get segmentation).
  7. Configure trunking on EX_ports (not required but if you have more than one link, please do, same commands as for E_port trunking)
  8. Configure LSAN zones (same as normal zoning; zonecreate “lsan_zone_fabric”, “wwn; wwn2; wwn3”; cfgadd “zone_cfg”, “lsan_zone_fabric”; cfgenable “zone_cfg”). Use lsanzoneshow -s. Shows imported/exist/configured/initializing.
    fcrphydevshow, fcrproxydevshow are also useful.
  9. Confirm that it’s working (fcrfabricshow, switchshow, portcfgshow, portexport 7, portshow 7)
So what you have to do is: assign FIDs, configure IFLs and LSAN zones. 

A little theory

Phantom domains.

Front domains -> always there

Translate domains (also xlate domains…).  -> only there when devices are online and zoned

The FC router has a pool of wwns and proxy ids that it assigns to devices.
Basically a host that wants to communicate with a target in another fabric communicates with a proxy WWN in its own fabric (so the FC router is like a middle man that passes frames back n forth).

A little more information

When a PLOGI, PDISC, ADISC frame arrives at the FC router, SID and DID are checked. If they are zoned in both SID and DID edge fabrics (islands), the frame is forwarded to DID. If not, only PLOGI is dropped; edge fabrics’ zoning enforcement takes care of the rest.
I found this document on EMC’s webpage (it’s from 2007 so a bit outdated and it has EMC’s names of the Brocade products) but it explains the concept pretty nicely.
Also, this post is for me to study for the BCFP, I find that I learn better when I write things down with a keyboard ;)
There may be mistakes in here but that’s just how it is, I tried to keep it as factual as possible and used several sources.
There will most likely be more of these posts coming up.