Brocade SAN Switch Firmware Upgrades


Overview

This is my guide/template to upgrading Fabric OS (FOS) – Firmware – on the Brocade SAN Switches. If you have any additions, comments or questions please go ahead and comment or if you have any questions you can find my e-mail on http://guldmyr.com. The post have been updated over 100 times according to my wordpress revisions, first update in January 2011.

This article was originally built from my experience with HP branded Brocade SAN Switches – not with any other OEM or pure Brocade switches. I have however since beginning this document gotten experience with other vendors.
I do not think others are different except for licenses and some default fabric.ops.
I made a comparison of two downloads of the 6.3.1b Fabric OS Firmware (one via IBM and one from HP). You can find a link to the “IBM” firmwares and release notes after 6.x in that article too. I found that they are very similar and the HP firmware works on the IBM switch and vice versa. Another example is that firmware gotten from HDS works on an HP branded Brocade switch.

When you see 5.2.x this mean any version in the Fabric OS 5.2.x series. For upgrades this would generally mean the latest available in that series (like 5.2.3 for 5.2.x or 6.0.0c for 6.0.x) unless of course there is a problem with the latest. See http://hp.com/go/myadvisory for signing up to advisories/notices on HP products. You can also go to the HP Knowledge Base and search for advisories. Other vendors should also have this but I have no list of this.

Carefully plan the upgrade, it takes time but it is rewarding and worth it.

Updates in this article:

Show archived updates.
2014-02-07: Added new link to HP’s page for FOS 5.2 to 6.3  Thanks Leo R!
2014-02-01: As of 2014-02-01 HP does not allow anybody without a valid support agreement to download firmwares. Release notes and at least some firmware links appears to still be working. Expect difficulty and broken links while hunting for firmwares. Fabric OS firmwares downloaded from IBM’s site works on HP switches too, but there might be some differences (although I couldn’t find any important ones when I compared 6.3.1b). So far it seems this restriction of access to firmwares only applies to HP servers.
2014-02-11: Added 7.2.0b and 7.1.1c (HP have 7.1.1c release notes up but IBM does not – to find Brocade version go to IBM’s download the firmware page that’s on Brocade.com and get the release notes from there). Also 3 years anniversary on this post on 2014-02-01!
2014-02-17: But be careful with 7.2.0b – IBM has a note on their 7.x page about 7.2.0b saying: “IBM recommends that customers not deploy FOS 7.2.0b if virtual switch capability is needed. Virtual switch users should migrate to an earlier version as soon as possible.
2014-03-17: The problem with 7.2.0b was likely DEFECT000491192 fixed in 7.2.0c and later also DEFECT000494570 was fixed in 7.2.0d. 7.2.0x seems a bit unstable at the moment. 7.2.1 is currently available for download via HP’s pages but not via IBM/Brocade’s. Also no release notes available. Archived 2013 updates.
2014-04-18: Added 6.4.3f and 7.1.1c and 7.1.2 is out. Updated migration paths a bit. The Brocade release notes of 7.1.x actually have a decent list of the migration path needed now. See the section “Recommended Migration Paths to FOS v7.1.2″.

Steps

  1. decide
  2. prepare
  3. upgrade

Decide

One major release at a time is required for the upgrades after 5.2.x, see details below at the release notes section.

If you have to upgrade many steps, you should upgrade to the latest in the series (or if it’s very new, probably safest to go with the second newest, just check the release notes of the newest to make sure nothing related is fixed).

If the switch is on 5.1.x you can go directly to 5.3.x.

What I usually recommend is this path:
5.0.1d -> 5.2.3 -> 5.3.2c -> 6.0.1a -> 6.1.2c -> 6.2.2f -> 6.3.2e -> 6.4.3f > 7.0.2d > 7.1.2 > 7.2.0d

It’s also possible to upgrade from a version earlier than 6.4.1b to 7.x – but this is a disruptive upgrade (meaning ports will go offline/online during upgrade). 

Brocade now has a document that describes a process of determining the ‘ideal’ version of Fabric OS you should be running. It is called Brocade FOS Target Path.
There is also a section (Recommended Migration Paths to FOS ) in the release notes describing how to get to the release you’re reading notes for.

There are newer releases being released every now and then, in several series at the same time. You can think of it as releasing updates for Windows XP and 7 at the same time.
For example in February 2011 6.4.1a and 6.2.2e were released by HP. You can see this on HP’s site if you look at the date next to the download. Quite often Fabric OS versions are not released by the OEMs at the same time, for example HP released 7.1.0a before IBM.

Which is the recommended one? Usually it’s the latest one in the highest series that the switch supports. If you have storage from more than one vendor you may want to check with all and see if they all support the version you want to upgrade to.

HP: HP B-series Connectivity stream (available in HP SPOCK).
Brocade: “Brocade FOS Target Path
Other: Check out Brocade’s Compatibility Page - there are links in there to other vendors’ compatibility matrices, for example IBM, HDS, EMC, Fujitsu.
Brocade also has their own “Brocade Fabric OS 7.x Compatibility Matrix” which lists compatibility with other vendors.

You could in principle also say that (some blades in directors are excepted from these generalizations):

2G cannot upgrade to Fabric OS 6.x
4G and 8G can be on Fabric OS 6.x
All 4G except some 4/8 & 4/16 (that’s 200E) and HP’s P- and C-class 4G blade switches (4012 & 4024) can run 6.4.x
8G can run Fabric OS 6.4.x
8G and above can run Fabric OS 7.x
16G needs to be on Fabric OS 7.x

Do you want to use the latest one in each series? Probably.
Do check for published advisories and the release notes in the firmwares.
Some models or blades may work on 7.0.x and not on 7.1.x or vice versa. You need to read the release notes.

If you go to downloads for HP’s 4/16 there is a link that also takes you to the older FOS firmwares. If you don’t click through it also only have the firmwares that this switch supports. So the latest on there at the moment is 6.2.2f.

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=1143933&prodTypeId=12169&prodSeriesId=1143930&swLang=8&taskId=135&swEnvOID=54

On the link above you can also download HP’s branded NA (Network Advisor, previously known as DCFM – Data Center Fabric Manager), see notes about that below.

If you click on manuals on the left side you will also be able to download release notes and other guides and references.

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=3185363&prodTypeId=12169&prodSeriesId=3185358&swLang=8&taskId=135&swEnvOID=54

6.3.x and 6.4.x. can be found in the link above. 

5.0.x firmwares can also be found on the download page for 2/64 on hp.com

http://ftp.hp.com/pub/softlib/software12/COL22074/co-86832-6/FOS-Drawer_Statement.htm

Firmware Upgrade Order

You also probably want to decide on an order to upgrade the firmware on the switches.
It’s possible to do it via DCFM (now called Network Advisor, used to be something else) one switch at a time or even in parallel. I’d advice against doing it in parallel. One at a time and one step at a time seems the most cautious one. It’s not too bad to run a SAN with switches in different firmwares. One idea is to have all switches of one model on the same firmware. If you need to upgrade in several steps, do one step at a time.

Also switches that are of higher importance like Principal Switch, Core Switches or Seed Switches for DCFM/NA. Should you start with these or perhaps start with another switch of less importance to make sure the upgrade goes smoothly?

With more recent firmwares (6.4 and 7.x) it’s possible to jump more than one hop – if you are ok with disruptions in the network. Nice if you need to upgrade switches that aren’t in production.

Release notes:

FOS HP_Link IBM_Link

5.2.3: link
5.3.1a: link
6.0.0b: link (6.0.0c Brocade release note in .pdf)
6.1.2b: link (6.1.2c Brocade release notes in .pdf)
6.2.2f: link (6.2.2f Brocade release note in .pdf)
6.3.2e: link (6.3.2e Brocade release note in .pdf)
6.4.3f: link (6.4.3e Brocade release note in .pdf)
7.0.2d: link (7.0.2c Brocade release note in .pdf)
7.1.2: 7.1.2 Brocade release note in .pdf (HP version of 7.1.1c)
7.2.0d: link (7.2.0d Brocade release note in .pdf)

Notes from the release notes:

Upgrading from Fabric OS 5.0.x to 5.2.3 is supported
Upgrading from Fabric OS 5.1.x to 5.3.1a is supported, but upgrading from Fabric OS 5.0.x or a previous release directly to 5.3.1a is not.
Upgrading to Fabric OS 6.0.0b is only allowed from Fabric OS 5.3.x. (6.0.0c is a special upgrade version, only meant to be used in between firmware upgrades)
Upgrading to Fabric OS 6.1.2b is allowed only from Fabric OS 6.0.0b
Upgrading to Fabric OS 6.2.2f is allowed only from Fabric OS 6.1.0a or later.
Upgrading to Fabric OS 6.3.2d is allowed only from Fabric OS 6.2.0a or later.
Upgrading to Fabric OS 6.4.2b is allowed only from Fabric OS 6.3.x. You can upgrade non-disruptively from 6.2
Upgrading to Fabric OS 7.0.2c can be done non-disruptively from Fabric OS 6.4.1a or later.
Upgrading to Fabric OS 7.1.2 can be non-disruptively upgraded from 7.0.x and 7.1.x. With caveats: For example, any previously existing error log entries with FOS v7.1.0 will be permanently lost once upgraded to FOS v7.1.2.
Upgrading to Fabric OS 7.2.0b can be done non-disruptively from 7.1.x. Disruptively from 7.0.x is supported.

About non-disruptively: This means you can go to 7.0.x  from earlier  than 6.4.1a but ports will go offline during the upgrade.
See the release notes for more details.

 

DCFM: Data Centre Fabric Manager.

From 6.2.2a release notes:

With the introduction of Fabric OS 6.1.1, certain features and functions were removed from Web Tools (resident in the firmware) and migrated to the DCFM management application. HP recommends that, before you upgrade to Fabric OS 6.1.1x or later, if DCFM is not running on your fabric, you review the Web Tools functionality moved to DCFM, page 29 in these release notes and take note of what has changed so you can assess the impact on your fabric.

Fabric OS 7.x cannot be managed by DCFM 10.4 or BNA 11.0. You need BNA 11.1.0, see the release notes for 7.x.

Updates to documents

Sometimes Brocade releases updates to the manuals without actually updating the manuals. On HP’s page you can find them as “Documentation Updates”, “Fabric OS Administrator’s Guide Update”, or something like that.

Prepare

Download old Brocade Fabric OS Firmware.

Basically, you need to update in steps.

To get FOS 5.2.1b and 6.0.0c firmware: Contact OEM Vendor or Brocade. I’ve found that two vendors have the firmware available online for free: HP and IBM, see below:
For HP products: here and click ‘Obtain Software‘, it has from 5.2.3 to 6.3.0d (but you’d rather use 6.3.2e). Eventually the link points to http://ftp.hp.com/pub/softlib/software12/COL22074/co-86832-6/FOS-Drawer_Statement.htm – this link sometimes changes.
Link to IBM’s page for downloading FOS 6 firmwares. This has firmwares going back all the way to FOS 2.6, it even has Fabric OS 6.0.0c and 5.2.3. On the page they have listed release notes and a little further down there is a link called “Release 6 Firmware“.
Actually, if you click on ‘Release 6 Firmware’ you are taken to a page on brocade.com where you can find many different firmwares, including 5.x and 7.x
IBM also have a link about FOS 7.x.

Also note that some features does not exist/work on older Fabric OS. For example on Fabric OS 5.1.x DHCP and SCP may not work (which forces you to use static IP and ftp).

Equivalent Product Names

Page with the equivalent Brocade and HP product names.
Page with the model number as seen in switchshow and HP’s model and Brocade’s model. This is a good one.
Page for correlating IBM and Brocade product names.
Page correlating EMC and Brocade Product names

Recommendations

HP recommend that you upgrade one fabric and one switch at a time.
Waiting a week or at least a couple of days after you upgrade the first fabric is a good idea – gives you time to see if anything went wrong, if you can fix it and if you can do anything different next time.
See HP SPOCK for more details in regards to compatibility and interop modes.
The HP B-series Connectivity Stream lists the recommended firmware and all the supported ones for each switch model. It also has a list of the supported SFPs. Find it by clicking on “Switches” in the left-hand navigation pane under the “Other Hardware” section. The Connectivity Stream is great and it is updated often so I will not link directly to it. You need an HP Passport to log on to HP SPOCK – it is free to create and you do not need a contract or product in warranty.

Other vendors have similar matrices. HP for example does not have a list stating which Fabric OS firmware is supported with which HP P6000 firmware. The idea is that you go with the general recommendation of Fabric OS firmware.

Do read the release notes for the firmware(s) you decide on: for example not all 4GB SAN-switches can run the 6.4.x FOS. The 8- and 16-port 4Gbps switches (Brocade 200E) do not run 6.4.x or 6.3.x.
Only 8Gb and 16Gb switches can run the 7.x.x FOS.
The release notes also have the fixes, enhancements, upgrade paths and supported switches.
Generally the Brocade versions of the release notes are more verbose when it comes to fixes, but if you have an HP branded it might be easier to use the HP one as that has the HP names of the products. Also it might be hard to find the Brocade release notes if you do not have a contract with Brocade. Other vendors (like IBM/Fujitsu/HDS) provide you with the Brocade version of the release notes. You can find the release notes from their support pages.

Do  consider updating OS patches, HBA drivers/firmware, management softwares and storage drivers/firmware. For example Qlogic have driver updates to their drivers that prevent HP blades from getting stuck in G_port after a reboot.

Upgrading Tools

SANLoader is an unofficial HP tool to upgrade firmwares. With this you do not have to create an ftpserver etc. Contact HP Support, they may give this to you.
This is meant to be used when the other ways does not work, but it helps out a lot as you do not have to set up an FTP/SCP server.

Sanloader used to (winter 2010) not work well on Windows 7 and may not work flawlessly on the pre 6.x firmwares.

Other ways:

  • Set up a ftp/scp server and upgrade via the CLI (command line interface).
  • Use DCFM ( Data Center Fabric Manager – now called Network Advisory ) to upgrade firmware.
  • Firmware can also be upgraded through the web interface (click on switch admin and then on firmware download). You will still need an FTP/SCP server for this though. See the web tools admin guide page 73-74 (FOS 6.2.x but it hasn’t moved).

FileZilla is a free FTP-server that works well. There are many alternatives around.
For FTP clients:

  • /usr/bin/ftp in Ubuntu Server
  • WinSCP for a free opensource Windows alternative that does both ftp and SCP (and more).

For SCP any machine with Linux and sshd on should work. You can also get an scp-server running on Windows, OpenSSH would work. Both protocols are old, SCP is safer while FTP is sending data in clear text.

Personally I like doing this via the CLI. The DCFM way gives you the possibility to upgrade in parallel, but that’s also risky. If you use a Linux server to provide the firmwares via SCP don’t forget to let the switches in via firewall or tcp.wrapper ( /etc/hosts.allow ). If you do the upgrade via ftp – make sure that passive and active ftp both works.

How to access the SAN-switch

The most common way is to access the CLI of a Brocade switch by connect to the IP of it with an ssh- or telnet-client, PuTTy is the name of a free Windows client. Telnet is unsafe so do try to use the ssh at all costs.
It’s also possible to access the switch CLI via a serial cable, however as the firmware files are several 100MB (approaching 1GB for 6.4.x) that’s not really viable when upgrading firmware. Hyperterminal is a free windows tool that comes by default in some Windows versions. You can also use PuTTy for serial access.
To access the web interface just point the web browser to http://ip.of.the.switch. It requires Java. The Java version that’s supported is listed in the release notes of the Fabric OS.

Upgrade

Here on HP’s Support Forum are some more notes about v6.x. Basic steps:

Note: version 6 does not require to specify the exact folder location SWBDxx: it just needs the root containing “the install” file

1) Unpack the downloaded firmware in the FTP or SCP download directory
2) Start the FTP/SCP Server and allow access
3) Connect to the CLI of the switch via telnet or ssh
4) Type this in the CLI: firmwaredownload
5) Answer all questions: when it asks for File Name be sure to write /v6.4.1b, that is the folder under which you find all the SWBDxx folders. Failing to do so makes it impossible to download the firmware
6) Wait for reboot of the switch and reconnect, check the firmware version with the “version” command

More notes about the upgrade

CLI Command to start the update process is firmwaredownload – this starts the interactive version, it is possible to specify user, directory, host directly via the CLI. See the Command Reference Guide for details. There are reference guides for each major Fabric OS release.

Specifying Directory

Please use forward slashes when specifying directories.

For example when you unzip the firmware file and it creates a sub-folder in the FTP-root that is called v5.3.1a then you need to specify /v5.3.1a as the directory.

For firmwares prior to 5.3.x you have to specify the release.plist - /v5.2.2a/release.plist.
However it says in the release notes for 5.2.3 that release.plist is no longer needed.

In some cases you may have to specify the sub directory.
For example the 4/16 HP Switch is a Brocade 200E with switchtype 34. So you would then use directory SWBD34 – /v5.3.1a/SWDB34. You can also try with /v5.3.1a/release.plist, /v5.3.1a/SWDB34/release.plist or /v5.3.1a/install. However with 5.3.1a you should not have to so /v5.3.1a should be enough.

firmwaredownload example:

switch:admin> firmwaredownload
Server Name or IP Address: IP.TO.SCP.SERVER
User Name: username
File Name: /path/to/v6.2.2e
Network Protocol(1-auto-select, 2-FTP, 3-SCP) [1]: 3
Password:
Server IP: IP.TO.SCP.SERVER, Protocol IPv4
Checking system settings for firmwaredownload...
System settings check passed.
You can run firmwaredownloadstatus to get the status
 of this command.
This command will cause a warm/non-disruptive boot on the switch,
 but will require that existing telnet, secure telnet or SSH sessions
 be restarted.
Do you want to continue [Y]: y
 Firmware is being downloaded to the switch. This step may take up to 30 minutes.
 Preparing for firmwaredownload...
 Start to install packages...
 dir ##################################################
 [[lots of these for all packets]] ##################################################
 [[also stuff like these are seen many times:]]
 warning: /etc/fabos/pki/switch.0.rootcrt created as /etc/fabos/pki/switch.0.rootcrt.rpmnew
 kernel-module-ipsec ##################################################
 Removing unneeded files, please wait ...
 Finished removing unneeded files.
All packages have been downloaded successfully.
 Firmware has been downloaded to the secondary partition of the switch.
 HA Rebooting ...

Transfer Protocol and Connectivity

If you are using SCP and that does not work, please try with FTP. If neither works, see if something else can log on to the FTP/SCP server. And of course, make sure the right permissions/root directory are set on the FTP-server. If your FTP/SCP server has log files, check them. If it works from one client but not from the switch, check the logs and see if there’s a difference.

If you are logged on as root on the SAN-switch you can use the scp- or ssh-client on the switch to confirm connectivity, like this:

ssh username@server ls /tmp/v6.0.1a to list the /tmp/v6.0.1a on the SCP server.

You need to be root to run the above command.

If that also does not work, you have some kind of networking problem – you can try direct connecting a laptop to the LAN interface of the switch. To see the network settings on the switch: ifmodeshow and ipaddrshow

Example of 4/256 SAN Director Firmware upgrade:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c01639537

Also has example of how the firmwaredownload looks/works.

Another example with some pictures – this one – by IBM in a pdf. It says AIX all over the place but the instructions are for the SAN switches anyway.

Passwords

Sometimes when upgrading from 6.1.1d to 6.2.2 we have seen that the passwords have gotten reset.

Default password is then “password” or “fibranne”.

You can reset the password with the CLI command “passwd admin” to reset password on the admin account.

If you forget all passwords it might be possible to be able to reset it via the serial cable interface while booting the switch.

On EMC branded switches the default password might be: Serv4EMC

CF Cards

If your switch is out of warranty/contract and it’s still working. I’d suggest making a copy(dd  in linux for example) of the CF-card. Then if the CF card decides to fail you can just get a new one from random_electronic store and dd the contents of the flash back.

http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003855

Licenses

When replacing a switch make sure that the licenses are correct.
If for example you have a switch with ‘power pack’ – then for HP there is a special spare part number for a switch with power pack and one without. Power pack is a grouping of licenses,  which are in the pack differs between models.

HP’s page to generate new licenses is: https://www.webkey.external.hp.com


About this entry