Recovering from a Cisco 9500 Firmware ISSU Upgrade

Today, I had a activity to execute an IOS XE upgrade on a Cisco 9500 switch using from the current version to a newer version which is running in SSO mode.

I started by performing pre-checks on the verify that the switch is in the correct state and working as expected.

The pre-checks included:

  • Check the switch current IOS XE version

    C9500# show versions | in IOS XE
  • Check the switches connection

    C9500# show switch
  • Check the Flash free space

    C9500# dir flash: | in free
  • Check the state of switch ISSU

    C9500# show issu state detail
  • Check the install state

    C9500# show install summary

  • Check the switch is in SSO mode

    C9500# show redundancy
  • Check the switch that auto-boot is enabled

    C9500# show boot system
  • Check the exiting boot file on flash

    C9500# dir flash: | in bin


    Once the pre-checks has been completed. It is time to clean up the previous installation files.
    Use the following commands to perform this activity:
  • Copy the old image and backup current configuration to TFTP or USB drive

    C9500# copy bootflash:<old_image.bin> usbflash
    C9500# copy running_config usbflash

    OR

    C9500# copy bootflash:<old_image.bin> tftp:
    C9500# copy running_config tftp:
  • Remove any old image installation files from the flash (note that if you copy new image before this command, it will be included in the list of files to be removed)

    C9500# install remove inactive
  • Copy new image to be switch flash

    C9500# copy usbflash:<new_image.bin> bootflash:
    C9500# copy tftp:new_image.cfg bootflash:
  • Verify new image hash with Cisco website

    C9500# verify /md5 flash:<new_image.bin>
  • Start the ISSU to upgrade IOS XE

    C9500# install add file flash:<new_image.bin> activate issu commit

My colleague had recommended and experiencing switch upgrades first hand, it is a good practice to monitor the switch’s IOS XE upgrades from the console port while the switch is reloading.
You have visibility of any errors that shows on the console.
It is worth having the console session logged during the activity for reference that may be used for troubleshooting and support.

Recovering from the standby switch that booted into ROMMON mode with the error failed to Boot URL not found or incorrect.

  • Executed these commands on the standby switch in SSO mode

    ROMMON 1> dir flash:
    ROMMON 1> boot flash:<image-name.bin>
    OR
    ROMMON 1> boot flash:package.conf00-

The standby switch booted into the previous state.

  • Aborted the ISSU state on the active switch before standby switch loaded

    C9500# install abort issu

  • If ISSU failed to abort, these are other useful commands that was not necessary in my case. Enable service install that is required to clear install state

    C9500# configure terminal
    C9500(config)# service internal
    C9500(config)# end

    C9500# clear install state

I hope these commands were useful for managing switch upgrades.

reference:

https://www.cisco.com/c/en/us/support/docs/interfaces-modules/catalyst-6000-backplane-clock-modules/223261-troubleshoot-issu-upgrade-failures-on.html

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-18/configuration_guide/ha/b_1718_ha_9500_cg/configuring_issu_on_stackwise_virtual.html

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9500-series-switches/214406-in-service-software-upgrade-issu-on-ca.html#toc-hId–458892867

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9500-series-switches/222279-upgrading-catalyst-9500-switches.html

Disable IE Enhanced Security Configuration in Windows Server via script

I have always want to use a script to disable Internet Explorer Enhanced Security Configuration (IE ESC) in Windows Server. It is an annoying activity when it is required to perform tasked using the IE in older versions of windows servers.

It is important to note that IE ESC is very important to the security of legacy servers from common past vulnerabilities which is not a risk in current versions of server OS.

This activity must be taken with precaution and on servers with access restriction to the Internet. It is encouraged that IT professional implement all measures to secure the legacy application running on these servers that must be maintained in an segmented or isolated environment. It is the responsibility of the IT professional to use a secure alternative browser after this activity has been performed.

Now here is a batch script to the rescue taken from this Microsoft FAQ that can be runned on all your servers. Save the script in a batch file and run it from the command line.

The script must run as an administrator or a logon script to work since it is making changes to the registry.

Note: The script was created for Windows Server 2003 and 2008. However, it will work for other versions such as Windows Server 2019.

ECHO OFF
REM  IEHarden Removal Project
REM  HasVersionInfo: Yes
REM  Author: Axelr
REM  Productname: Remove IE Enhanced Security
REM  Comments: Helps remove the IE Enhanced Security Component of Windows 2003 and 2008(including R2)
REM  IEHarden Removal Project End
ECHO ON
::Related Article
::933991 Standard users cannot turn off the Internet Explorer Enhanced Security feature on a Windows Server 2003-based terminal server
::http://support.microsoft.com/default.aspx?scid=kb;EN-US;933991
:: Rem out if you like to Backup the registry keys
::REG EXPORT "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" "%TEMP%.HKEY_LOCAL_MACHINE.SOFTWARE.Microsoft.Active Setup.Installed Components.A509B1A7-37EF-4b3f-8CFC-4F3A74704073.reg"
::REG EXPORT "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" "%TEMP%.HKEY_LOCAL_MACHINE.SOFTWARE.Microsoft.Active Setup.Installed Components.A509B1A8-37EF-4b3f-8CFC-4F3A74704073.reg"
REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" /v "IsInstalled" /t REG_DWORD /d 0 /f
REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}" /v "IsInstalled" /t REG_DWORD /d 0 /f
::x64
REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432node\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}" /v "IsInstalled" /t REG_DWORD /d 0 /f
::Disables IE Harden for user if set to 1 which is enabled
REG ADD "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap" /v "IEHarden" /t REG_DWORD /d 0 /f
REG ADD "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap" /v "IEHarden" /t REG_DWORD /d 0 /f
REG ADD "HKEY_CURRENT_USER\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap" /v "IEHarden" /t REG_DWORD /d 0 /f
::Removing line below as it is not needed for Windows 2003 scenarios. You may need to enable it for Windows 2008 scenarios
::Rundll32 iesetup.dll,IEHardenLMSettings
Rundll32 iesetup.dll,IEHardenUser
Rundll32 iesetup.dll,IEHardenAdmin
Rundll32 iesetup.dll,IEHardenMachineNow
::This apply to Windows 2003 Servers
REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OC Manager\Subcomponents" /v "iehardenadmin" /f /va
REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OC Manager\Subcomponents" /v "iehardenuser" /f /va
REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OC Manager\Subcomponents" /v "iehardenadmin" /t REG_DWORD /d 0 /f
REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OC Manager\Subcomponents" /v "iehardenuser" /t REG_DWORD /d 0 /f
::REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}" /f /va
::REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}" /f /va
:: Optional to remove warning on first IE Run and set home page to blank. remove the :: from lines below
:: 32-bit HKCU Keys
REG DELETE "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main" /v "First Home Page" /f
REG ADD "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main" /v "Default_Page_URL" /t REG_SZ /d "about:blank" /f
REG ADD "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main" /v "Start Page" /t REG_SZ /d "about:blank" /f
:: This will disable a warning the user may get regarding Protected Mode being disable for intranet, which is the default.
:: See article http://social.technet.microsoft.com/Forums/lv-LV/winserverTS/thread/34719084-5bdb-4590-9ebf-e190e8784ec7
:: Intranet Protected mode is disable. Warning should not appear and this key will disable the warning
REG ADD "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main" /v "NoProtectedModeBanner" /t REG_DWORD /d 1 /f
:: Removing Terminal Server Shadowing x86 32bit
REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Terminal Server\Install\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap" /v "IEHarden" /f
:: Removing Terminal Server Shadowing Wow6432Node
REG DELETE "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Terminal Server\Install\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap" /v "IEHarden" /f

The script may display errors for some commands depending on the versions of Windows servers due the registry keys not used in that versions.

Most importantly, ensure to verify the settings changed for the Internet Explorer ESC.