Is there a checklist for replacing failed ASA hardware?

Question

I have a failing Cisco ASA 5505 that is being replaced under a support contract. It is a single-point-of-failure in production at a branch office that can tolerate downtime. I will physically replace the device at that site myself. Prior to the hardware failure the configuration was just fine. My goal is to get back to the working state prior to the hardware failure.

While I have worked a fair amount with Cisco ASAs, I have never had to perform a hardware replacement so this is new territory for me. What I am looking for is a checklist, "runbook", or procedure to serve the following purpose:

• Help to ensure I don't miss some critical configuration step.
• Help to avoid some critical configuration error.
• Aid in advance preparation to reduce the downtime while the hardware swap takes place.
• Provide some advance insight into how long this is likely to take.

Off the top of my head the following things that I ought to consider come to mind:

• Document cable connections.
• Export the current running-config.
• Document the old ASA's firmware version.
• Change the new ASA's firmware to the same version as the old one.
• Do admin passwords need to be set separately from the config?
• Where are VPN user passwords stored?
• What about certificates?

Things start to get pretty vague near the bottom of that list. I also feel like I might be missing some important items entirely.

What are all the things that need to be done when replacing failed Cisco ASA hardware?

Answer 1

One thing to add to the things collected is the licensed features (show activation-key detail)

This all depends on the asa model and running version. Some parts will be in the config, others in flash files, and yet others in "private" flash files. The quickest and most complete procedure would be to swap the compact flash. Keep in mind the activation key is stored there as well, so get that from the new system before the swap.

For example, my ASA has webvpn configuration components that aren't in the running config, or visible flash filesystem.

---

For all models, you'll have to open the case to get the CF. ASA5505, it's along the edge of the board, visible from the outside, but not removable until the cover is lifted. ASA5510/20/... the CF is on a board near the external CF.

Answer 2

I opted to use the Cisco support methods of swapping out my ASA. You might also consider Ricky Beam's suggestion to swap CF cards which is unsupported by Cisco

---

I just finished the swap and definitely have much more clarity about what needs to happen to gracefully swap an ASA5505. There will be, of course, site-specific considerations, but I have tried to mention anything that probably applies in general.

Setup and Pre-Configuration

These tasks can all be done before taking the old ASA out of service.

• Find a Second ASA Power Supply - AFAICT Cisco doesn't send you a second power supply to help you to replace failing hardware. If your ASA needs an external power supply and you only have one, you're going to have to take down the old ASA while you configure the new one. This creates a lot more downtime than is necessary.
• Document Cable Connections - You are going to be ripping and replacing-with-same. There is also the the possibility of reverting back to the original ASA if the replacement is DOA. There is also a likelihood that you'll need to perform partial connections as you get the replacement ASA up and running so you need a pretty clear way of matching cables to ports on both the old and new ASA.
• Diff show activation-key detail - Connect to the console of the new ASA and dump show activation-key detail to a file. Do the same on the old ASA. Diff the two files and see what stands out. The replacement ASA I received was activated to the wrong license and there was a wait to get new activation keys.
• Download Binaries from Cisco.com and old ASA - Download same-version binaries for the firmware and ASDM from both cisco.com and the old ASA. Compare checksums of the old and new binaries of each. If the cause of the problems was file corruption, the checksums won't match which might provide a clue.
• Save Output of Various Commands from old ASA - There are a number of items that are necessary or might come in handy that can only be obtained from the old ASA. I recommend running and saving the output of the following commands:
  • show version
  • show activation-key detail
  • show asdm image
  • show running-config
• Confirm Workstation/ASA Connectivity - You're almost certainly going to need both console and SSH access to set up the new ASA. You might as well ensure that those connections are working with the old ASA prior to replacing it with the new ASA.
• Set up a workstation with an independent network connection - There's a good chance you'll need to be connected via SSH to the new unconfigured ASA at the same time you need google. Accordingly, you need a connection to the internet that doesn't depend on troubled ASA.
• Load the new ASA with the right Binaries - I think there are a few ways of accomplishing this. I used the console of the new ASA to alter its running configuration just enough to enable TFTP of the appropriate firmware and ASDM binaries to the new ASA's flash.
• Change the new ASA's Firmware Version - Use the console to issue boot system, wr mem, and reload commands to switch the firmware version to the same as the old ASA
• Change the new ASA's ASDM Image - Use the console to issue asdm image and wr mem commands to change the asdm image to the same as the old ASA.
• Apply and Iterate on Running Config - I determined that it took some iteration to get the new ASA to accept all aspects of the old ASA's running-config. Here are the steps to that iteration:
  1. Paste the old ASA's running configuration into the console of the new ASA.
  2. Save the output of show running-config to a file.
  3. Diff that file with the configuration of the old ASA so you can see what is different.
  4. Issue corrective commands to try to correct the differences. Go back to step 2 and repeat until the old and new running configurations are the same.
• Generate New Crypto Keys - Use the console to issue a crypto key generate rsa command. Without issuing that command, you cannot connect to the ASA over SSH.

Rip and Replace

Once all the setup and configuration tasks are done, it should just be a matter of swapping hardware. This should take only minutes.

• remove the old ASA
• transfer any brackets and modules to the new ASA
• mount the new ASA
• connect all cables except power to the new ASA (you should connect the console port to a terminal so you can watch the boot)
• connect power to the new ASA
• monitor the boot sequence for success

Post-Replacement Tests

At this point, at least basic connectivity should be working on the ASA. To prove the different aspects of workingness, do at least the following:

• from the console, ping the following:
  • 8.8.8.8
  • the management workstation that will be connecting over SSH
  • any other node that will help you test that the new hardware is working as desired
• connect over SSH to the new ASA
• connect using ASDM to the new ASA
• test any VPN connections

Answer 3

You need to check which module (hardware or software) is installed on the asa with the command "show module all". The hardware module can be the IDS/IPS, the SSM, the CX or the FIREpower and the software module can be the CX or the FIREpower. Each module need a specific backup.

For the backup of the asa:

• starting from version 9.3 you can use the command "backup"
• before version 9.3 you can use the graphically asdm backup feature TOOLS -> Backup Configurations or using the cli backup all the options present on the asdm backup.

Reference