13

I've got an ASA 5550 that is performing loads and loads of operations (AnyConnect, NAT, ACL, RADIUS, etc, etc). It isn't particularly overloaded in terms of CPU & Memory, but it has an uptime of over 3.5 years.

Lately I have been attempting to deploy another IPSEC tunnel (via cryptomap) along with a NAT Exempt rule, but the ASA is exhibiting very strange behaviour. Sometimes when I add ACE's a mass of text pops up from nowhere in the description field. No matter what I do, my tests with the on-box PacketTracer tool do not yield the results I expect (for example - I see the packet hitting the Any/Any rule at the bottom of the ACL, even though there is a specifically configured ACE at the top of said ACL).

Anyway, the question is this: Has anyone ever actually solved anything by rebooting an ASA? It isn't my favourite option, but with the very strange behaviours I am seeing troubleshooting is becoming fruitless.

Improve this question

5 Answers 5

Reset to default
18

Short answer: Yes.

Longer answer: :-) There are bugs in every piece of software. The longer it runs, the more likely one is going to setup shop in your network. But more to the point, the longer it's gone without a reboot, the more little bits of "old" configuration and/or status will be left lingering. In IOS, no interface foo will emit a warning that it's not completely destroyed and configuration elements may reappear if you recreate the interface -- shouldn't happen in an ASA but in rare cases, it does. I've also seen phantom NAT entries after deleting them from the config. (that one actually is a bug)

When dealing with IPSec/crypto, I've found a whole lot of crazy can be cleared up by a reload. In one case (pix 6.3.5) it wouldn't re-establish a VPN tunnel until I did.

[edit] A word on reboots in general: I tend to reboot things just to make sure they will. All too often I've had various systems (routers, firewalls, servers) running for extended periods -- constantly being modified, and when something ends up restarting them (usually a power outage, but "oops, wrong machine" happens too) they rarely come back up exactly as they were before... someone forgot to make X start at boot, or some odd interaction of parts makes something not startup as expected. I admit, it's less of a concern for more static parts of one's infrastructure.

Improve this answer
1
  • 1
    Great answer, and I fully agree that you need to make sure your devices boot as expected. I also agree that reloads are sometimes necessary (in fact, may be the only recourse) and can restore service faster. I just have run into too many cases where a reload is perceived as the fix rather than a step to resolve the current symptoms. No exploration of root cause is done and no pressure is put on the vendor to fix the problem if it is in their code. Even worse are the cases I have come across where "a reload every [period]" is the standing fix, when there is a code upgrade with a real fix.
    – YLearn
    May 21, 2013 at 1:06
7

Generally, I don't recommend a reboot as a resolution to a problem unless you know you are dealing with a bug that introduces something like a memory leak or a cache overflow condition.

With an ASA running an image at least 3.5 years old, have you checked the Cisco bug toolkit? Odds are that any bugs in the platform will be documented and you can see if any look to apply.

I would also recommend opening a TAC case if you have support.

Reboots in my mind gloss over other problems and can make it very difficult (if not impossible) to find root cause. Ultimately without understanding the root cause, you don't know that you fixed anything and I find that very dangerous, especially on a "security" platform.

For instance, maybe you have a security vulnerability in the code that is being exploited by an outside source. While the reboot may cut off their connection and alleviate the symptoms, it does nothing to address the problem.

Improve this answer
2
  • I agree with you 100%. Obviously, some updates and patches need to be done on the device. I've yet to do a bug toolkit search, because identifying this particular issue isn't an easy thing to do - so where do you start searching? But, to fill in the gaps, this particular change will be temporary, as a larger project to redesign the network is underway.
    – BrianK
    May 21, 2013 at 1:50
  • 1
    Sounds like you have a good stance on things. TAC isn't what it used to be, but I always recommend a TAC case (if you aren't used to it, the bug tool can be quirky). Let them find out which bug it is, although you may have to push them to do so. Just make sure to capture as much data before the reboot as possible as some details will be lost (running processes, memory usage, etc). A "show tech" should get most of what you need on a Cisco platform.
    – YLearn
    May 21, 2013 at 2:14
3

As mentioned, risk management and vulnerability management should be your concerns. I'd say there are at least 10-20 known vulnerabilities for your ASA software version, assuming you had the latest firmware installed at the time represented by uptime.

Tools.cisco.com link, with vulns for the past year (some aren't relevant, but this should give you a good idea)

Some other tools that may help you:

  • Cisco Security IntelliShield Alert Manager - determine if network, hardware, and software assets are vulnerable to new and existing threats

  • Cisco IOS Software Checker. I don't know if there's something similar for the ASA, but perhaps someone could chime in?

  • Router Configuration Auditing: RedSeal may include version checks (it's been several years since I've used worked with it), as well as plenty of other security tools for networks

  • Vulnerability Management: Nessus has community and commercial versions, and there is plenty of other software like this out there

Improve this answer
2

I have recently encountered similar problems from an ASA running 8.2(2)16 with ~2.5 years uptime, whereby object-groups specified in crypto map ACLs were not being matched. Adding an ACL statement that the object-group already encompassed caused interesting traffic to be matched. Very frustrating.

A colleague advised they had seen this behaviour previously and that a reload resolved it in that instance.

Improve this answer
0

When you say a load of 'random' text is appearing when adding ACE's, are you manually typing these ACE's in or are you pasting them from some other source (like notepad).

I have seen issues before where if you are pasting a lot of lines into a device it can get overloaded and some corruption occurs, pasting less lines usually fixes it or using a function on your terminal program to 'paste slow' to allow for a small time gap between each line.

Improve this answer
1
  • I am manually creating a new ACE via ASDM. If the rule contains a particular source network (whether I use the network object, a group object, or simply type the subnet) the ACE appears with about 30 lines of description. The text isn't completely "random", it appears to be comments that were used once, on an ACE somewhere... But I haven't ever typed it all in...
    – BrianK
    May 23, 2013 at 4:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.