Analysis of the Palo Alto Cache Poison Issue

By | January 3, 2013

I admit to being a little tough on Palo Alto Networks (PAN). I called them a cult (see Cult of Palo Alto Networks) and criticized them (along with Gartner) for pointlessly creating the Next-Generation Firewall market segment (see UTM v NGFW – A Single Shade of Gray).

Now along comes some bad news for PAN.  In the last week, the Internet has had numerous postings about a method to bypass Palo Alto’s Application Identification capabilities (called AppID). Allegedly, Check Point is behind this disclosure but the issue is actually over a year old. There are some nice videos posted that describe the vulnerability, which are allegedly from CheckPoint:  PAN Security Bypass  and a followup here PAN Security Bypass Take 2.

I spoke with technical people at Fortinet, WatchGuard and SonicWall about this issue.  All of them were able to demonstrate that their products are not vulnerable to this issue.  I was able to independently validate this on a Fortinet, but did not have other devices to test.  I will have to take SonicWall and WatchGuard at their word.  Juniper and Cisco do not do application filtering, so they are not part of this issue.

PAN responded to this with the following blog entry:  AppIP Cache Pollution – Merry Christmas from CheckPoint.

Analysis

On one hand, I feel badly for PAN.  This issue is being blown out of proportion somewhat.  It is a serious weakness, but not crippling.  There are ways to configure their appliances to remedy this issue.  As such, I have a modicum of sympathy for PAN.  They make a very innovative product that has just experienced its first swat of reality.  Welcome to the club PAN, you are now like every other technology company on earth who has had a beloved technology lose some of its luster from a flaw.

On the other hand, PAN asked for it.  PAN has been arrogant to the market.  They have repeatedly acted as if their products were simply “above” the fray and not worthy of comparison with others.  They deftly got Gartner to go along with them on this charade and create a whole new market segment for them, NGFW.

CheckPoint may be behaving unfairly, but PAN kind of asked for it.  PAN blazed on to the scene claiming revolutionary technology that upon deeper inspection has proven to be only moderately evolutionary.  They have used marketing muscle to rewrite the history of security to suit their needs.

PAN painted a big red target on their backs. This is the outcome.  Your competitors are going to shoot at you every chance they get.  And CheckPoint is especially bitter. Their market share has been in free-fall for years.  What do they have to lose at this point?

PAN has joined an illustrious crowd, however.  They have become the market target.  Much like Microsoft in the past, Apple today, and Google and Facebook are becoming.  As such, PAN can congratulate themselves on being able to crack open the firewall market and quickly race to the top.  However, the view from the top is not always rosy.  Down below are a lot of angry companies who feel PAN’s race to the top was unwarranted.

They are partially correct.  PAN got to the top mostly on their merits. While their product is only marginally innovative, their marketing is phenomenal.  PAN should become a case study for business schools everywhere on the power of unified marketing and control of language.  Everybody in the PAN marketing department should be awarded a lifetime achievement award.  They took a good to okay product and made it look like a billion carat diamond.

However, marketing only goes so far.  At some point, marketing must meet up with reality.  And PAN just got a big dose of reality.

The reality is that this is a very serious flaw. Cache poisoning is a common attack tactic. It is a method many novice hackers use and their are ample tools out there to facilitate this attack. Moreover, this calls into question all of PAN’s performance numbers, since apparently all of their published performance statistics are based on having this “fast pass” feature enabled. What does turning this off, and thereby correcting this weakness, do to appliance performance?  Moreover, this entire vulnerability calls into question PAN’s entire approach to security, casting off the tried and true use of stateful packet inspection for application identification.

Also, this problem has been around for a while.  According to my research and what others have reported, this issue was first disclosed at CanSecWest in 2011. PAN just released a whole new operating system, why wasn’t this issue corrected? Is it even correctable?

There are a lot of disturbing questions that arise from this. I do not mean to be overly critical of PAN. I know many people cast me as a Fortinet-phile.  While I do like Fortinet I also am the first to admit that Fortinet has plenty of weaknesses too.  I actually think Fortinet and PAN are the pinnacle of the UTM market right now.

Advice to PAN

Be humble.  Do not turn on CheckPoint and be rude or derisive.  This Blog entry posted on December 28 should be taken down and promptly rewritten.  Thank the industry for detecting this.  Make it clear you understand the issue and are working with clients to remedy their concerns.

You come out swinging and acting tough, then we all know you got caught with your pants down.  That blog post is arrogant and dismissive.  Get rid of it. Call up McAfee and have a conversation with their management about how to handle a crisis.  Call up Tim Cook at Apple, get a lesson from him in humility.

PAN is a good technology.  It is innovative.  It can be an integral part of a secure network.  But nobody is going to see that if you let this become a schoolyard slap fight. Grow up and be humble.  Those who made it to the top because they were good do not need to attack others when they make a mistake. Great people and great companies know when to admit they made a mistake.

7 thoughts on “Analysis of the Palo Alto Cache Poison Issue

  1. Heath

    Just wondering if PA ever fixed these issues in their latest releases?

    Reply
  2. SE

    So if you enable application default as a remedy anyone can run anything as long as it doesn’t change the ports?
    Doesn’t this sound a lot like a packet filter?

    On the other hand, one could disable the cache, get a big performance hit and misidentify a lot of applications.

    Sounds like PA are having an architectural issue solving this otherwise itt would have been solved promptly since its a gaping hole in the whole AppID story.

    Reply
  3. egress_data

    A bit confused by this:
    “There are ways to configure their appliances to remedy this issue.”

    If you re-watch the 2nd video (PAN Security Bypass Take 2), they are still able to use cache poisoning to by-pass the firewall even with utilizing application-default for allowed services, and any for the explicitly blocked applications (vendor recommendation). When you disable the application cache, you essentially disable the heuristic layer (and application catch rate) of how their App-ID works. Most P2P applications now show up as “unknown”. Not much of a remedy…unless you know something that PAN has yet to release.

    Reply
  4. Natt

    May I response to this statement “Juniper and Cisco do not do application filtering”?

    Juniper SRX series have rich feature of application filter. They can compare to other NGFW product vendor. Also, Cache poisoning issue on NGFW was introduced by Brad Woodberg (Juniper employee) at Defcon 2011.

    Reply
  5. edub

    Having followed paloalto/PA/Palo Alto/?? for a while now, I am still reluctant to jump on the bandwagon given the continued brashness in press releases, amateur approach to release management (looks like they have the ASCEND communications syndrome- too many releases of poor quality and break stuff too) and bodacious flaws in the super slick “extra” features (most of which I already cover with AV, content, e-mail and router filtering along with other things that have been around for a longgg time…

    Even some of the long time security peeps are singing the praises but I’m still not sold. Who else is out there that comes close to the potential of what PA claims to be able to pull off?

    Reply
  6. wtf_pan

    What IS IT with this new industry trend of trading off security for performance, off all things on a firewall?

    First it was DSRI: disable server response inspection. That should make you cringe right off the bat. Clarified here: http://media.paloaltonetworks.com/documents/Single_Pass_Parallel_Processing_Architecture.pdf.
    Hey, it boosts the unit’s performance by double so thats good!

    Then AppID caching, already discussed ad-nauseam. Yet another performance boosting, security crippling technique.

    PAN’s response is laughable: “oh you really want a no-port approach and all this positive enforcement is good for you, but basically to fix this gapping hole you have to tell the firewall to go back to considering the app’s default ports and start using negative enforcement web filtering rather than our application rules”. Back to square one.

    Then you can talk about PAN’s stream inspection of malware – its an IPS that picks up malware and attacks so yeah, single pass. You cannot inspect a stream of data transfer and really come out expecting that this will have the same malware detection capabilities as a store and forward approach. Devoid of sandboxing, heuristics, etc.. but again, its faster.

    I’m done ranting.

    Disclaimer: I use firewalls, and I prefer the ones that offer performance without trading off security.

    Reply
  7. spack

    Great advice to PAN. I had the PAN guys at my office a few weeks ago. The demo they gave was good. I was impressed as was the other network guys. We were going to do a proof of concept, but then I started researching the company. I also talked to some companies using PAN. The stories were all pretty consistent, it looked great but then once we started using it, it was hell.

    I really like your blog. You have a real no-nonsense approach and the security world needs more of that.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


*