Stolen Pixels #260: The Dark Fortress

By Shamus Posted Tuesday May 24, 2011

Filed under: Column 47 comments

splash_psn.jpg

I have crafted for you a comic about Sony and their PSN hi-jinks.

One thing I found strange about the security announcement was the part where they promised “additional firewalls”. A firewall just blocks types of traffic, or traffic from certain locations. See, you can’t “stack” firewalls. Or at least, doing so shouldn’t make things any more safe. Using two firewalls for the same entry point would be like mashing two metal detectors together and making people walk through both at once. It doesn’t let you find twice as much metal, or find metal twice as fast. It doesn’t make one super-sensitive metal detector.

You should have exactly one firewall on every machine, and that firewall should only allow traffic that the machine is specifically designed to handle. The webserver should only allow web traffic, and block everything else. The database should only allow database traffic, and block everything else. The mail server shouldn’t accept web traffic and the FTP server should only speak to a narrow band of trusted IP addresses, ideally machines inside of Sony offices.

So what is the deal with adding “more firewalls”? Were there machines with NO firewalls on them? Or are they stacking firewalls? Or was this the layman’s way of saying, “We closed a bunch of ports that we shouldn’t have left open in the first place”?

 


From The Archives:
 

47 thoughts on “Stolen Pixels #260: The Dark Fortress

  1. Raygereio says:

    Or was this the layman's way of saying, “We closed a bunch of ports that we shouldn't have left open in the first place”?

    I think it meant: “Here’s an empty phrase with no meaning whatsover to convince you we improved security to prevent this from happening again. We used this empty phrase because we don’t want to interupt this announcement with a five hour long lecture on every single security upgrade we did”.

    Upside to using that empty phrase is that the majority of the people with little to no knowledge about computers will go: “Huh, additional firewalls? Yeah, I guess that’s good. Firewalls are a good thing to have, right?”
    Downside to it is that everyone that has slightly more computer related knowledge will go: “Wuh? What does that even mean? That’s not how that works.”
    Luckily for Sony the first mentioned group is probably a wee bit larger then the latter one.

    It just smells like something meaningless some marketing guy wrote. Actually Mr. Hirai’s entire speech smelled like that. Which is why I was really surprised when people liked it.

    1. Dev Null says:

      Also, we’ve reversed the polarity of the neutron flow with our sonic screwdrivers…

      1. Audacity says:

        Sonic screwdrivers? Ha! Everyone knows they should have used hydrospanners. Nothing builds better security than a wrench made of water!

      2. Bret says:

        Don’t those leave time a bit wibbly?

      3. Ninjariffic says:

        Bounce the graviton particle beam off the main deflector dish?

        Set up a neutrino field inside a can peas, hold on to Geordie’s visor and sing in to Data’s knees?

        1. Bryan says:

          Grab a tribble and some chewing gum, and stop the warp-core breach?

          Hehe. Voltaire. :-)

    2. Rayen says:

      I was actually thinking this. We are talking about PS3 players. They needed to say something that lent credibility to their security improvements and instead of going over what they really did, they used a buzzword that everyone knows and is synonymous with security. Added bonus the hackers don’t know quite what to expect if they try to hack again because sony didn’t really say what they did. It doesn’t have to be right or even make sense, the PSN users will go oh good. and the people who actually know about such things will have conversations like this on computers.

      1. Alexander The 1st says:

        Yeah, it’s generally nicer of them to be “We used multiple firewalls” instead of “Well, we have security measures in place, but we can’t tell you what we did, because that would tip off the hackers, and we want to keep them in the dark.”.

        1. JA says:

          Odd, saying that “we have extra security now, but telling you what they are would give the hackers a leg up” is a lot more reassuring to me than, “blah, blah; nonsense, nonsense”.

    3. Yar Kramer says:

      The reason people liked Mr. Hirai’s speech was because he actually behaved in an apologetic manner towards the customers and only the customers, and only used terminology which is empty in hindsight, rather than doing a deadpan “A bad thing has happened — Not that it’s remotely our fault or anything!” speech targetted at lawyers and shareholders that used transparently-empty terminology.

      1. Raygereio says:

        I’ll admit that I liked the speech. It was well written, well preformed and well produced. But that’s all it was: well done. It didn’t have any substance beyond that. We didn’t get solid information on what really happened, for instance.

        All we really got was some Sony-guy saying they’re sorry. Something which we already got multiple times, just not in such a well directed manner. Erm… okay, I guess. I don’t know. Maybe I’m just immune to those evil marketing schemes. I knew that tinfoil hat was a good investment.

        1. Alexander The 1st says:

          Their blog went into more specifics, like detailing the difference between hashing and encryption regarding the passwords, for example.

  2. Nathon says:

    1 firewall per machine is pretty serious overkill. These are not pieces of windows software that close ports; they’re big boxes that sift through traffic and look for nasty bits. Adding firewalls in parallel can actually improve throughput and reduce the temptation to let some traffic route around the firewall. There are also fancy things like IPSs that protect against stuff firewalls only dream about, but throwing those terms around in a press release for general consumption seems silly.

    Anyway, my point was that “more firewalls” likely means “more diverse defenses” and “more layers of defense”. Of course, this is all speculation. Maybe they did mean that they added iptables to their ipchains.

    1. Sumanai says:

      A thought occurred to me, maybe they put additional firewalls (servers) in parallel. So secure server would be behind its own firewall instead of a shared one.

    2. mneme says:

      This. The modern idea of firewalls as port blockers on machines (or using iptables on your out-facing boxen) is just that, modern — and is really best as a second line of defense.

      A firewall, traditionally, is router which has enough info on what should and shoudn’t be entering your system that it can block stuff from even reaching machines — meaning if there’s a bug or workaround for the destination machine, it may still not reach it. It also can do basic but essential stuff like catching forged packets that claim to be from inside your network (which is one reason simply having per-machine firewalling isn’t sufficient), can give you statistics (or block traffic patterns) for traffic across machines, and can act as the first line for dos attacks. Obviously, a firewall isn’t sufficient — with just a firewall, you’ve got a hard outer shell and a soft chewy center (what I usually call “the lobster defense”), but it’s an important part of security.

      My guess is that what Sony means is that they’ve segmented their architecture into more classes of machines and put firewalls between those classes — so capturing an external machine doesn’t immediately give you the ability to launch an attack on the database (or at least doesn’t give you the database outright), and it’s much harder to escallate an attack into something expensive.

      1. I was going to say, I’m pretty sure the credit union my housemate works for uses multiple layers of nested defenses (including firewalls), but they have a hierarchical system setup and all kinds of disaster recovery and so forth. They also have numerous servers that are more public access than other servers, so there is often a firewall in between the more public servers and the more secure ones.

    3. krellen says:

      I suspect they’re talking about having set up a DMZ for their servers. Possibly they’ve gone from a single firewall to a dual firewall structure; it’s also conceivable that they’ve simply gone from a non-DMZ set up to a DMZ set up.

      1. Nathon says:

        I find it hard to believe that a global network like Sony’s didn’t already have its public facing servers in an isolated subnet. I don’t think that alone would have prevented this from happening, either.

        1. Yar Kramer says:

          Knowing the gigantic security holes they knew about but refused to fix in the first place, nothing would surprise me now.

      2. Strangeite says:

        A Demilitarized Zone has worked wonders on the Korean peninsula. Although you run the risk of the servers on the north side of the DMZ becoming meglomaniacs with a penchant for American action films.

        1. krellen says:

          Fortunately, I did link the article about the term’s use in computing, and not by the military.

        2. Will says:

          I’m pretty sure the servers on the north side of the DMZ were megalomaniacs before the DMZ was put in place, in fact i’m pretty sure that is why the DMZ was put in place. Can’t have your meglomaniacal servers taking over everything and declaring themselves glorious leaders.

    4. DanMan says:

      I agree. One core philosophy in security is to validate on every transfer point. For example, the form to post a comment on this blog. If I don’t enter an email address or name, the webpage should give me an error to prevent me from accidentaly inputting invalid data. However, the PHP server should not assume that the webpage validation did it’s job, since I could try to submit a request outside this form. It should validate the data to make sure I am not intentionally inputting invalid data.

      It’s possible that Sony had its servers set up so that if you send a request from behind the firewall, then you are allowed the information you are asking for. They may be adding firewalls between servers. Rather than assuming “you are requesting data from behind the firewall, you must be allowed” they are interigating the request. For example, the data server should not be making FTP requests.

  3. Sucal says:

    For some reason this whole situation is reminding me of the newest Die Hard Movie. At least I’m not living in Japan or America, when the evil hackers steal all your internets

    1. Klay F. says:

      You better be careful or the hackers will hack your computer and make it blow up by remote.

  4. Dev Null says:

    and the FTP server should only speak to a narrow band of trusted IP addresses

    and the FTP server should be in the basement with all the other antiques, not plugged into anything.

    Fixed that for ya.

    1. Nathon says:

      Sadly, a lot of web hosts still use FTP. It’s horrible.

    2. Shamus says:

      Ha!

      I ALMOST did the same joke, then decided to play it straight. And then you made the joke anyway. And I still laughed.

      Win.

      1. Dev Null says:

        Great minds…

        (must have been off doing something useful, like curing cancer.)

  5. ozmasis says:

    I think they mean actual walls of fire, should go great with the barb wire

    1. Alexander The 1st says:

      I’m sold on that security.

      When all else fails, use Fire.

  6. John Lopez says:

    Security is about defense in depth. A flaw in one part of the system should not compromise the entire network… and there will be flaws. Even vendors like Cisco issue security patches, so relying on one “firewall” layer for all your security needs is just waiting for that flaw to manifest.

    This isn’t to say that layers makes things safer in the default case (when everything is working) but it can give you that fall-back situation. In my case, I am responsible for an insurance companies network. We have firewalls on the inbound connections that block most inbound connections. The individual servers also have software firewalls that allow them to only accept packets intended for them.

    I hope I never see a rejected packet on those servers, because it means the first line of defense has failed in some manner. Does that mean I should uninstall my second line? No, no more than I would remove the two phase VPN login followed by the user account login. If someone’s computer is stolen, reaching my VPN is easy (assuming the user cached the credentials) but they will hopefully be stopped by the second layer.

    Defense in depth never achieves perfection, but it can work around bugs, lost equipment and new exploits more often than a single line can.

  7. Daemian Lucifer says:

    But how can it not work?Its moar!Moar is betterer!!

    1. Alexander The 1st says:

      As I mentioned above, when all else fails, use Fire.

      If Fire isn’t working, you’re not using enough of it.

  8. anna says:

    Other people have already covered this ground well, but by way of example…

    Let’s say Anna wants to attack Victor’s box. She knows he has some juicy credit card numbers on it. Anna starts to poke around at his network. DNS reports that his website is running on 1.2.3.4. Using something like the following:

    nmap -O -sS 1.2.3.4

    she determines that he’s only got ports 80 and 443 open, and that the box actually sitting one hop in front of 1.2.3.4 is some sort of big, dedicated hardware firewall (I’m not sure if nmap’s -O flag can detect anything that useful, actually, but let’s say this is super-nmap-from-the-future). Anna also knows that the target webserver is running Linux, thanks to netcraft. So, she figures there’s a good chance ssh is running on the box as well.

    Anna asks her friends who play with a lot of network hardware, and finds out that there’s a known vulnerability on some versions of the Big Dedicated Hardware Firewall that tricks the firewall into thinking a packet should be accepted even though it has nominally closed the port in question. This vulnerability was fixed in a very recent firmware update, so there’s a good chance Victor has not updated to it yet. She writes up an exploit for this vulnerability, and tries it out.

    Success! Anna can now punch a hole right through this firewall. She opens port 22 to 1.2.3.4. She knows about a vulnerability in openssh that hasn’t been noticed by anyone on the openssh dev team yet, and she’s been keeping it to herself for just such an occasion. She pulls up the exploit tool she wrote for it, and hits 1.2.3.4

    Now, right here is where a second firewall is useful. If the *machine itself* is running a firewall, it could have a couple rules like so (written in iptables syntax):

    -A INPUT -p tcp –src 1.2.3.0/24 –dstport 22 -j ACCEPT
    -A INPUT -p tcp –dstport 22 -j REJECT

    This would only allow ssh traffic from 1.2.3.0/24 (the only machines that *should* be able to reach port 22 anyway, with that Big Dedicated Hardware Firewall on the edge of the network). This little extra precaution could totally save the day here, though. Anna won’t be able to use her ssh exploit, and will be back at square one.

    Security needs to be approached holistically. Keep your packages and firmware updated, of course. Use firewalls, and tools like SELinux that provide comprehensive Mandatory Access Control on the box itself – SELinux is designed, in part, to prevent a vulnerability in one service from compromising the entire machine.

    And yes, sometimes multiple firewalls can be a good idea.

    1. Shamus says:

      VERY educational. Thanks. :)

  9. Ancorehraq sis says:

    Will Sony add some moar firewalls when they get hacked again in a couple of weeks? :)

  10. Jeremy says:

    You typically have a firewall on your ingress that does basic filtering for the whole internal network, and maybe with host or subnet specific traffic policies (and/or those hosts/subnets may have their own firewalls doing filtering).

    What Sony may mean is that they’ve done finer-grain firewall partitioning of their internal networks so that a compromise on one doesn’t leak into others.

    Unless they’ve completely rebuilt their entire infrastructure – which is very unlikely – they’ve probably still got compromised hosts floating around in there, and they’re trying to contain them rather than fix them.

    And if they’ve got compromised hosts, its likely they’ve been compromised for a *long* time, quietly sneaking data out and moving around the network. And its only when one of the attackers pulled the trigger and started selling the data they’d been stealing that Sony realized how big a problem they have on their hands. (And they probably chose the timing to use the Sony v. Anonymous as distraction cover, because this is much larger than something cooked up recently.)

    They’re pretty much screwed. I think they’re hoping that their ongoing liabilities will be less than the cost of rebuilding everything, because they basically can’t afford to do that.

    Needless to say, never, *EVER* enter anything you want to keep off the black info market into a Sony server. Ever.

  11. Lisa says:

    I sometimes enjoy the confusing things that can come out of a company when the information has been through multiple layers.

    My interwebs banking was broken one day, so I rank the Help Desk. They confirmed that, yes, it was broken, and the reason was that there was a “broken link slowly taking over the internet”. I can only imagine the Chinese Whispers which took whatever the tech person said to produce that gem.

    Luckily they fixed it before the whole Internet was corrupted.

  12. ehlijen says:

    But they used multiple firewalls on the new BSG, complete with dramatic ‘firewall one breached!…firewall two breached!’

    Surely TV doesn’t lie to us?

    Also, I like how you spelled ‘Dark’ with an umlaut to make it sound more like ‘derp’ :D

    1. Mari says:

      Hack the planet! Hack the planet!

      Because technology and Hollywood should never, ever meet.

  13. Ingvar M says:

    In principle, I agree. Adding firewalls doesn’t necessarily make anything any securer.

    Unless you’re segmenting your server farm into multiple trust zones. Hopefully that is what they mean. Alas, I suspect they mean “if we make noise, maybe all our problems go away. Make more noise, flapping meat.”

  14. Will says:

    I think this is more of “We don’t want to sit here and explain how we’re updating our security systems so lets just say more firewalls because most people don’t understand what those are but know it has something to do with security” type of thing.

  15. K says:

    The title of this page is off. It should read “The Därk Fortress”. Have some ¨ to copy paste on your as so they become äs.

  16. Mari says:

    I just want to add, as a Texan, that I cringe when I see the words “barb wire.” “Barb Wire” was a name brand from Glidden Steel. Barbed wire is the group of poke-y wires that make you bleed. It’s completely pedantic and nobody outside of ranching historians really give a rat’s rear end but I have to say it.

  17. Zamalan says:

    I suppose it could still mean they are behind each other. If they are exact copies then obviously that won’t do anything. But if they are different and a hacker circumvented 1 firewall via an exploit or that firewall he’s then stuck on the second where that exploit won’t work. I suppose the administrators would then have a bit more time to notice and/or react to the intrusion.

    Like a fort with a wall is just as secure as one with 2 walls, the invaders need only 1 ladder to cross both. But if there’s a wall and then a moat they’ll need a ladder and a boat to get in. The defenders will probably see the invading army pouring over the wall, with enough time to react before they either fetch a boat or fashion a makeshift raft from their ladders…

    Or was that a bit too simplistic of me? I’m only a simple programmer, networks are not my cup of tea.

  18. WWWebb says:

    Suddenly this comic is topical again!! Oh if only the rest of Sony had learned the lessons of the Playstation Network. Those faces should have been much scarier. Oh and the red cables may mean “come on in” in North Korea.

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun. Your email address will not be published. Required fields are marked*

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>

Leave a Reply to Mari Cancel reply

Your email address will not be published.