White Paper: How Convergence Has Made America More Vulnerable and Less Secure

by Steve Keller

For years, security managers of my age group sought the holy grail. The holy grail was the totally integrated system where security, fire, access control, CCTV and building management shared one server and everything could be viewed on one terminal.  We got over that in the 1980’s when it soon became apparent that while this saved money, it substantially reduced security. Some dinosaurs still seek that holy grail out of ignorance to the risk they are taking and many younger managers place too much faith in computer technology. How unwise and ill advised this is. Why do I take note of this? One of the first things the salesman will tell you when he explains either the access control system or the CCTV system he is selling you is that the two systems can be mated together and when an alarm occurs on the access control system, the software will trigger a camera action to occur and the camera will swivel, zoom, tilt, pan and lo and behold the image will appear on the master monitor, saving valuable seconds so the operator can more effectively deal with the alarm. So, when I tell clients that I will design their access control and CCTV systems so that they are not integrated in any way, they look at me like I have two heads. “My God, Keller, where have you been since 1980? Don’t you know that I have been waiting since 1980 to achieve this holy grail?”

Why would I make such a recommendation? Let me count the ways:

  1. 1. When converging the various security systems onto one terminal was the goal of us all, the world was a different place. No one had really heard of computer hacking, viruses and denial of service attacks. There was no spy software that could report to the criminal gang when your alarms were on or off. Software that captures keystrokes and steals passwords and usernames had not been invented. And no one had yet proposed converging even more systems, each serviced by a different technician from a different company, on to one massive network or, worse, running all of these systems on one big server located in the IT computer room and out of our physical control. You didn’t have to worry about each of these technicians having his own ”back door” into the network and because there was no such thing as an internet as we know it today, the tech who left himself “back door” access to the system couldn’t have done it from any computer in the world as he can today.

  2. 2.When we first wanted an alarm to trigger a camera image on a master monitor, computers were new and security officers likely only ever used one at work. Today they all own several and their ability to respond more quickly to something happening on their computer screen is far easier for them to do quickly and efficiently than in the past when they lacked the familiarity with computers that we all have today.

  3. 3. If we wanted the picture to call up to a master monitor then, we had to do it via the software in the access control system because it wasn’t going to happen any other way. Today, the powerful full featured CCTV systems have their own built in ability to call themselves up to a master monitor even if they are not connected to the alarm system.

Hear this! Believe this! You cannot have it both ways. You must decide now which path to take.   If you insist upon integrating your alarm and your CCTV systems then the CCTV system can’t be on the institution-wide network. Non-security employees will lose the benefits of this system and you will have more difficulty getting the system that you would like.  Why can’t you still place the CCTV system on the building wide network? Because the building wide network inevitably will touch the internet. It will inevitably get a virus. It can be hacked, attacked and brought down. As many as fifty different companies have systems on that network from the intercoms to the phone system to the building management system to the lighting control system to the ticketing system to the audio tour system, etc. Each one has a technician whom you don’t know and who can probably dial in from anywhere and get access to the network. And if you’ve been paying attention to my security blog, most of these guys probably use a default password I could find out from the company website by downloading the user manual. Geez! Under these conditions the fourteen year old next door could hack your security system. Bet on it. Remember. It’s a different world today. Your alarms will trigger a camera and it will call up to a master monitor but at what cost? Regardless of what the salesman tells you, if you mate the two systems, a risk to one becomes a risk to the other. At what cost do you finally achieve a holy grail that has lost its purpose?

And hear this also. If you buy a good CCTV system you will not lose a thing. If you want a camera to call up when a door opens, simply have this done through the CCTV system itself! Your mission critical alarm system remains isolated on its own network and life is good. And if you buy a non-Windows based CCTV system, the odds are that you won’t get a virus from your CCTV system being on the building wide network.

There are three reasons we feel pressure to “converge”. The first is a myth that was hopefully exploded above. The second is the politics of power discussed in the previous chapter.  And the last is to save money. If everything can be on one network, we save the cost of building a dedicated network for Security. And we can take advantage of the economy of having everything serviced under one service contract. And the IT people find converging everything onto one network to be efficient and easier for them to deal with.

If you take nothing away from this paper, I want you to burn this into your minds and quote it often to your superiors. “Economy and efficiency are not the sole objectives of museum administration”.

When a museum curator decides to display an artifact, she first selects an exhibit case. Most of the time she commissions the design and construction of a new case and ignores the dozens that are in storage from prior exhibitions. When you ask why she did this, there is always some reason such as the “scale” of the existing cases are not “just right” for the proper display and enjoyment of the piece. It is well worth it to the curator to build a new case that is “just right”.  When a museum educator establishes a program there are often costs that could realistically be reduced but education is deemed to be a primary objective of the museum and therefore worthy of the added cost to achieve perfection.  But when you ask for new security technology you are often told that the museum can’t afford the Rolls Royce and that you need to learn to settle for the Yugo. This attitude prevails even though nearly every museum’s mission statement begins with a statement like “The primary goals of the museum are to preserve, protect, and educate . . .” Why is it that they get the part about “to educate” but miss the part about “protect”?   The desire to converge every system that can possibly be networked from ticket sales to intercoms to telephones to word processing to email to fire to HVAC to building controls to access control to alarm monitoring to CCTV to object protection to guard watch patrol to the copier machine’s paper management is based on the desire to reduce costs. This is the reason.  This reason is bolstered by IT managers who want things to be easier for themselves or to build a power base and by security managers who are dinosaurs and who still seek the holy grail. But the reason is “economy”.  Period. End of story.  If it was cheaper and the IT manager got more power and prestige by having a network for every system, I guarantee you that we wouldn’t be having this discussion.

By having everything “standards-compliant” and under one hardware service agreement and managed from one room by one staff of experts, there are indeed savings.  But these other systems are rarely mission critical and the failure of a ticketing system or phone system, while serious, do not carry the ramifications of a neutralized alarm system. While some systems are mission critical such as the fire system or the air conditioning system in an art museum, no one has ever, in the history of the world, been reported to have hacked into a fire detection system or an air conditioning system for the purpose of defeating the system to commit a crime. Only the security-related systems are both mission critical and subject to criminal attempts to neutralize them so they can carry out a crime so massive that it destroys the mere purpose for the museum to exist by permanently denying it of its collections.  Read that sentence again.

Economy and efficiency are not the sole objective of security management, either, and sometimes we need to spend a bit more to make security perfect because perfection is critical when the criticality of a loss is so high.  This does not mean we must always buy the Rolls Royce but it does mean that we may have to pass up opportunities that expose us to known and unknown risks or are clearly unwise. Until the computer industry can prove that they have resolved the hundreds of extremely obvious and exploitable flaws in the technology, we must take steps to isolate our systems from all others even if it costs a bit more. What obvious and exploitable flaws, you say?  Last week Microsoft issued five patches to their supposedly high-security Vista program. The week before, they issued nine to patch security flaws. These are not the first nor will they be the last patches they will issue. What flaws, indeed.  T. J. Max just lost 94 MILLION credit card names and numbers. Ninety-four MILLION. The company will likely pay fines and costs associated with this loss that exceed the typical museum budget for a ten year period!

If you intend to use leading edge technology then you darn well better understand the risks. You wouldn’t pick up an armed weapon if you didn’t know how to do so safely but you will run to any new technology that comes along because the salesman from Microsoft or your alarm company tells you it is safe and secure to do so. No one loves technology more than I do. I live it daily. I play in it nightly. I soak up information about it like a sponge. My free time is spent exploring it. And as a result I respect it. Too many people, especially young people, put a blind faith in computers without knowing how little they truly know about them, how they work, and how they can be exploited.

Every night I sign on to my computer with a fake name, go to a secret internet chat area exclusively for computer hackers, and mingle with the young, brilliant people whose lives revolve around hacking into other people’s computers. I rarely speak and I have adopted the persona of a young man who is fascinated by hacking and is learning from these old masters. Think of it as an undercover assignment that we security consultants engage in for amusement. I have become all-to-familiar with how incredibly easy it is to hack computers and how easy it is to exploit human weaknesses and benefit from their bad habits and lazy attitude about passwords and security. I know people who can hack your computer if it touches the internet in any way. Your museum’s network touches the internet, for sure.  I didn’t say that these guy might be able to hack your network, I guarantee it, and you can take that to the bank. If you are a regular reader of my security blog you know that I report on weekly successful hacks of places far more secure than your network, like the hacks on the CIA, the Pentagon and the servers at Microsoft. (Click here to see an actual CIA web page the morning it was hacked.--Central (Un)Intelligence Agency.pdf).  Don’t tell me about VPN networks or how smart your IT manager is. I know a fourteen year old who can teach him a lesson. My job is to never give him the chance, rather than give him relatively easy access, then rely on people or technology to protect us. That is exactly what you are doing when you refuse to allow your security alarm and access control systems to ride on the building wide network.

In August of 2007, I reported on the details on my blog of the cyberwar where Russian criminals launched a major denial of service attack against the country of Estonia in retaliation for them removing a Russian war memorial. Estonia, believe it or not, is one of the most wired countries in the world, having built a national internet from scratch with the help of Nokia. They are so advanced that they pay for their parking meter by waving their cell phone in front of it and pressing a button. The parking fee is read from the meter, the cell phone sends the transaction to their bank and their bank transfers funds to the city.  In the cyberwar, a small group of criminals called “bot-wranglers” spread a Trojan horse to computers just like yours around the world and planted a “bot” that they could control on demand.  At an exact moment in time they unleashed the largest bot attack in history, literally shutting down the government, all TV and radio stations, all newspapers, the communication systems like phones and radios, and all international communications, literally isolating the country from the world for several days. Banks and ATMs closed. Businesses came to a halt. The U.S. put forces on alert because no one could find out what was happening and it was suspected that Russia had invaded them.

What this means to us is that if even if they can’t hack your computer network and manipulate your security system computers, they can almost certainly launch a denial of service attack against your museum causing it to overload the network and forcing a shut down of your network.  We now know, for a fact, that this can happen and that a small number of criminals can make it happen in a big enough way to close down an entire country. The cyberwar on Estonia took this from a proof of concept to an actual event.  So what do you suppose happens if your security systems are using this converged network, too?  In a converged system, your phone system fails, your computers fail, your cameras fail.  Everything on that network fails, giving ownership of your building to the criminals who attacked you. They have proven that they can modify their attack by the hour, overcoming every countermeasure your IT department can implement. But they don’t have to totally defeat your system. All they have to do is make it “burp” for the few seconds they need to force open an alarmed door whose alarm can no longer annunciate in your control room. The only solution is to isolate your security system on its own dedicated network that doesn’t touch the internet or any other network.  Read that sentence again.

The next thing I want you to take home with you from this paper is the fact that mission critical security systems must remain on their own dedicated network and this network should have a minimum number of very closely controlled portals into it and except for a few moments when you download virus definitions and software updates, it must not touch the internet or any other network that touches the internet. Terminals that give access to your network should be limited and secured both physically and using passwords. A VPN provides good security but not perfect security so your dedicated network must be physically separate and physically dedicated and it should be a VPN just in case someone somehow adds themselves to your closed network. Every possible way to defeat a network has not yet been discovered so don’t let anyone tell you that your network can’t be defeated. And your dedicated network cannot be connected to the outside world in any way. This means no internet access on that network, no email, nothing but the security systems it serves.  This also means that you can’t dial in from home to check the status of your system or acknowledge alarms.  It means that your service provider can’t dial in to do diagnostics. I’m not saying that you can’t have email in your control room, just provide a separate PC for that. And I’m not saying that you can’t take advantage of the dial in feature for diagnostics offered by your service provider. Just make the physical connection, allow the brief diagnostics to occur, then physically disconnect from the internet to reduce the chance of hacking or virus attack. I am saying that you cannot dial in from home over the internet to check on the status of an alarm or decide whether to haul your rear out of bed and drive in to the museum to check it at 4 AM or not.  This type of remote decision making has always been banned by the national standards and your fine arts insurer and was the cause of the third largest art theft in U.S. history. I’m aware that this poses a hardship for you. It’s just not worth the risk!

To avoid human error and exploitation, the terminals in the security control room should not be equipped with any external drives. No CD/DVD drive, no USB port that is not used, and no floppy drive. Updates to the system should be made from the secure server under the tight control of the security manager to prevent a guard on the night shift from accidentally introducing a Zero day or virgin virus that can’t be detected by your virus software, thus shutting down your system. A zero day virus is one that is being introduced to the world that day, has not yet been seen in the wild, and no one has made a virus definition to neutralize it. It remains a zero day virus until it is identified and a definition is deployed to stop it. If this sounds far fetched, then you should find a new career because this is the stuff of crimes in this new high tech world we live in. Just a few years ago no one could possibly believe that anyone could actually connect a computer to a phone line and intercept a phone call from your alarm system to the central station and neutralize it. They told me I was nuts when I warned them against this. If I sent a CD professionally printed with a photo and the words “Girls Gone Wild:  Coed Edition” to your night security guard who works in the security control room on your night shift, at the museum so he gets it just as he is reporting for work, how much do you want to bet he’d put it in the CD drive to check it out? If it has a zero day virus on it, your building will be mine. Want the email address of a guy who will write you a virus? I know several.

What exactly is this dedicated network and how costly is it to build?  I’m not talking about duplicating your entire building-wide network. I’m only talking about connecting the various alarm control panels for your access control system with the security control room and any other monitoring locations. In most buildings this is an extremely small project. What systems should be on this dedicated, isolated security network?  Your alarm and access control system for one. Any object protection system like ArtGuard or ISIS should be on a separate network if possible so that if the primary security network goes down or is taken down for service, the object protection system network remains up and running, but placing them on separate networks is often unrealistic.  If you want to prevent anyone outside of Security from seeing your CCTV system images, then put the CCTV system on the network with the access control system. If you have a separate network for the object protection system, then put the CCTV system on it.  If you have plenty of money and achieving the highest level of security is your goal, the build a separate network just for the CCTV system. It’s all about achieving redundancy after you have assured that the primary network is secured from every possible threat by isolating it.

Is the CCTV system at great risk of infection by a virus if it operates on the building wide network?  That depends. The vast majority of viruses are written for Windows-based systems. There are 60,000 Windows viruses, 40 Macintosh viruses (only two are dangerous), and 40 Linux viruses. Scott Granneman, writer for “SecurityFocus”, a computer security publication said, “To mess up a Linux box, you need to work at it; to mess up your Windows box, you just need to work on it”.  Quality systems like Acuity-vct give you the option of running either a Windows or a Linux operating system on your server to control your CCTV system whereas most DVR systems run a copy of Windows on each DVR. You can decide which operating system is best for you, create a VPN to further isolate the system, then hop on the building-wide network knowing that you have greatly reduced your chances of catching a cold every time Mary downloads some financial deal from this nice Nigerian man who wants to make her rich. Since you are never going to allow a user of your CCTV system to operate at root level in your server it is unlikely that you could be affected by any of the 40 Linux viruses, anyway, since they need to be at root to do real damage. So for added security, select a CCTV system that uses Linux or Unix or Red Hat or some derivative.

Another concern that I have with the concept of convergence and moving the servers outside the control of the security department involves the internal security of your museum.  A typical museum IT department has probably fifteen to thirty employees when there is a converged network. This is fifteen to thirty highly intelligent employees who have access to your security system. Last year I predicted that the next Billion dollar theft that occurs that is entirely unsolvable will be carried out by an IT department employee who will commit the perfect crime. They are the only criminals I truly fear because they are the only ones capable of doing the deed and totally concealing all of the evidence.  For that reason, I strongly oppose convergence, I strongly oppose integrating the alarm and CCTV systems, and I advocate a totally closed dedicated security network that remains under the complete control of the security manager. The dedicated network that carries your alarm and access control system can be serviced by fully qualified IT professionals employed by Security who have been fully vetted just like your locksmith or Security Operations Manager should have been. The less mission critical CCTV system can be adequately protected while running on the building-wide network where it can be serviced by conventional means.

Another concern is that when a alarm system resides on a shared network, vendors and service technicians for all systems residing on that network are occasionally in need of access to that network regardless of how self-sufficient the IT department is. In a converged network, the technicians from the intercom vendor, the building management system and others who share the network with security have access to both the network and the IT closets where the remote racks reside. We’ve shown in previous posts to our Blog how vulnerable this can be since most technicians maintain a secret password (called a back door) to the systems they service just in case they need to dial in and do service. Needless to say, this isn’t a good idea for any network let alone one that carries your security alarms. Just to show you how bad this really is, nearly all technicians use the same back door password for every one of their regular clients. This way they don’t have to remember a different one for each job site. If you are the administrator of your access control system, go to the administrator level and check the list of passwords. With most systems, you can see the actual list and who has what password so you can program a former employee out of the system. Look for the service tech’s password.  Now ask him what other companies he services. He’ll tell you.  Odds are very good that this same password will get you in to every system he services! You now have the password for the bank down the block or the hospital he services or the other museum!

We are almost always required to share rack space in computer closets with others, and provisions are rarely made for expansion. When someone representing another system has access to our wires and cables in a rack, this increases the chance of accidental disconnection of something. This then necessitates greater security at these closets that increase the cost of the security system. If the IT manager who controls the space in the IT room doesn’t do his job well with regard to space management in the racks, you may lose space in your rack designated for spare capacity that you will need in the future.  The last thing that you want is the technician for the ticket printing system messing in your rack.  In one client museum, 26 card readers are dedicated to protecting remote IT closets while only four are dedicated to collection storage to secure these spaces.  Racks in a security control room do not need locking glass doors or dedicated readers, but they do if they are located in a shared IT closet. I’d rather put my resources toward securing the collection.

When a security system is entirely under the control of one or two individuals in a Security department it is possible to swear in court that no one has tampered with or changed the security system. It is easy to swear that no one could possibly have bypassed the card reader, deleted a camera image sequence or otherwise bypassed the system to commit a crime in, say, a storage room, but when 20 or more employees and up to fifty contractors have virtual if not physical access to the system it is more difficult to show internal integrity to the courts. This is not as great a problem with CCTV systems as it can be with access control systems because tampering with digital video recordings can be detected.

Finally, what if your museum is targeted by a denial of service attack or fails to detect a virus before it reaches your server and this brings down your CCTV system as well? Your alarm and access control system which ride on a dedicated security network will continue to function and before too long a decision will be made to close the building since other major systems like heating and air conditioning will also probably be effected. Your essential systems remain in operation.

The concept of convergence is not good for security managers or the profession as a whole, it leads to reduced security and increased vulnerability, it assures that the security system, that traditionally has run without downtime, will now be brought down for nightly maintenance, usually in the middle of the night when you need it most, and it leads to false economy as anyone who has incurred a major heist in their museum can tell you because of the massive cost it has to the institution in many ways. Since the CCTV system is not truly mission critical but there are advantages to giving access to the camera’s images to employees outside the Security department, you can safely place it on the building wide network and actually regain a level of redundancy you didn’t have before.  If you follow my advice in how you approach this new technology, you can take advantage of the very best it has to offer without incurring the great risk it brings. If you don’t follow this advice, you substantially increase the list of very serious risks that can affect your institution’s security. And some of these risks are so serious that they are likely to be the new frontier of crime that we all face.