Verizon tames ops center chaos with IM

By Steven Vaughan-Nichols
Posted on ZDNet News: Apr 18, 2002 12:00:00 AM

Today, the pace at the national ISP is still fast, but it's less frantic, thanks to an application many once dismissed as just for consumers--instant messaging (IM). Verizon is using the NetLert Communications' NetLert server and clients so that in-house operations staff and Verizon's engineering staff can work together on network problems in real time. Messages are recorded and logged in the server during IM transactions. The logged messages can be used for solving future problems, while the log data can be used for management reporting.

Group Manager of Operations Kevin Lee says that "with our 24-by-7 NOC, we were always trying to reach multiple people at once, and we had a hard time managing the work load. We were also getting a lot of phone calls from within the company and we were always teleconferencing with engineering." But telephone conferences tended to get confusing and e-mail wasn't real time enough. "We really wanted to open up the line of communications in the staff so that everyone was on the same page with respect to who was doing what with whom to which project."

Keith Longabaugh, the NOC duty manager for operations support, agrees. "We wanted a way to talk with each other in real time," he says, "while at the same time being able to have a techbridge (Verizon internal teleconference) or work on a project."

Of course, any IM service can do that, but Verizon wanted more. Specifically, it wanted a system it could host itself that provided real-time IM with no latency. "We wanted to host messages for the whole staff and get those messages to everyone as fast as possible," Lee says. "AOL Instant Messenger (AIM) did only the last part of the job." He says AIM lacked other features, such as tight security and in-house message storage.

Though outsourcing IM to one of the large providers would be free, Verizon Online wanted to manage and maintain its own hosted system. That way, the company would be immune to outsourced host outages and most IM worm attacks and spam assaults.

Verizon wanted a package that gave the NOC centralized management--a universal team message folder, with the ability to use any user name. (On the public hosted systems like Yahoo Messenger, easy-to-recognize names have long been taken.) Verizon also wanted an alarm function for critical messages, and an in-house server system. Last, but far from least, it wanted security.The in-house IT staff started evaluating IM server/client combinations in April of 2001. It looked at Microsoft Messenger, AIM, Yahoo Messenger, NetLert and several others. Verizon did not, however, look at open source IM systems such as Jabber. "We wanted something that was proprietary that we could control and own," says Lee.

Message content security and control over message logs were the most important issues. While the remotely hosted services had many of the features Verizon was looking for, they had neither the security nor the centralized management the NOC needed. NetLert, however, was built from the ground up to be hosted internally and uses SSL to secure all IM sessions.

After four months of test driving the products, Verizon Online's IT staff picked NetLert, and by late August 2001 had the NetLert system up and running for the entire NOC.

Longabaugh estimates, "On the average, we're saving 30 minutes to an hour per day per person." At that rate, even at NetLert's maximum cost of $40 a seat for life, Verizon Online was saving money within the deployment's first two days.

It also helped that NetLert, with its Java-based server and client, requires minimal resources. Longabaugh describes the NetLert servers, each supporting up to 100 concurrent NetLert users, as utility boxes--generic 500MHz Pentium III servers with 256MB of RAM running Windows NT 4 or Windows 2000 Server. The servers run not just NetLert but also Internet Information Server and SQL Server and work as file servers.

For the clients, "we run mostly Windows 2000 Professional and some XP Professional on everything from Pentiums to Pentium 4s. Including our remote users, we're running just about every Microsoft system known to man." Remote users maintain Verizon's security by connecting through Check Point's VPN-1.

Verizon today isn't using all of NetLert's capacity. The company, for example, first elected to use NetLert's own user directory and is only now hooking NetLert's user directory into the Windows NT domain model. NetLert can also be used with LDAP or Novell Directory Services. Additionally, Java-based NetLert can run on every client OS from OS/2 to Linux to Macintosh.

In the six months since NetLert went to work for Verizon, the company has had no complaints about its IM system. "No problems with it whatsoever," says Longabaugh. "It's turned our NOC from chaos to organized chaos."

Has your organization benefitted from instant messenging? Share your thoughts in TalkBack, or e-mail us.Verizon tames op center chaos with IM