Disaster Favor

• By Michael Fickes
• 03/01/13

Some businesses can afford to use the cloud as an IT disaster backup solution. Colleges and universities, however, must think about education first. Backing up IT in case of disaster comes second.

Many smaller liberal arts institutions don’t even have secondary server rooms as backups. If an earthquake or flood destroys the single primary server room on campus, an institution won’t be able to issue paychecks or deposit tuition payments. New students won’t be able to register. All of the data stored in the learning management system will be inaccessible to students as well as professors. School may well be over for the year.

Because the results can be so dire, more and more schools are building secondary server rooms for disaster backup.

The University of Puget Sound in Tacoma, WA, and Pomona College in Claremont, CA, both have primary and secondary server rooms. But their technology chiefs don’t believe that’s enough.

William E. Morse, Jr., J.D., chief technology officer and associate vice president for technology services at Puget Sound, and Kenneth Pflueger, chief information officer at Pomona, both worry that an earthquake might destroy their primary and secondary computer rooms.

Morse and Pflueger have created an innovative alliance to avoid disaster. Both have agreed to provide space for the other school’s third set of backup servers.

Soon, if an earthquake or other disaster knocks out Pomona’s primary and secondary server rooms, Pflueger’s IT team will connect to the servers backing up services and data in the University of Puget Sound’s primary server room. Likewise, Morse’s IT team will soon be able to connect to servers backing up its services and data in Pomona’s primary server room.

Both men had the idea about six years ago and have been looking for a partner school. They realized that the partners would need to be located far enough away from each other that the same disaster would not affect both.

One day, in the course of his search for a partner, Pflueger emailed Puget Sound’s IT group about his idea. Morse’s positive response led to a working lunch at an EDUCAUSE technology conference, and the alliance was underway.

Pflueger and Morse laid out a path for the project. First, they would develop an agreement that officials from both schools would sign. The agreement would articulate what each school would provide for the other in a way that would limit the liability for both. “We wanted to make sure that the good deeds we were going to do for each other would go unpunished,” quips Morse.

The Agreement

The agreement itself and the parties to the agreement were both key to the success of the project.

“In my research, I came across two schools that had carried out a similar project,” recalls Pflueger. “That alliance just faded away when one of the two IT directors took another job. We decided that it would be important to get an institutional commitment to the plan from the highest level. It would not just be an agreement between William and me.”

The agreement would also encompass the spirit of the planned alliance. “We agreed that this would be a favor that we would do for each other,” says Morse. “No money would change hands.”

The institutional commitment came by way of a memorandum of understanding (MOU) outlining an agreement that each school would set aside space for a full rack in each school’s data center for use by the other school. The presidents of each school signed the MOU.

Next, Morse and Pflueger each retained attorneys to negotiate the terms of the agreement. “It’s important to understand all the details that would be involved in an arrangement like this,” Morse says. “It took about two months to complete the agreement.”

The agreement itself runs to 7,600 words. Essentially, it says that each school will buy, install, and maintain its own server rack, servers, and disk drives for data storage in the space provided in the other school’s data center.

“As part of the agreement, we had to verify that Pomona’s server room met our requirements,” Morse says. “Pomona had to verify that our server room met its requirements.”

Questions about server rooms involved backup power, air-conditioning and temperature controls, fire protection, access control, door alarms, cameras, and other security precautions.

In addition to space, the host school would provide power and network connectivity.

The agreement also assigns responsibilities to the host school and the guest school for protecting confidential information.

The Technology Plan

“Virtualization technology that has come about within the last three years or so is what enables us to do this,” says Pflueger. “We used to have about 150 individual servers in our data center. Today, thanks to virtualization, those services are running on five megaservers.”

Virtualization means creating a number of virtual servers, acting as individual servers, on a single server. In the case of Pomona, each of the five megaservers are running approximately 30 virtual servers that provide various services across campus, such as learning management, financial management, and email.

Morse has decided to use recently retired server hardware for the installation at Pomona. “Using older equipment makes the equipment costs negligible,” he says. “The older servers will work because the site will only be used in production in case of a disaster. Most of the time, the equipment will only be updating and running — with no one using the system.”

Core Services

Under the agreement, each school will decide on the core services it wants to back up, a task that is still in the works.

“We’re going to put our ERP (enterprise resource planning) system on our Puget Sound rack,” Pflueger says. “That includes our student information system, financials, financial aid, and other systems. We’ll also have our learning management system. With these systems we’ll be able to pay our bills and conduct classes.”

What about email? Pflueger says he hasn’t decided whether or not to back up email in Puget Sound. He has outsourced student email to a cloud service and is considering doing the same with faculty and staff email.

Pflueger’s plan is to backup data every night in Puget Sound, which means switching over would entail the loss of one day of data. “We could backup during the day, but I think it is too bandwidth-intensive, meaning expensive.”

In Case of Emergency

The agreement names one official from each school who will declare an emergency. At that time, the appropriate host school will make the emergency system available to the guest school.

Both Pflueger and Morse are working out the procedures necessary to bringing the remote sites online. When complete, the procedures will be written up and included in the disaster recovery files. When that happens, the procedures will be incorporated automatically into each school’s disaster response and recovery drills.

With any luck, that’s all the live use these two disaster recovery centers will receive.