This page Copyright (c) C F Systems, 2004-2006

Design for Fast Computer System Recovery

This page was written shortly after we deployed several of the described systems in December, 2004 as separate high-speed Internet units. In early 2006 we added this paragraph and a few other updating notes. These systems have performed very well. We have restored these systems to an earlier known good state over a dozen times with minimal disruption. In fact, most of these restorations have been simply because something in the system stopped working properly or because we really didn't like some software we had just installed, so not necessarily due to the Internet connection. Rather than searching the web to find a solution to these inevitable Windows system problems - and perhaps not succeeding - it is easier to just restore the system to when it was working properly. In fact, we have deployed this same system strategy on our main working computers and it is a real joy to be able to fully recover a system after installing a new application that proves to be ill-behaved..

As most of us already know, when you attach a computer to the Internet, particularly the high-speed Internet, it becomes an open invitation to disaster. To prevent disaster, you can do as most experts recommend:

1) Install firewalls.

2) Install anti-virus software, anti-spyware software.

3) Install all the latest patches for the defective operating system, applications, and drivers.

4) Define a list of trusted sites and only allow those sites to use ActiveX, Java, and scripts.

But check around. You'll find that even all of the above does not work. Systems maintained by the experts don't have as many problems as the average system but most still have the same problems; problems that are often very serious and time consuming. Taking the same points as above, this is because:

1) At the very least, firewalls have to let through anything that appears to be a communication that has been requested by the user's computer and many requests are automatic or caused by automatic human responses. Nowadays many applications written in the marvelous Microsoft dot-net system require that the firewall allow them to access the Internet in order to work, even though they do not actually need the Internet, as evidenced by the fact that the same applications run just fine on computers that are not attached into the Internet.

2) Anti-virus software cannot recognize and protect against the latest virus until it has already struck and become widespread enough to be properly reported. Anti-spyware is always at least one or two steps behind the spyware producers. (Note: Since we first wrote this, this phenomenon has become more widely recognized as the "day zero" problem.)

3) If you use automatic updating of all software, it puts your system in a state of perpetually beta-testing the new, untested software in the patches. You never know when that might disable a critical application or even the entire system for an indeterminate amount of time. If you do the sensible thing and wait awhile before installing patches it opens your system to flaws all hackers now know about, thanks to the update announcement. Also, there is an enormous problem just trying to keep track of what needs to be updated.

4) We all know how well the "trusted site system" works. A design in the same fine tradition that brought you the Registry, where every possible problem is piled in one large heap so nearly any failure can adversely affect the entire system. Sites which shouldn't be trusted get into the trusted sites list because you can do hardly anything without enabling scripts and so many sites also "require" ActiveX. Even just allowing scripts opens your computer to potentially serious trouble, and nearly everything on the net requires scripts.

So, as most of us already know, when you attach a computer to the Internet, particularly the high-speed Internet, it becomes an open invitation to disaster. Attaching a computer which we also expect to use for other serious work is hazardous. The more cautious among us hamstring the Internet connection enough that it is tedious and sometimes impossible to work on the Internet and we still worry about it. The less cautious among us plunge ahead and have to deal more frequently with disruptions and lost time.

Isolate and Design for Fast Computer System Recovery

We at C F Systems have worked out another approach. Computer systems have become cheap; for substantially less than $500 a very impressive computer can be had. Depending upon your circumstances, that can easily be less than the cost, time and otherwise, of one serious internet incident. We decided to add a second computer to each of our work stations. This second computer is used for all Internet contact and is set up so that in the event of a disaster, it can easily be completely restored to a known good state without even requiring that any data be lost except in the most extreme of cases. This computer is connected to our regular workstation computer by a fast link for transferring files back and forth, but not through a network. Thus there is no way that files can be placed on our workstation or programs installed on our workstation without our specifically knowing about it. If there is any reason to suspect that a file we would like to transfer to the workstation might be a problem, it can be opened or run and thoroughly checked on the Internet computer first, so if there is a problem any damage can be easily fixed and without disturbing the workstation itself.

It will be obvious from the following that the same quick recovery system that we use for our separate Internet computer could be slightly modified and used for the Workstation itself. More frequent backups and systems saves and archives would be required and those would likely be of substantially larger size. This is a matter of judgment for your particular work environment. Since our workstations are not connected to the Internet, they are much more stable since there is no need to constantly update their software with the latest, possibly buggy patches. If there is a disruption requiring a system restore the workstation remains available during the disruption. On the Internet computer, the system restore may take from a few minutes up to as much as a half hour depending upon the time taken to selectively restore user data.

This document describes our new internet computer systems. It requires a certain level of computer systems knowledge to set up such a system and the description will assume that level of knowledge. If the terms are not familiar and you do not fully understand what we are doing, do not attempt to do it yourself. In any event, this is a complex system. Although we believe that this document contains good and valuable information, we offer this only for descriptive purposes and we will not be responsible for any consequences that arise from the use of this information.

The Hardware

For the Internet computer we are using small boxes obtained from JDR, which have essentially "everything" built into the motherboard, the exception being a telephone line modem connection, which we do not need. The box has network, sound card, four USB 2 ports, serial and parallel ports, two extra slots, space for one hard drive, one floppy drive and one CDROM drive. We added an 80 GB hard drive, 256MB of memory, floppy and CDROM drive to make up the basic computer. There is no reason another brand or even a full-sized computer would not do just as well. We do like this small box, however. It runs cool and quiet and despite having room for only one hard drive, it is quite possible to attach a second drive to the IDE cable and run with the cover off the box to do such things as direct copying between drives.

Tiny Computer


We place this box on top of or near to the workstation computer and connect both machines to the same display, keyboard, and mouse using a "KVM" switch. Such switches vary widely in price. The one we are primarily using (see above photo) is also available from JDR for around $30 and has built-in connection cables for the two computers. That is not much more than the price of just the cables that are required for other switches and these inexpensive switches seem to work well. For file transfer, the Internet computer and the workstation are also connected by a special USB cable, used with the FastLynx software described below. Finally, we connect a stereo cable between the line-out jack on the Internet computer and the line-in jack on the workstation so that the workstation's sound system will play whatever sounds the Internet computer generates. In one case where sound is more important we use a fiber optic cable for that purpose.

Computer Stack

The Software

We chose Windows 2000 as the operating system for the Internet computers for a number of reasons. We have not tested the system described here with other operating systems. (Many people prefer Windows 98SE for the Internet since it has fewer vulnerabilities than Windows 2000 or XP, but we are less worried about vulnerabilities for this computer.) In particular, we have not tried this system with Windows XP. There is no reason why it should not work with XP providing that the activation process does not complain and we do not believe that it will. We would certainly like to hear from anyone who does try this method using XP. Of course XP has a built-in system restore feature, but it is far from being a complete system recovery. Sometimes it would work satisfactorily, but other times not, which makes it useless for this task.

You will, of course, require some basic office software for preliminary examination and testing of files, but as most of your work will remain on the workstation, there is no need for much more than your browser, e-mail program, and whatever other programs you use in directly dealing with the Internet. That will likely include a software firewall even if you have a hardware firewall (we use both) so that you can keep track of and control which programs use the internet, especially as servers. We have found ZoneAlarm version 5.5 performs very well at that task. You may want to run a virus protection program. Our internal policy has been that we generally do not run virus checkers, preferring instead to use an e-mail program (Eudora) set with its own mail reader which will not allow automatic activation of e-mail viruses upon opening the e-mail. We also have a set of simple rules about file attachments, and a rule about never going directly to a url sent in an e-mail. Thus even brand new viruses do not affect us. So far, that policy has thwarted all but one such attack on our systems and that was from a Trojan horse in a game downloaded and installed against the rules by an employee nearly ten years ago. (The success of this policy continues for our three high speed Internet systems after a year of use.) But we do not even intend to imply that is the right course for everyone. By all means, use a virus checker program if you wish. Spyware is another matter and one that we are still evaluating. Although we take precautions against spyware, we believe we will experience spyware attacks. If they are infrequent, or at least if the difficult-to-deal-with ones are infrequent, we will simply restore the system following such an attack. If attacks are frequent, we may install a spyware checker. (So far, one year later, we have not needed a spyware checker.)

The system requires a few items of software that you probably are not already using. One of these is a disk partitioning program. For this we specifically recommend 7tools Partition Manager 2005, which can be found available on the web. We actually tried three different partitioning programs before we were able to do what we wanted and what is required for the methods explained below. If the other programs were capable of doing what we wanted, it was not evident to us as we tried them, but as setting up this system was a learning process perhaps we missed something. The 7tools performs well and performs quickly. The only problem with 7tools is that it only works under Windows. If the system won't boot, you can't use 7tools to fix it. (This can happen if you forget to make the primary partition active during a recovery. We used a copy of PowerQuest's Partition Magic 8 when that happened to us during testing.) Vcom's Copy Commander is also very useful for directly copying between disk partitions on two drives if you intend to set up more than one system. (Those of you who are familiar with the free Knoppix - QTParted will recognize that it also can be used to set the primary partition active.) And finally, Sewell's FastLynx 3.3 is also available over the Internet although you will have to order the special USB cables that it requires to communicate between computers. FastLynx does not use network software to communicate across the USB connection and thus even malicious software that gets through to your Internet computer cannot get to or even access your workstation. Unfortunately for our purposes, FastLynx does have the capability of doing its thing across the Internet as well as across the special USB cable, so someone else running FastLynx could conceivably get to your Internet computer. This rather remote possibility is thwarted first by being sure that FastLynx has its Internet link capability turned off and second by setting ZoneAlarm to not allow communication between FastLynx and the Internet in either direction. FastLynx, shown below, is a fairly normal-looking application, but it was also the only satisfactory communication program that we found. FasyLynx occasionally requires a reboot of the Internet computer in order to reestablish communication, but this has been rare and it general it has worked very well.


Partitioning the Hard Drive

The key to making this system work is properly partitioning the hard drive. C F Systems has traditionally used the C: partition for systems, as is the case for nearly everyone, but we have also tended to use an F: partition for applications that are not system-related; the dividing line is a bit fuzzy. We also tend to have a major data area, with no programs installed in it as partition G: Our uses for partitions D: and E: vary from system to system and in the case of the Internet computer they do not exist as such.

While it is not necessary to copy this pattern of usage, it must be understood in order to make sense of the following, and we do have what we consider good reasons for using this pattern. First, we try to keep the system partition, C:, as free as possible of all data that has direct meaning to us as computer users. In particular, we use Microsoft's Internet Explorer which insists on storing its data on C:, so it is necessary for us to be concerned with being able to retrieve the Favorites and Cookies folders. Other than that, applications which prefer to store user-relevant data in their own nest (such as Eudora) are relegated to F:. The only program applications which we intentionally place on C: are those required to make the system boot without problems, usually directly related to specific hardware. The rule of thumb is that the system should be able to boot up even if the F: partition has disappeared. All user data files which for which we can direct placement, which is most data, goes on G:. This makes it a simple matter to produce systematic, successive backups each day, week, or whatever of all data according to date, getting all files that are important to us as users, without getting the current state of all system files - basically meaningless to us - as well. We do such incremental data backups regularly, but on the new Internet computers we also do an entirely different system backup.

To build our system using the C:, F:, and G: system with an 80GB disk we decided on the following partitioning:

Partition Letter Content Size, GB Label Comment
C: System 10 C-10 Active
Extended Partition Follows   70   Contains all the following
*: System copy1 10+ CCOPY  
*: System copy 2 10+ CCOPY2  
F: Applications 10 F-10  
*: Applications copy 10+ FCOPY  
G: Main Data 30 G-35  

Setting this up can be a little more tricky than it looks because disk partitioners do quite a bit of rounding and adjusting of numbers to accommodate the hardware and system software requirements and partition sizes are never exactly what you specify. The sizes in the above table are nominal guidelines. It very likely will be easier or even necessary to give letters (D: E:, etc.) to the unlettered partitions as they are being set up and remove and rearrange the letters after the partitioning is completed. Be sure that you make the "copy" partitions slightly larger than the "original" for each or you will not be able to make the required copies later on.

The way this arrangement works in practice is that at a time when the system is working well and changes have been made that should be retained (usually new applications or drivers installed) the system partition C: is copied to the unlettered partition CCOPY, and the application partition F: is copied to the unlettered partition FCOPY. These partitions are temporarily given letters (D: and E:) and Winzip is used to make complete zipped copies of the CCOPY and FCOPY partitions for archive (with the archive zips placed on the data partition G:). Note that if we tried to make a zipped copy of C: directly we would run into serious problems because important system files would be in use and would not be included in the copy. After making the zip archive copies, the letters are removed from CCOPY and FCOPY partitions, making them completely inaccessible. Thus we have on the disk a readily available copy of the last known good system.

In the event of trouble, to revert to the known good system we first, if possible, make zipped copies of C: and F:. (There is an alternate procedure if this cannot be done.) We then start a sequence in which the problematic C: is copied to CCOPY2 so we may extract data from it if necessary. Then CCOPY, the known good system is copied to C: and the matching FCOPY is copied to F: and with C: made active, the last known good system has been restored. Data may be retrieved from the zipped copies of the bad C: and F:.

The copying of partitions by 7tools is quite fast, taking only a few minutes. The slowest part of the described procedure is making archive copies using Winzip, which may take ten or even twenty minutes by itself. But the archive copies are important in case what you thought was the last known good system really was not good. The complete partitions can easily be restores from the archive zips.

Sample Backup and Recovery Procedures

Our internal document CFS-249, Saving and Restoring the System on the Fast Internet Computers , (283KB PDF) details using 7tools and Winzip for the system backup and restore procedures with the specific system setup we use.

E-mail us (cfs@colorperfect.com). We try to respond to all legitimate e-mails that we receive. If you do not get a response, please see the NOTICE on our Privacy Policy Page.
Privacy Policy and E-Mail
Go to C F Systems Home Page