16-Nov-2007: The VAX Revolution
Coincidentally, the day after I wrote the previous entry it was announced that IBM had agreed to purchase Ottawa-based
Cognos for five billion dollars. Cognos' main products are business management tools. In the early 1980's, the
period I was writing about, Cognos was called Quasar and its main product was PowerHouse, a predecessor of today's
complex data-mining and business management tools. Also, about the same time that I was getting my feet wet in the
computer world, another company, Oracle, was producing the first user-friendly data-base management tools.
There are a couple of interesting points about Cognos (Quasar) and Oracle I want to touch on in this entry. But, first
I want to try to convey the importance of these new tools. A major component of computer use is the ability to
search through data quickly and link it to other data. When I wrote my little credit-request management application
at Energy, Mines, and Resources (EMR), I used the existing tools available to me. This meant that I designed and used an indexed-sequential
file. Indexed-sequential files were developed to speed up computer searches. They did this by the way the files were designed. Each record
of data (for example: the date, credit request number, user account number, type of problem, and the status of the request would constitute
an entire record) had one or more keys assigned to it. In my sample record, I could decide that I wanted to index the date, credit request number, and
its status. That way I could quickly retrieve a single request, or all the requests entered on a given date, or get a list of all the
records whose status was "pending." I would set up those keys when I defined the file and the computer would identify each record with a unique
tag (actually a relative memory address), then prepare three lists: one for each key with its record tag. These lists could be sorted very quickly by the computer. When I wanted
a list of all records with a given date, then the computer had only to search a simple list of keys and match the tags to the records. There are various
search algorithms that sped up searching the lists of keys and there were tools to reorganize and optimize the files.
As you may have noticed, this was something that a non-programmer could not set up, but it was something a non-programmer could use
as long as there was a "user-friendly" interface. The common user interfaces at the time were either documentation so that
someone with rudimentary computer skills could make use of it, or by interactive prompts. The former was used by those who needed to generate
reports, for example; the latter by people who needed to have their computer communications controlled by the computer. This might be the
case, for example, for a receptionist who used a program to look up department telephone numbers. She would type a simple command, like: "Phone" and
the program would prompt her for the name of the person whose number she required and would display it for her. Though, in the early
1980's a receptionist would most likely still be relying on a rolodex or printed lists of telephone numbers.
Applications like PowerHouse and Oracle were developed to help people trained in business management use computers to do what previously
only programmers could do: set up databases and mine them, and in terms that they understood. Oracle had developed a Structured Query Language
(SQL) that non-programmers could learn quickly to help them in their management and use of information. An SQL database search command might look something
like: "Return all records where the date is between 01-January-1982 and 01-March-1982 and where the status is pending." Oracle and PowerHouse included
linked packages that would let a non-programmer user change and reorganize data and produce reports tailored to their specific needs. All of this was very new
at the time and it took a new kind of computer to make their use widespread.
In the early 1980's there were basically two kinds of computers available: mainframes and personal computers. Personal computers were still a new phenomenon
and, generally speaking, you had to be able to write a lot of your own code if you wanted to use them. And, they were not powerful enough to use in business.
Mainframes demanded trained specialists to look after and interact with them. Friendly user interfaces were unknown (until Apple introduced the first Graphical User
Interface (GUI) now familiar in its Windows incarnations), as mainframes processed either computer programs
or simple, one-line commands. However, there was a middle-ground. A new class of computers, called "mid-range" was being developed, led mainly by
Digital Equipment Corp., or DEC, as it was familiarly known. These computers could support multiple users and programs, unlike personal computers, and
were far less expensive and far easier to use than mainframes. Additionally, they gave the user access to resources much greater than
what was available in a personal computer (typically, personal computers at the time had use of 64 KB of memory space, while the users of the VAXes I looked
after had use of about one MB of memory). The VAX processors handled 32 bits of information at a time, while personal computers then could handle only 8 bits.
Mid-range computers were sometimes called "department-level" computers because they could be used to help
manage a small business, or a department within a larger organization. DEC's VAX line of computers introduced an improvement in the computing world that is still
a key element in most computers today, though it was unheard of at the time: virtual memory. What this means is that the amount of memory required by
a computer to hold executing programs and data was no longer limited by the physical size of the memory board. With a simple algorithm memory could be
"swapped out" (written to a hard disk drive) and retrieved when it was required. The operating system was called VMS (for "Virtual Memory System") and
the many of the engineers who designed it later designed Windows NT, the basis of subsequent Microsoft operating systems. (It's no
coincidence that Windows New Technology (the full name of Win NT) abbreviates to WNT, each letter one removed from its equivalent in VMS.)
Though IBM had a mid-range line of computers, they were extremely expensive and difficult to use. At one time in the computer room
we had two Cyber mainframes, an IBM 360, and five VAXes. It took a team of several dozen people to manage the Cybers, including two CDC employees on-site
five days a week. It took a team of about a dozen, plus frequent visits from IBM personnel, to keep the 360 going. I, single-handedly,
looked after all the management needs of the VAXes. At one time we compared the raw computing power of the three type of machines. The Cybers
crunched data very quickly: that's what they were designed to do; however, they were very slow when it came to processing text. The VAXes were almost
as fast as the Cybers at number crunching but processed text almost as fast as they processed numbers. The IBM system was one hundred times (two decimal places)
slower than the VAXes at
either job. The results of the study were quickly suppressed because they were an embarrassment to the groups promoting IBM. Their
jobs depended on them having slow, expensive, difficult-to-use and -maintain machines in the computer room.
Both Oracle and PowerHouse were originally designed and sold on VMS machines, and VAXes became very common in computer rooms everywhere. Unfortunately, often the
technical expertise to manage these machines was in short supply—a factor that led to the popular conception that they were unreliable. At EMR, when my temporary
stint as front desk technical support came to an end, I was assigned the job of understanding the first VAX 11/780 in our computer room and of setting up system management
and operation procedures. I spent two three-week stints in Toronto getting the training I needed to master the day-to-day operations. I developed an almost intuitive
understanding of these machines. Within a couple of years I was delivering talks at national conventions of systems folks on performance management of VAXes; in other words,
how to get the most out of a VAX and, hopefully, keep the user community happy.
VAXes and VMS were taking the mystery out of managing and using computers and making the power of computing available to more people for more applications.
I've already mentioned Oracle and PowerHouse, but there were many tools being developed. DEC promoted its own office management "solution" called All-in-1. What
All-in-1 basically did was provide management of office memos and documents, including document tracking. Businesses and government departments were anxious for such tools. EMR, at
the time, had a group of about thirty to forty translators. This group purchased a VAX 11/750 (about 0.5 MIPS) in order to do just that: track documents as they
progressed through the various hands from beginning to end of the translation process. It was a tool that they had a need for, but their technical lead on the project had no computer
expertise at all and had no idea how to scope the technical requirements to the required tasks. Also, DEC had a habit of underselling their products in order
to gain a foothold. They hoped then to leverage their position into larger sales that would, eventually, meet the end users' requirements. (I saw this happen too often in my twenty-plus
years to ascribe it to one bad salesman.)
So, there I was, in the position of system manager of a system that could, with the best tuning and performance management I could deliver, support no more than twelve
simultaneous users (fourteen if I stretched a point). The absolute maximum in any case was sixteen, as that was the number of slots available for lines in their communications box
(though one line was dedicated to printing).
Their technical lead took his frustrations out on me. He would telephone daily to make outrageous demands. At one time he demanded that I change the display that he
saw when he checked the stats on his session to show a BAUD rate for his connection as much higher than what he was actually using. It didn't matter to him that he
was at the maximum BAUD rate available and what he was seeing was just a confirmation that he was, in fact, using the maximum rate. He wanted to see 9600 instead of
4800 and no amount of exasperated explanation from me was going to change his mind. We battled like that for months. At one point he wanted to add another communications
box to his system by plugging it into one of the slots on the current communications box. It made no sense at all to do this and would have had no impact on the fact that the system
was scaled to handle a dozen or so users. Originally he had full access to the inner workings of the system, but I documented his abuses of this privilege and his
manager asked me to take away his employee's system access. The final outcome was that he was fired and the entire project was scaled back and eventually phased out.
But the fact remained that there was a growing demand for this kind of computer resource. Management saw the savings they could realize by handing over labour-intensive
tasks, like tracking documents or inventory, to mid-sized computers. And the brightest of them saw the potential for access to information in itself as a money-maker. I'm sure my
experience with the earliest attempts to make the leap from multi-million dollar specialized scientific number-crunching machines to general purpose machines that smaller
groups could afford and manage, was being recreated around the world. Some of the projects must have been successful, but not without some pain. But, by the end
of the 1980's every government department used VAXes in one capacity or another. In fact, the governments of Russia and China depended almost
exclusively on VAXes (which was behind some of the earliest cases of computer hacking). In about 1988 I was invited to a tour of the CN-CP telecommunications hub
in Toronto. They had warehouse-sized rooms filled with VAXes, managing the country's most vital communications links. I recall being at a meeting where the
possibility of a single repository of all government departments' financial records was being discussed. IBM was touted as the standard. A representative from
the Department of Corrections stood up and said he had two hundred VAXes and he was not going to retrofit them so they could communicate with IBM. IBM should change
so it could talk to other computers. The Department of Fisheries and Oceans voiced a similar objection. The Department of Defence was not there, but I
knew that VAXes were being used extensively to manage Canada's military. (This was twenty years ago, so I don't feel that I am spilling any classified information. I have
no idea what technology these departments are using today—and if I did I wouldn't tell you.)
Then, about 1984, DEC introduced another revolutionary idea: clustering. Essentially what this meant was that two or more computers could be linked together
to act like a single larger computer. In DEC's approach, the integration was seamless. Only the system manager knew which machine's CPU and memory a user
was accessing. The disk drives (and hence the data) could be accessed in exactly the same way from any of the cluster members. Not only was this a huge leap forward
in terms of efficiency and performance issues, but it meant that VAXes now had the potential for infinite scalability. My EMR translation sector could have simply bought a
second VAX, created a cluster with their existing 11/750, and had enough resources for their project to be a success. Other fallout from clustering involved greatly enhanced reliability.
A machine could be taken out of the cluster, all the applications and users would be automatically moved to another machine, and the machine serviced. The possibility
then, of running a computer twenty-four hours a day, seven days a week was now realized. No more endless waits by users for a system under maintenance or emergency repair
to be brought back
into service. With the implementation of the RAID standards, disk drives could be mirrored, so that disks would be backing each other up in real time as data was modified and being written to them. The configurations
this allowed meant that disk access could be shared between, say, three disk drives, effectively increasing the response time by nearly three-fold. A disk could drop out
of the set and no one notice.
Eventually, as
communications improved, VAX clusters could be mirrored remotely. In other words, clusters could be separated by up to 500 miles, and they would
still function as one unit. In the event that a computer room in New York lost power, users could continue working, and data would be secure, on a system
located in Pennsylvania. This is why today many huge corporations, like banks and stock exchanges, that require fault-free twenty-four by seven service rely on
the VAX descendants, the Alpha line of computers. However, that is largely unknown to the general public because of the general perception that only IBM can deliver
the huge computing resources that such companies need. There are a lot of myths floating about the computing world; fortunately, there are very smart people
in some areas who put performance and security ahead of buying what the rest of the herd is buying. And when you read about major banks, stock exchanges,
governments, or major corporations losing millions because their computer systems were down for hours or days, I am sure that they are not using Alpha clustering (now owned by Hewlett-Packard).
Anyhow, that was the exciting changing world I lived in. I became a VAX/VMS systems specialist at a time when the demand for such specialists could not
be met, and I played an active role in some of the other less-dramatic changes that were occurring in the computing world.
I went from a $11,000/year junior operator to a self-employed consultant charging $475-$500/day for my services—and getting it without argument.