• see Gordon Bell slide show
• see Cray Research and Cray computers FAQ
• 64 bit words
• details (local)

• A vector machine, with 8 groups of 64 words
• You can load a group (vector) of words with one instruction
• You can multiply or add(subtract) one group (vector) by another vector giving a third vector.
• There is one product and sum produced each clock - 12 nanoseconds
• The machine was also very fast in scalar operations - badly beating many competing machines in this also important function. It was about twice as fast as the 7600 in scalar (normal - not vector) operations. 64 scalar registers available for easier code sequencing and optimizing.
• Pipelines could be linked - you could have the multiply pipeline connect to the add pipeline for the relatively common
  (vector A times vector B) plus vector C giving vector D
• All the above concurrently, a floating point result every 12 nanoseconds
• The above gives the sometimes quoted speed of 160 million floating point operations (Mflops) per second
• And of course while the above computing is going on, a write is sending a previous 64 word "vector" to memory, or filling a 64 word vector with new data from memory.
• 1 millions words of main memory was an available options
• Used ECL (Emitter Coupled Logic) - very fast, very power hungry
• There was no divide instruction! If you needed to divide, you performed a "reciprical approximation" ( makes 1/value ) then multiplied by the reciprical. (Same thing - faster in hardware - a fast divide is the bane of computer designers.)
• Gorden Bell says the Cray 1 "is remarkably similar to the 6600... and extended for vectors"

• Production model had 8 processors
• Ran in a tank of cooling liquid called fluorinert, the liquid that is used for heart transplant. This liquid was pumped through the Cray 2 and the heat of the modules boiled the liquid which was then cooled by refrigeration and recirculated.
• This helped keep the system at a uniform, stable temperature and within the speed range of the semiconductors (their speed is temperature dependent)
• To trouble shoot this system, you needed to drain the coolant. How to run it at a stable temperature for trouble shooting? - Turn the power on and run the machine for 2 milliseconds, then turn off the power for at least 1 second. Examine the results.

The Cray 1 was first delivered in 1976. This was around the same time that 8-bit microprocessors were beginning to gain popularity, typical memory components were 1K bit SRAM and 4 K bit DRAM. Most machines were operating at about a 1 MHz clock rate, had 32-bit words, and large mainframes had 1 MB to 8 MB of RAM.

The Cray 1 had (Baron and Higbie CS manual)

• 64-bit words
• 8 MB of RAM
• 16-way interleaving on low-order bits
• 50 ns memory cycle
• 12.5 ns clock cycle (80 MHz)
• 12 pipelined functional units

The Cray 1 has 3 basic data types: addesses (24-bit integer), integers (64-bit), floating point (64-bit, 48-bit mantissa).

The 12 functional units are divided into four groups.

Group 1 -- Vector units

The machine itself is divided into six major subsystems

• Memory
• Instruction component
• Address component
• Scalar component
• Vector component
• I/O component
• Instruction Component

Cray 1 instructions are 32 or 16 bits, so from 2 to 4 instructions can be packed into a word. Instructions are thus addressed on 16-bit boundaries while data is addressed on 64-bit boundaries.

The instruction unit has four 16-word instruction buffers, three instruction registers, and one instruction counter. Each 16-bit field in a word is called an instruction parcel.

The three instruction registers are

• Next Instruction Parcel -- holds first parcel of the next instruction, prefetched from buffer
• Current Instruction Parcel -- holds the high-order portion of the instruction to be issued
• Lower Instruction Parcel -- holds low-order portion of instruction to be issued

For a 32-bit instruction, the low-order portion is fetched to the NIP and then moved to the LIP. There is no mechanism for discarding instructions in the pipe -- once in the CIP/LIP, they will be issued. At most they will be delayed for some time.

The instruction buffers are tied to the memory via the 16-way interleaving, so it is possible to fill a buffer in 4 clock cycles (recall that the clock is 12.5 ns and memory is 50 ns). Buffers are filled on a demand basis in a round-robin pattern. They thus act as an instruction cache of 256 instructions, organized into four lines of 64 instructions. Each buffer has its own address comparator, so we would call this a fully associative cache (easy to implement when there are only 4 lines). The buffers cannot be written to -- a write bypasses the instruction cache and only goes to main memory.

Scalar instruction issue requires that all of the instruction's required resources be free -- otherwise the instruction waits. Vector instruction issue in the Cray involves reserving functional units, including memory, operand registers and result registers, and then releasing an instruction once all of its resources are available. In addition, some data paths are shared between the vector and scalar components, and these must be available.

The control unit is able to detect when a result register for one vector operation is an operand for another vector operation and, if the two vector instructions do not conflict in any other resource requirements, it sets up a vector chaining operation between the two instructions.

Address Component

There are 8 24-bit address registers, 64 24-bit spill registers, an adder, and a multiplier in this component. Its purpose is to perform index arithmetic and send the results to the scalar and vector components so that they can fetch the appropriate operands.

Arithmetic is performed on the address registers directly. The spill registers are used to hold address values that do not fit into the address registers. A set of 8 addresses can be transferred between the address registers and their spill registers in a single cycle. Thus, they bear a certain similarity to the register windows of the SPARC (or vice versa). The spill registers can be thought of as an explicitly managed data cache with 8 lines. Their value is that they reduce the traffic to main memory, freeing that resource for vector operations.

Scalar Component

Similar to the address component, the scalar component has 8 64-bit registers and 64 64-bit spill registers. It has sole access to four functional units: Integer Add, Logical, Shift, and Population Count. The Scalar Component also has access to three functional units that are shared with the Vector Component: Floating Add, Multiply, and Reciprocal Approximation.

Because the scalar component has its own integer units, it can always execute integer operations in parallel with a vector operation. However, for floating point, the vector unit takes priority.

Vector Component

The are 8 64-word vector registers in the vector component. It takes four memory loads to fill a vector register. Normally, this would require 16 instruction cycles. However, careful pipelining in the memory unit reduces the time to just 11 cycles.

A vector mask register contains a bit-map of the elements in a register operand that will participate in an instruction. A vector length register determines whether fewer than 64 operands are contained in a set of vector operands. Manipulating these values is the primary reason for the population and leading zeros counter.

Vector loads and stores specify the first location, the length, and the stride.

I/O Component

The I/O component has 24 programmable I/O channel units. I/O has the lowest priority for memory access.

Cray XM-P

• Extended the Cray-1 architecture to 4-way multiprocessing.
• Cycle reduced to 8.5 ns (117 MHz)
• Increased instruction buffers to 32 words
• Added a multiport memory system.
• Redesigned the vector unit to support arbitrary chaining.
• Added Gather/Scatter to support sparse arrays.
• Increased memory to 16 M words, 32-way interleave
• Provides a set of shared registers to support fine-grained (loop-level) multiprocessing. There are N+1 sets of these registers for an N-processor system. They include eight address registers, 8 scalar registers, and 32 binary semaphores.
• The I/O system was improved and a solid state disk cache was added.

Cray YM-P

• Extends the XM-P architecture to 8 processors.
• Cycle reduced to 6 ns (166 MHz)
• Extends memory to 128 M words

Cray 2

• One foreground and four background processors.
• 4.1 ns cycle (244 MHz)
• Up to 256 M words of memory
• 64 or 128 way interleave depending on configuration
• Eliminates the spill registers in favor of a 16K word cache
• Cache feeds all three computational components with 4-cycle access time
• Has 8 16-word instruction buffers
• Foreground processor controls the I/O subsystem, which has up to 4 high speed communication channels (4 Gb/s).

Practical Considerations in Supercomputer Design To achieve such high speeds, high-power (i.e. hot) drivers are employed, signals are detected with specialized analog circuits, conductors are all shielded and precisely tuned in both impedance and length, and data is encoded with error-correcting so that losses can be recovered.

In addition, the circuits are usually designed to operate in balanced mode so that there is no change in power drawn as drivers switch. As one driver switches from low to high, another switches from high to low, so that the power supply sees a DC load and there is no coupling of switching noise back into the logic via the power supply. In addition, using balanced signal lines can increase the signal to noise ratio by 6dB, although these are not often used. In a design such as the Cray-1, roughly 40% of the transistors supposedly do nothing but balance the power loading.

Even so, these machines dissipate large amounts of heat. The IBM 3090 uses special thermal conduction modules in which a multichip substrate is mounted in a carrier with built-in plumbing for a chilled water jacket. CDC used a similar system in its designs, and on one instance a maintenance crew pumped live steam through the building air conditioning system, which crossed over to the processor, with predictable results. This raises the issue that these machines usually need thermal shut-down systems, and possibly even fire suppression gear.

The Cray-1 series uses piped freon, and each board has a copper sheet to conduct heat to the edges of the cage, where freon lines draw it away. The first Cray-1 was in fact delayed six months due to problems in the cooling system: lubricant that is normally mixed with the freon to keep the compressor running would leak through the seals as a mist and eventually coat the boards with oil until they shorted out.

The Cray-2 is unique in that it uses a liquid bath to cool the processor boards. A special nonconductive liquid (flourinert) is pumped through the system and the chips are immersed in this.

Special fountains aerate the liquid, and reservoirs are provided for storing the liquid when it is pumped out for service. This is somewhat remeniscent of the oil cooling bath that was sometimes used in magnetic core memory units.

The ETA-10 was originally going to use a liquid nitrogen bath, but I believe this turned out to be too difficult to implement (on a side note, I have known scientific labs where the researchers deal with cooling problems in air-cooled machines by opening a tank of liquid nitrogen at the inlet, but that's not quite the same).

As a final note, Lawrence Livermore National Labs has announced that it will henceforth buy no more vector supercomputers. The handwriting is clearly on the wall for this breed of system, and all of the major manufacturers are moving, finally, to parallel processing.