Computers and digital devices, from smart phones to simple appliances, have a central processing unit, or CPU, that act as their brain. The CPU fetches instructions from a program and data from the device's memory. It then performs basic arithmetic, logic, control and input/output operations specified by the instructions. These CPUs come in different designs, sizes and features, but their fundamental operations remain the same. Watch this video to learn how a CPU works.

We all want fast devices that can store large amounts of data, and for this we need a fast CPU with a large amount of fast memory to support it. However, there is always a trilemma among speed, size and cost of the memory. Fast memory is usually small and expensive, while large and affordable memory is generally slow. Memory hierarchy lets computer designers optimize their machine’s memory in an economical way by combining different types of memory into layers and choosing what data to store in each layer.

Internal register

The CPU uses internal registers to temporarily hold instructions and data for immediate processing. They are the fastest, most frequently used type of memory, made from a group of fundamental memory circuits called flip-flops. A CPU only has a handful of registers, and they are task specific: their purpose is either to hold numbers or instructions to manipulate those numbers. After each operation, the content in these registers gets replaced by the next set of instructions and data from a larger memory pool within the CPU, called the cache memory.

Cache memory

Without a cache memory, a fast CPU could waste most of its time waiting for data from the slower main memory, causing your device to become very slow. Cache is the internal memory pool of a CPU, made from flip-flops similar to those of the internal registers with the main differences being that the cache is much larger.  They are not task specific and they can be randomly-accessed. Cache contains a partial copy of instructions and data from the main memory that is likely to be processed in the near future. But how does it know what to copy? It has help from a sophisticated bit of hardware called the prefetcher, which does a good job of guessing the content needed for that CPU. 
Main memory

Your running application generally comprises millions of instructions, and the data to be processed can be many times more. They reside in the computer’s main memory, waiting to be fetched by the CPU to process. The main memory is designed to be randomly accessible, hence the name Random-Access Memory, or RAM (as opposed to a sequentially accessed medium such as a cassette tape). Read here to learn the basics of DRAM.

Flash memory in long-term storage

The floating-gate transistor in flash memory has been around longer thank you may think. It has been used since the mid-80's in some types of read-only memory (ROM) to store the device's firmware that is unlikely to be changed or updated. These ROMs are not strictly read-only, but can actually be erased and re-programmed, although with slow speed and often requiring special equipment. Interested in knowing more? Watch this video to learn how floating gate transistors and flash memory work. 

With recent advances in flash memory technology that help improve write speed and also reduce cost, flash memory has gained its popularity in the form of the Solid State Drive (SSD). It has now become an alternative mass storage device to the mechanical hard drives. Although it is several times more expensive, it does have advantages. Read here and here to see how SSDs and hard drives compare against each other.

As opposed to the Single-Level-Cell (SLC) SSD which uses one floating gate transistor to store one bit, newer SSDs lower the cost further by storing 2 bits (or 3 bits) in a single floating gate transistor, thus doubling or tripling their storage capacity. These techniques are called Multi-Level-Cell (MLC) or Triple-Level-Cell (TLC). Read here and here to learn what MLC and TLC drives must sacrifice for that increased capacity. The biggest drawback of flash memory is its limited number of writes - around 10,000 to 100,000 cycles - before the floating gate transistor starts wearing down and can't store any data. Read this white paper from SNIA to learn how engineers try to get around this problem and improve reliability of SSD drives.

Future of computer memory

There is currently much ongoing research to find better, bigger and faster memories. Find out more about some promising memory technologies below:

Ferroelectric RAM

Magnetic RAM

Phase change memory

5-D data storage

Next Section »