Multithreading (computer architecture)

A computer brain can do many tasks.

This is called multithreading. It helps the computer stay busy. Sometimes, one task has to wait. It might wait for data to arrive. Instead of sitting still, the brain switches. It starts a new task right away. This keeps the computer moving fast. It uses all its parts well. This makes the whole system work better. It is a smart way to work.

A computer brain is called a CPU. It can do many tasks at once. This way of working is called multithreading.

Most computers do many tasks at the same time. Sometimes, one task has to stop and wait. It might wait for data to come from memory. This wait can take a long time. Instead of staying still, the CPU switches to a new task. This keeps the parts of the CPU busy. This helps the whole system work faster.

There are different ways to do this. One way is coarse-grained multithreading. In this way, a task runs until it must wait. Then, the CPU switches to another task. Another way is fine-grained multithreading. This way switches tasks very often. It can switch tasks every single cycle.

The most advanced way is called simultaneous multithreading, or SMT. In SMT, the CPU can work on many tasks at the exact same time. This uses all the available parts of the CPU. Some companies, like Intel, use this to make computers better. It helps the computer get more work done in less time.

A computer brain is called a CPU. It can do many things at once using a method called multithreading.

This is important because it helps the computer finish more work. It is very hard to make a single task go faster. Instead, engineers focus on making the whole system more efficient. Most computer systems use multitasking to handle many threads at once. This keeps the parts of the CPU from sitting idle. When one task stops, another can take its place.

Multithreading works by switching between different paths of work. One way is called coarse-grained multithreading. In this way, a task runs until it hits a long wait. This wait might happen if the CPU needs data from memory. This can take hundreds of cycles to finish. Instead of waiting, the CPU switches to a ready task. The hardware does this by keeping two sets of registers. Registers are small storage spots that hold the current task's info.

There is also a faster way called fine-grained multithreading. This method switches tasks very often. It can even switch every single CPU cycle. This was once called barrel processing. In that name, the parts of the processor act like staves in a barrel. This type of switching helps remove pauses in the work pipeline. However, it requires more hardware to track each task's ID. It also needs larger caches to prevent tasks from interfering with each other.

The most advanced version is simultaneous multithreading, or SMT. A normal processor usually handles one task at a time. An SMT processor can issue instructions from many threads in one cycle. This uses up all the empty slots in the processor. Many famous companies use this technology today. Intel calls its version Hyper-Threading Technology. IBM uses it in their POWER and z13 series chips. AMD also uses it in their Zen microarchitectures.

Using multithreading has both good and bad sides. It can make a program run much faster. For example, some tests show a 100% speed improvement. But, multiple threads can also fight over the same hardware. This struggle is called contention. It can actually make a single task run slower. Some special programs, like video encoders, do not need this help. They are already tuned to use the hardware perfectly.

Multithreading is a method used in computer architecture to increase efficiency. It allows a central processing unit (CPU), or a single core, to provide multiple threads of execution. A thread is essentially a path of work within a computer program. Since the late 1990s, engineers have used multithreading to improve throughput computing. This is important because making a single thread run faster has become very difficult. Instead of focusing on one task, computers now focus on finishing more total work.

To understand how this works, we must look at why a CPU might sit idle. Sometimes, a thread hits a stall, which is a long pause in work. This often happens during a cache miss, when the CPU must wait for data from off-chip memory. This wait can last for hundreds of CPU cycles. In a multithreaded system, the processor does not just sit still during this wait. Instead, the hardware switches from the stalled thread to another thread that is ready to run. This process keeps the computing resources busy and improves overall performance.

There are several distinct types of multithreading used in modern computers. The simplest version is known as block, cooperative, or coarse-grained multithreading. In this type, one thread runs until it is blocked by a specific event. To make switching fast, the hardware often replicates certain parts. For example, the processor might be built with two sets of registers. Registers are small storage areas that hold a thread's current information. By having multiple sets, the hardware can switch between threads very quickly.

Another type is fine-grained multithreading, which is also called interleaved or time-sliced multithreading. This method is more aggressive because it switches threads every single CPU cycle. This approach aims to remove all data dependency stalls from the execution pipeline. In the past, this was sometimes called barrel processing. This name comes from the idea that the pipeline stages act like the staves of a barrel. However, this method has extra hardware costs. Each stage of the pipeline must track the thread ID of every instruction it processes.

Simultaneous multithreading (SMT) is the most advanced type of this technology. While other types are called temporal multithreading because they issue one thread at a time, SMT is different. An SMT processor can issue instructions from multiple different threads during the very same CPU cycle. This is used in superscalar processors to fill unused issue slots. By using instructions from many threads at once, the processor exploits parallelism across the entire system. This prevents the waste that occurs when a single thread cannot use all available resources.

Many major technology companies have implemented these advanced methods. Intel uses a version of SMT called Hyper-Threading Technology. IBM uses it in their POWER and z13 through z15 series chips. AMD also uses SMT in their Bulldozer and Zen microarchitectures. Sun Microsystems used it in the UltraSPARC T2, and Cray produced the XMT. These different implementations show how vital multithreading has become to high-performance computing.

While multithreading is powerful, it does have some significant disadvantages. Multiple threads often share hardware resources like caches or translation lookaside buffers (TLBs). When threads fight over these shared parts, it is called contention. This contention can actually degrade the performance of a single thread. Furthermore, merging data from two different processes can be very expensive. This overhead, caused by synchronization and inter-process communication, can be two or more orders of magnitude higher than single-thread processing.

Despite these challenges, the benefits can be quite large. Intel claims that Hyper-Threading can provide up to a 30% improvement in efficiency. In some cases, a synthetic program can see a 100% speed improvement when run in parallel. However, not all software benefits from this. For instance, hand-tuned assembly language programs, like some video encoders, do not suffer from idle resources. These specialized programs might actually see slower performance when hardware multithreading is active.