Distributed computing

Distributed computing is a specialized field of computer science. It focuses on the study of distributed systems. These systems consist of multiple components located on different networked computers. Instead of one machine doing all the work, these components work together to achieve a common goal. This goal might be solving a massive computational problem. It might also be coordinating shared resources for many different users. In these systems, the user often perceives the entire collection of processors as a single, unified unit.

To understand how these systems function, we must look at their core mechanism: message passing. In a distributed system, each computational entity, or node, is autonomous. This means each node has its own local, private memory. Because they do not share memory, they must communicate by sending messages to one another. This process allows them to coordinate their actions and share necessary information. A problem is typically divided into many smaller tasks. Each task is then solved by one or more computers through this continuous exchange of messages.

There are several ways to organize these systems, known as architectural patterns. In a client-server architecture, smart clients contact a central server to request data. In a three-tier architecture, the intelligence is moved to a middle tier to simplify deployment. This is how most web applications function today. An n-tier architecture goes even further by forwarding requests to various enterprise services. Another common pattern is peer-to-peer, or P2P. In P2P systems, there are no special machines managing the network. Instead, all machines are called peers, and they divide responsibilities uniformly. Examples of this include the BitTorrent and bitcoin networks.

Distributed computing is often compared to parallel computing, though they have distinct differences. In parallel computing, all processors may have access to a single shared memory to exchange information. This is considered a tightly coupled form of computing. In contrast, distributed computing is a loosely coupled form. In these systems, every processor has its own private, distributed memory. While the terms sometimes overlap, a rule of thumb exists: high-performance parallel computation usually uses parallel algorithms. Meanwhile, the coordination of a large-scale distributed system relies on distributed algorithms.

The history of this field is tied to the evolution of networking. The roots of using concurrent processes with message-passing date back to operating system studies in the 1960s. In the 1970s, the invention of Ethernet provided the first widespread local-area networks. During the late 1960s, ARPANET was introduced as a predecessor to the Internet. ARPANET e-mail, invented in the early 1970s, became one of the earliest large-scale distributed applications. Later, in the 1980s, networks like Usenet and FidoNet supported distributed discussion systems. Distributed computing finally became its own branch of computer science in the late 1970s and early 1980s.

Building these systems offers great advantages, such as scalability and durability. A system is considered scalable if the marginal cost of an additional workload remains nearly constant. Distributed systems are also more durable because they can tolerate the failure of individual components. If one computer in the network fails, the entire system does not fail. However, these benefits come with higher costs. Distributed architectures require more hardware, such as servers, gateways, firewalls, and proxies. They also require complex setups involving new subnets to function correctly.

Modern developers use different methods to handle data within these systems, such as events and messages. An event represents a specific fact or a change in state, like an order being placed. These are often broadcast to many listeners to promote scalability. Messages are broader; they can be commands, events, or even large data payloads. These tools allow for different delivery guarantees, such as at-least-once or exactly-once delivery. By combining these approaches, modern architectures can manage complex workflows and massive amounts of data across the globe.