Distributed computing

Many computers work as one team.

They talk to each other to finish a big job. They send messages to share work. This helps the team reach a goal.

Each computer has its own memory. This means they keep their own notes. If one computer breaks, the team keeps going. The whole system does not stop. This makes the team very strong. It is like many friends helping you. Do you like working in a team?

Computers can work together as a team. This is called distributed computing.

In this system, many computers talk to each other. They send messages to share information. Each computer has its own local memory. This means they keep their own private notes.

This is different from parallel computing. In parallel computing, all parts share one memory.

Working in a team has some hard parts. The computers must coordinate their actions. They also have to work without a single clock to tell time. One big plus is that the system is strong. If one computer fails, the whole system does not fail. This makes it very durable.

There are different ways to set up these teams. In a client-server setup, smart clients ask a server for data. In a peer-to-peer system, every machine is a peer. This means all machines share the work equally. Examples include BitTorrent and the bitcoin network. These systems can be very large and help solve big problems.

Distributed computing is a special way for computers to work together. Instead of one machine doing all the work, many computers join as a team. These computers are called nodes or computational entities. Each node has its own private memory to store its own notes.

They use a network to talk to one another. This allows them to reach a common goal. This teamwork helps solve very large problems. It can also help many people use the same service at once.

How does this team of computers actually work? It starts with a big problem that gets divided into many small tasks. Each task is solved by one or more computers. To stay organized, they use something called message passing.

This is like sending a letter to a teammate to share information. Some systems use a client-server setup where smart clients ask a server for data. Other systems use a peer-to-peer setup. In peer-to-peer, every machine is a peer and shares the work equally.

This field of study has a long and interesting history. Scientists began studying how processes communicate in the 1960s. In the 1970s, local-area networks like Ethernet were invented. This was a very important step for connecting computers.

ARPANET was also a major part of this early era. It was a predecessor to the modern Internet. One of its most famous uses was email. People started using email in the early 1970s. This was one of the first large-scale distributed applications ever made.

There are many specific facts about how these systems behave. Distributed systems can be more scalable than a single machine. This means they can handle more work easily. A system is scalable if the cost of adding more work stays nearly constant.

These systems are also very durable. If one computer in the team fails, the whole system does not fail. However, these systems can cost more to build. You need extra hardware like servers, gateways, and firewalls. They also need new subnets and proxies to work correctly.

You can see distributed computing in many things you use today. Massive multiplayer online games use these systems to keep many players connected. Peer-to-peer applications like BitTorrent also use this method. Even the bitcoin network works this way using peers.

It is different from parallel computing. In parallel computing, all processors share one memory. In distributed computing, every processor has its own private memory. This makes distributed computing a more loosely coupled way to work. It allows computers to stay connected even if they are far apart.

Distributed computing is a specialized field of computer science. It focuses on the study of distributed systems. These systems consist of multiple components located on different networked computers. Instead of one machine doing all the work, these components work together to achieve a common goal. This goal might be solving a massive computational problem. It might also be coordinating shared resources for many different users. In these systems, the user often perceives the entire collection of processors as a single, unified unit.

To understand how these systems function, we must look at their core mechanism: message passing. In a distributed system, each computational entity, or node, is autonomous. This means each node has its own local, private memory. Because they do not share memory, they must communicate by sending messages to one another. This process allows them to coordinate their actions and share necessary information. A problem is typically divided into many smaller tasks. Each task is then solved by one or more computers through this continuous exchange of messages.

There are several ways to organize these systems, known as architectural patterns. In a client-server architecture, smart clients contact a central server to request data. In a three-tier architecture, the intelligence is moved to a middle tier to simplify deployment. This is how most web applications function today. An n-tier architecture goes even further by forwarding requests to various enterprise services. Another common pattern is peer-to-peer, or P2P. In P2P systems, there are no special machines managing the network. Instead, all machines are called peers, and they divide responsibilities uniformly. Examples of this include the BitTorrent and bitcoin networks.

Distributed computing is often compared to parallel computing, though they have distinct differences. In parallel computing, all processors may have access to a single shared memory to exchange information. This is considered a tightly coupled form of computing. In contrast, distributed computing is a loosely coupled form. In these systems, every processor has its own private, distributed memory. While the terms sometimes overlap, a rule of thumb exists: high-performance parallel computation usually uses parallel algorithms. Meanwhile, the coordination of a large-scale distributed system relies on distributed algorithms.

The history of this field is tied to the evolution of networking. The roots of using concurrent processes with message-passing date back to operating system studies in the 1960s. In the 1970s, the invention of Ethernet provided the first widespread local-area networks. During the late 1960s, ARPANET was introduced as a predecessor to the Internet. ARPANET e-mail, invented in the early 1970s, became one of the earliest large-scale distributed applications. Later, in the 1980s, networks like Usenet and FidoNet supported distributed discussion systems. Distributed computing finally became its own branch of computer science in the late 1970s and early 1980s.

Building these systems offers great advantages, such as scalability and durability. A system is considered scalable if the marginal cost of an additional workload remains nearly constant. Distributed systems are also more durable because they can tolerate the failure of individual components. If one computer in the network fails, the entire system does not fail. However, these benefits come with higher costs. Distributed architectures require more hardware, such as servers, gateways, firewalls, and proxies. They also require complex setups involving new subnets to function correctly.

Modern developers use different methods to handle data within these systems, such as events and messages. An event represents a specific fact or a change in state, like an order being placed. These are often broadcast to many listeners to promote scalability. Messages are broader; they can be commands, events, or even large data payloads. These tools allow for different delivery guarantees, such as at-least-once or exactly-once delivery. By combining these approaches, modern architectures can manage complex workflows and massive amounts of data across the globe.