What is distributed computing?

Distributed computing systems make things happen in a big way. Think about it the next time you play a massively multiplayer online (MMO) game. To effectively manage an MMO, it takes entire systems coming together and working in close conjunction to deliver a seamless online experience that’s simultaneously being shared by thousands of players in a real-time environment.

Distributed computing helps providers marshal that kind of coordinated synchronization and computer processing power toward a common goal.

And that’s just one example of the many, wide-ranging applications for distributed computing services now in use, including creating vivid 3D graphics for video animations, training neural networks that mimic the human brain's functionality, and solving extremely complex equations related to chemistry and physics.

Typically, the distributed computing model takes the most intensive computing tasks and workloads and deals with the most serious computational challenges, which is why it requires the use of multiple components and shared memory.

Distributed computing brings together multiple computers, servers and networks to accomplish computing tasks of widely varying sizes and purposes. In small, distributed computing systems with components near each other, these components can be linked through a local area network (LAN).

In larger distributed systems whose components are separated by geography, components are linked through wide area networks (WAN). The components in a distributed system share information through whichever type of network is being used. The best-known example of a distributed system is the Internet. Perhaps the most controversial example of a distributed system in recent times is cryptocurrency.

From a computational perspective, in terms of the number of components typically involved, even the most basic forms of distributed computing usually contain three basic components:

• Primary system controller

• System data store

• Database

However, there aren’t any hard and fast rules limiting what constitutes a circumstance of distributed computing or how many software components are needed to bring its functionality to life. There could be a handful of components working within that chain (or network), or the instance could incorporate considerably more.

This is one of the great advantages of using a distributed computing system; the system can be expanded by adding more machines. The other significant advantage is increased redundancy, so if one computer in the network fails for whatever reason, the work of the system continues unabated, despite that point of failure.

This concept of added redundancy goes hand in hand with an emphasis on fault tolerance. Fault tolerance is a corrective process that allows an OS to respond and correct a failure in software or hardware while the system continues to operate. Fault tolerance has come to be used as a general measure of ongoing business viability in the face of a disrupting failure.

So distributed computing systems aren’t bound by limits on the number of components. Nor is there any requirement that those components be physically located near each other and connected by local area networks. Thanks to the expanded capabilities of wide area networks, the computers of a distributed computing system can be separated by continents while still retaining their operability.

The goal of distributed computing systems is to make that distributed computing network—no matter how large and no matter where its components may be located—function as if it were a single computer. This coordination is accomplished through an elaborate system of message-passing between the various components.

Communication protocols govern that back-and-forth exchange of messages and create a form of relationship that exists between these components. This relationship is known as “coupling,” which is typically expressed in one of two forms:

• Loose coupling: The connection between two loosely coupled components is weak enough so that alterations to one component will not impact the other component.

• Tight coupling: The level of synchronization and parallelism is so great in tightly coupled components that a process called “clustering” uses redundant components to ensure ongoing system viability.

Distributed computing also deals with both the positive and negative effects of concurrency, which is the simultaneous execution of multiple operating instruction sequences. Chief among its positive qualities is the fact that concurrency enables the parallel computing of multiple process threads. (Parallel computing should not be confused with parallel processing, which is a process whereby runtime tasks are broken down into multiple smaller tasks.)

The negatives associated with concurrency often include increased latency and even bottlenecks wherein traffic grinds to a near-halt due to overloading the distributed computing system with an excessive number of component requests.

Concurrency differs from the concept of multithreading, which allows one program to remain running while other tasks are being performed in the background. Concurrency represents a larger drain on resources since multiple threads are running at the same time and accessing shared resources simultaneously.

Distributed computing types are classified according to the distributed computing architecture each employ.

Client-server systems use a client-server architecture that allows them to be used with more than a single system. In it, a client will direct input to the server in the form of a request. This communication will typically involve either a command that a particular task be performed or a request for more computing resources. The server then works to fulfill that task or allocate the requested resources. Once accomplished, the server responds to the client with a message concerning the action taken.

While centralized systems use client-server architecture, a peer system relies upon peer architecture (often called peer-to-peer architecture). Peer systems utilize nodes, which effectively can function as either client or server—either identifying needs and issuing requests or working to fulfill those needs and report back on operations. As the name implies, there’s no hierarchy in peer systems, so programs operating in peer-to-peer systems can communicate freely with each other and transfer data via peer networks.

Middleware can be thought of as a type of “middleman” that operates between two distinct applications, which is to say that middleware is itself an application that resides between two apps and supplies service to both. Middleware has an interpretive aspect as well. It functions as a translator between various interoperability apps that are being run on different systems and allows those apps to freely exchange data.

Three-tier systems are so named because of the number of layers used to represent a program’s functionality. As opposed to typical client-server architecture in which data is placed within the client system, the three-tier system instead keeps data stored in its middle layer, which is called the Data Layer. The Application Layer surrounds the Data Layer on one of its sides, while the Presentation Layer surrounds the Data Layer on the other side. Three-tier systems are routinely used in various web applications.

Sometimes referred to as multitiered distributed systems, N-tier systems are unlimited in their capacity for network functions, which they route to other apps for processing. The architecture of N-tier systems is like that found in three-tier systems. N-tier systems are often used as the architectural basis for numerous web services and data systems.

Distributed computing enables the use of different programming paradigms (which can be thought of as programming styles):

• Blockchain: Blockchain is a kind of distributed database or ledger that is both replicated and synchronized on the various computers that make up a network. A blockchain ensures one of distributed computing’s main goals—redundancy—because each computer in the chain holds a copy of the original source ledger, so data can’t be lost due to a single point of failure.

• Grid computing: Grid computing is a type of distributed computing that deals with non-interactive workloads, usually involving a combination of grid frameworks and middleware software. The scalable grid accessed through the user interface functions like a mega-sized file system.

• Microservices: Microservices is another form of distributed computing in which applications are broken down into much smaller components, often called “services.” Services frameworks are conjoined by the application program interface (API), which enables interaction between components.