Published: 12 July 2024
Contributors: Phill Powell, Ian Smalley
Distributed computing uses numerous computing resources in different operating locations for a single computing purpose.
Distributed computing systems make things happen in a big way. Think about it the next time you play a massively multiplayer online (MMO) game. To effectively manage an MMO, it takes entire systems coming together and working in close conjunction to deliver a seamless online experience that’s simultaneously being shared by thousands of players in a real-time environment.
Distributed computing helps providers marshal that kind of coordinated synchronization and computer processing power toward a common goal.
And that’s just one example of the many, wide-ranging applications for distributed computing services now in use, including creating vivid 3D graphics for video animations, training neural networks that mimic the human brain's functionality, and solving extremely complex equations related to chemistry and physics.
Typically, the distributed computing model takes the most intensive computing tasks and workloads and deals with the most serious computational challenges, which is why it requires the use of multiple components and shared memory.
Distributed computing brings together multiple computers, servers and networks to accomplish computing tasks of widely varying sizes and purposes. In small, distributed computing systems with components near each other, these components can be linked through a local area network (LAN).
In larger distributed systems whose components are separated by geography, components are linked through wide area networks (WAN). The components in a distributed system share information through whichever type of network is being used. The best-known example of a distributed system is the Internet. Perhaps the most controversial example of a distributed system in recent times is cryptocurrency.
From a computational perspective, in terms of the number of components typically involved, even the most basic forms of distributed computing usually contain three basic components:
- Primary system controller
- System data store
- Database
However, there aren’t any hard and fast rules limiting what constitutes a circumstance of distributed computing or how many software components are needed to bring its functionality to life. There could be a handful of components working within that chain (or network), or the instance could incorporate considerably more.
This is one of the great advantages of using a distributed computing system; the system can be expanded by adding more machines. The other significant advantage is increased redundancy, so if one computer in the network fails for whatever reason, the work of the system continues unabated, despite that point of failure.
This concept of added redundancy goes hand in hand with an emphasis on fault tolerance. Fault tolerance is a corrective process that allows an OS to respond and correct a failure in software or hardware while the system continues to operate. Fault tolerance has come to be used as a general measure of ongoing business viability in the face of a disrupting failure.
So distributed computing systems aren’t bound by limits on the number of components. Nor is there any requirement that those components be physically located near each other and connected by local area networks. Thanks to the expanded capabilities of wide area networks, the computers of a distributed computing system can be separated by continents while still retaining their operability.
The goal of distributed computing systems is to make that distributed computing network—no matter how large and no matter where its components may be located—function as if it were a single computer. This coordination is accomplished through an elaborate system of message-passing between the various components.
Communication protocols govern that back-and-forth exchange of messages and create a form of relationship that exists between these components. This relationship is known as “coupling,” which is typically expressed in one of two forms:
- Loose coupling: The connection between two loosely coupled components is weak enough so that alterations to one component will not impact the other component.
- Tight coupling: The level of synchronization and parallelism is so great in tightly coupled components that a process called “clustering” uses redundant components to ensure ongoing system viability.
Distributed computing also deals with both the positive and negative effects of concurrency, which is the simultaneous execution of multiple operating instruction sequences. Chief among its positive qualities is the fact that concurrency enables the parallel computing of multiple process threads. (Parallel computing should not be confused with parallel processing, which is a process whereby runtime tasks are broken down into multiple smaller tasks.)
The negatives associated with concurrency often include increased latency and even bottlenecks wherein traffic grinds to a near-halt due to overloading the distributed computing system with an excessive number of component requests.
Concurrency differs from the concept of multithreading, which allows one program to remain running while other tasks are being performed in the background. Concurrency represents a larger drain on resources since multiple threads are running at the same time and accessing shared resources simultaneously.
Some processing workloads are enormous, and more than most single systems can accommodate. Distributed computing shares such workloads among multiple pieces of equipment, so big jobs can be tackled capably.
Distributed computing operates based on the concept of shared assets, and it supports the sharing of resources like data, software and hardware between the various components within that distributed computing instance.
The online traffic of many organizations is subject to rapid and dramatic changes, possibly because of reported news or other factors. Distributed computing provides the flexibility companies need to withstand such surges.
Distributed computing ramps up performance levels by splitting large processing jobs into smaller computing tasks, which are shared by numerous computers. This results in a cumulative amount of processing speed and greater total storage.
While it’s true that distributed computing, with its multiple systems, possesses more up-front costs than centralized processing systems, over time, those costs average out because of cost efficiencies achieved through lower latency and faster response times, ultimately making distributed computing more cost-effective than centralized systems.
Distributed computing types are classified according to the distributed computing architecture each employ.
Client-server systems use a client-server architecture that allows them to be used with more than a single system. In it, a client will direct input to the server in the form of a request. This communication will typically involve either a command that a particular task be performed or a request for more computing resources. The server then works to fulfill that task or allocate the requested resources. Once accomplished, the server responds to the client with a message concerning the action taken.
While centralized systems use client-server architecture, a peer system relies upon peer architecture (often called peer-to-peer architecture). Peer systems utilize nodes, which effectively can function as either client or server—either identifying needs and issuing requests or working to fulfill those needs and report back on operations. As the name implies, there’s no hierarchy in peer systems, so programs operating in peer-to-peer systems can communicate freely with each other and transfer data via peer networks.
Middleware can be thought of as a type of “middleman” that operates between two distinct applications, which is to say that middleware is itself an application that resides between two apps and supplies service to both. Middleware has an interpretive aspect as well. It functions as a translator between various interoperability apps that are being run on different systems and allows those apps to freely exchange data.
Three-tier systems are so named because of the number of layers used to represent a program’s functionality. As opposed to typical client-server architecture in which data is placed within the client system, the three-tier system instead keeps data stored in its middle layer, which is called the Data Layer. The Application Layer surrounds the Data Layer on one of its sides, while the Presentation Layer surrounds the Data Layer on the other side. Three-tier systems are routinely used in various web applications.
Sometimes referred to as multitiered distributed systems, N-tier systems are unlimited in their capacity for network functions, which they route to other apps for processing. The architecture of N-tier systems is like that found in three-tier systems. N-tier systems are often used as the architectural basis for numerous web services and data systems.
Distributed computing enables the use of different programming paradigms (which can be thought of as programming styles):
- Blockchain: Blockchain is a kind of distributed database or ledger that is both replicated and synchronized on the various computers that make up a network. A blockchain ensures one of distributed computing’s main goals—redundancy—because each computer in the chain holds a copy of the original source ledger, so data can’t be lost due to a single point of failure.
- Grid computing: Grid computing is a type of distributed computing that deals with non-interactive workloads, usually involving a combination of grid frameworks and middleware software. The scalable grid accessed through the user interface functions like a mega-sized file system.
- Microservices: Microservices is another form of distributed computing in which applications are broken down into much smaller components, often called “services.” Services frameworks are conjoined by the application program interface (API), which enables interaction between components.
Online banking shows the dexterity of distributed computing by occurring in numerous places at once, often with different bank locations charged with housing one of the servers behind key service offerings (like customer service, security and transaction processing). Everything happens instantly and seamlessly, just as bank customers expect.
Both artificial intelligence (AI) and machine learning (ML) are in the process of revolutionizing many technological fields, but nowhere are their effects more pronounced right now than in the computing industry itself. Both exploding technologies make extensive use of distributed computing. The algorithms behind AI and ML need large volumes of data to train their models, and distributed computing is supplying the processing muscle that’s required.
Distributed computing is now being used by the energy and environmental sectors. On the energy side, distributed computing is helping smart-grid technology regulate usage and optimize energy consumption. Smart grids are also using distributed computing to assemble environmental data from different input devices, like sensors and smart meters.
Workers in specific areas of finance are already using distributed computing systems. Take risk management, in which financial institutions need vast realms of data to execute huge calculations to better inform decision-making about probability and risk assessments. Distributed computing ensures computational loads are shared evenly across multiple systems.
Distributed computing has assisted numerous scientific disciplines and contributed to some of science’s most notable achievements. Examples include the Large Hadron Collider, the world’s most powerful particle accelerator. The experiments behind it depend on extreme amounts of data collection and analysis, requiring the use of distributed computing. Similarly, distributed computing served the same purpose for the Human Genome Project, as it set out to map human DNA sequences. Distributed computing also performs essential roles in weather forecasting and climate modeling.
Various sophisticated operating systems run on IBM mainframes—the security-rich, resilient and agile platform for integrating into your hybrid cloud strategy.
Take advantage of automated provisioning and seamless management with high-performance computing (HPC) options from IBM, like integrated compute infrastructure and tooling, workload schedulers and high-performance storage. Manage computing demand peaks by extending capacity and pay for only what you use.
Automate operations, improve experiences and enhance safety measures. Act on insights closer to where data is created.
Get a Message Passing Interface (MPI) that speeds up application performance in distributed computing environments. In addition, it also supplies a portable interface based on open-source MPI. Discover advanced CPU affinity features, stronger performance and streamlined workload manager integrations and more.
High-performance computing is being used to tackle many of today’s most difficult computational tasks, like running artificial intelligence (AI) algorithms, automating stock trades and helping make self-driving cars that are safer to operate.
Discover how you can operate public cloud infrastructure in various locations with distributed cloud. Run the data centers you want—from any cloud providers, colocation centers or on-premises environment—and manage it all from one control pane.
Get the scalability your workloads demand with hyperscale. Build distributed databases and scale these database systems almost limitlessly. Get benefits like managed data centers, enhanced networking capabilities and more efficient performance.
The processing of data can overload a system and bog down its performance with latency issues. Edge computing combats this by relocating many computational tasks to edge devices, where they can deliver faster performance and safer operation.
Take advantage of added flexibility with the Apache Hadoop open-source software framework. Process large data sets with simple programming models. Hadoop gives companies a cost-effective way to store and process data in large-scale amounts.
Virtual machines (VMs) generate virtual representations of physical computers that run according to software instructions. Because VMs share resources, you can use a single machine that can successfully run multiple machines fully and virtually.