Download Ds unit 1 and more Study notes Algorithms and Programming in PDF only on Docsity!
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
UNIT-
DISTRIBUTED SYSTEMS
Basic Issues
- What is a Distributed System?
- Examples of Distributed Systems
- Advantages and Disadvantages
- Design Issues with Distributed Systems
- Preliminary Course Topics
What is a Distributed System? A distributed system is a collection of autonomous computers linked by a computer network that appear to the users of the system as a single computer. Some comments: System architecture: the machines are autonomous; this means they are computers which, in principle, could work independently; The user’s perception: the distributed system is perceived as a single system solving a certain problem (even though, in reality, we have several computers placed in different locations).
By running distributed system software the computers are enabled to:
- Coordinate their activities
- Share resources: hardware, software, data.
Examples of Distributed Systems
1. Network of workstations
Personal workstations + processors not assigned to specific users. Single file system, with all files accessible from all machines in the same way and using the same path name. For a certain command the system can look for the best place (workstation) to execute it.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
2. Automatic banking (teller machine) system
Primary requirements: security and reliability. Consistency of replicated data. Concurrent transactions (operations which involve accounts in different banks; simultaneous access from several users, etc). Fault tolerance
Some more examples of Distributed Real-Time Systems
Synchronization of physical clocks Scheduling with hard time constraints Real-time communication Fault tolerance
Advantages of Distributed System Performance: very often a collection of processors can provide higher performance (and better price/performance ratio) than a centralized computer. Distribution: many applications involve, by their nature, spatially separated machines (banking, commercial, automotive system). Reliability (fault tolerance): if some of the machines crash, the system can survive. Incremental growth: as requirements on processing power grow, new machines can be added incrementally. Sharing of data/resources: shared data is essential to many applications (banking, computer supported cooperative work, reservation systems); other resources can be also shared (e.g. expensive printers).
Disadvantage of Distributed System
Difficulties of developing distributed software: how should operating systems, programming languages and applications look like?
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
2. Communication: Issue: Components of a distributed system have to communicate in order to interact. This implies support at two levels: 1. Networking infrastructure (interconnections & network software). 2. Appropriate communication primitives and models and their implementation: a. communication primitives: i. send ii. receive iii. remote procedure call (RPC) b. communication models i. Client-Server communication: implies a message exchange between two processes: the process which requests a service and the one which provides it; ii. Group Multicast: the target of a message is a set of processes, which are members of a given group. 3. Performance Several factors are influencing the performance of a distributed system: The performance of individual workstations. The speed of the communication infrastructure. Extent to which reliability (fault tolerance) is provided (replication and preservation of coherence imply large overheads). Flexibility in workload allocation: for example, idle processors (workstations) could be allocated automatically to a user’s task. 4. Scalability The system should remain efficient even with a significant increase in the number of users and resources connected: Cost of adding resources should be reasonable; Performance loss with increased number of users and resources should be controlled; Software resources should not run out (number of bits allocated to addresses, number of entries in tables, etc.) 5. Heterogeneity Distributed applications are typically heterogeneous Different hardware: mainframes, workstations, PCs, servers, etc.; Different software: UNIX, MS-Windows, IBM OS/2, Real-time OSs, etc.;
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Unconventional devices: teller machines, telephone switches, robots, manufacturing systems, etc.; Diverse networks and protocols: Ethernet, FDDI, ATM, TCP/IP, Novell Netware, etc. An additional software layer called middleware used to mask heterogeneity.
6. Openness One of the important features of distributed systems is openness and flexibility: Every service is equally accessible to every client (local or remote); It is easy to implement, install and debug new services; Users can write and install their own services. Key aspect of openness: Standard interfaces and protocols (like Internet communication protocols) Support of heterogeneity (by adequate middleware, like CORBA) 7. Reliability and Fault Tolerance One of the main goals of building distributed systems is improvement of reliability.
Availability: If machines go down, the system should work with the reduced amount of resources. There should be a very small number of critical resources; Critical resources: resources which have to be up in order the distributed system to work. Key pieces of hardware and software (critical resources) should be replicated i.e. if one of them fails another one takes up - redundancy. Data on the system must not be lost, and copies stored redundantly on different servers must be kept consistent. The more copies kept, the better the availability, but keeping consistency becomes more difficult.
Fault-tolerance is a main issue related to reliability: the system has to detect faults and act in a reasonable way: Mask the fault: continue to work with possibly reduced performance but without loss of data/ information. Fail gracefully: react to the fault in a predictable way and possibly stop functionality for a short period, but without loss of data/information.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Client -Server Model The system is structured as a set of processes, called servers that offer services to the users, called clients.
The client-server model is usually based on a simple request/reply protocol, implemented with send/receive primitives or using remote procedure calls (RPC) or remote method invocation (RMI): The client sends a request (invocation) message to the server asking for some service; The server does the work and returns a result (e.g. the data requested) or an error code if the work could not be performed. A server can itself request services from other servers; thus, in this new relation, the server itself acts like a client.
Some problems with client-server: Centralization of service ⇒ poor scaling Limitations: capacity of server, bandwidth of network connecting the server.
Peer-to-Peer Model
All processes (objects) play similar role. Processes (objects) interact without particular distinction between clients and servers. The pattern of communication depends on the particular application. A large number of data objects are shared; any individual computer holds only a small part of the application database. Processing and communication loads for access to objects are distributed across many computers and access links. This is the most general and flexible model.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Peer-to-Peer tries to solve the problems in centralized system. It distributes shared resources widely share computing and communication loads. Problems with peer-to-peer: High complexity due to Cleverly place individual objects Retrieve the objects Maintain potentially large number of replicas.
2. Interaction Models
Issue: How do we handle time? Are there time limits on process execution, message delivery, and clock drifts? Synchronous distributed systems Asynchronous distributed systems
Synchronous Distributed Systems
Main features: Lower and upper bounds on execution time of processes can be set. Transmitted messages are received within a known bounded time. Drift rates between local clocks have a known bound.
Important consequences: In a synchronous distributed system there is a notion of global physical time (with a known relative precision depending on the drift rate). Only synchronous distributed systems have a predictable behavior in terms of timing. Only such systems can be used for hard real-time applications. In a synchronous distributed system it is possible and safe to use timeouts in order to detect failures of a process or communication link. It is difficult and costly to implement synchronous distributed systems.
Asynchronous Distributed Systems Many distributed systems (including those on the Internet) are asynchronous. No bound on process execution time (nothing can be assumed about speed, load, and reliability of computers). No bound on message transmission delays (nothing can be assumed about speed, load, reliability of interconnections) No bounds on drift rates between local clocks.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Arbitrary (Byzantine) Faults This is the most general and worst possible fault semantics. Intended processing steps or communications are omitted or/and unintended ones are executed. Results may not come at all or may come but carry wrong values.
Timing Faults Timing faults can occur in synchronous distributed systems, where time limits are set to process execution, communications, and clock drifts. A timing fault occurs if any of these time limits is exceeded.
COMMUNICATION IN DISTRIBUTED SYSTEM
Communication Models and their Layered Implementation
The communication between distributed objects by means of two models:
- Remote Method Invocation (RMI)
- Remote Procedure Call (RPC).
RMI, as well as RPC, are based on request and reply primitives. Request and reply are implemented based on the network protocol (e.g. TCP or UDP in case of the Internet).
Network Protocol Middleware and distributed applications have to be implemented on top of a network protocol. Such a protocol is implemented as several layers. In case of the Internet:
TCP (Transport Control Protocol) and UDP (User Datagram Protocol) are both transport protocols implemented on top of the Internet protocol (IP).
TCP (Transport Control Protocol) TCP is a reliable protocol. TCP guarantees the delivery to the receiving process of all data delivered by the sending process, in the same order.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
TCP implements additional mechanisms on top of IP to meet reliability guarantees. o Sequencing: A sequence number is attached to each transmitted segment (packet). At the receiver side, no segment is delivered until all lower numbered segments have been delivered. o Flow control: The sender takes care not to overwhelm the receiver (or intermediate nodes). This is based on periodic acknowledgements received by the sender from the receiver. o Retransmission and duplicate handling: If a segment is not acknowledged within a specified timeout, the sender retransmits it. Based on the sequence number, the receiver is able to detect and reject duplicates. o Buffering: Buffering is used to balance the flow between sender and receiver. If the receiving buffer is full, incoming segments are dropped. They will not be acknowledged and the sender will retransmit them. o Checksum: Each segment carries a checksum. If the received segment doesn’t match the checksum, it is dropped (and will be retransmitted)
UDP (User Datagram Protocol) UDP is a protocol that does not guarantee reliable transmission. UDP offers no guarantee of delivery. According to the IP, packets may be dropped because of congestion or network error. UDP adds no additional reliability mechanism to this. UDP provides a means of transmitting messages with minimal additional costs or transmission delays above those due to IP transmission. Its use is restricted to applications and services that do not require reliable delivery of messages. If reliable delivery is requested with UDP, reliability mechanisms have to be implemented at the application level.
Request and Reply Primitives Communication between processes and objects in a distributed system is performed by message passing. In a typical scenario (e.g. client-server model) such a communication is through request and reply messages
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Who are the players? Object A asks for a service Object B delivers the service
1. The proxy for object B If an object A holds a remote reference to a (remote) object B, there exists a proxy object for B on the machine which hosts A. The proxy is created when the remote object reference is used for the first time. For each method in B there exists a corresponding method in the proxy. The proxy is the local representative of the remote object ⇒ the remote invocation from A to B is initially handled like a local one from A to the proxy for B. At invocation, the corresponding proxy method marshals the arguments and builds the message to be sent, as a request, to the server. After reception of the reply, the proxy unmarshals the received message and sends the results, in an answer, to the invoker. 2. The skeleton for object B On the server side, there exists a skeleton object corresponding to a class, if an object of that class can be accessed by RMI. For each method in B there exists a corresponding method in the skeleton. The skeleton receives the request message, unmarshals it and invokes the corresponding method in the remote object; it waits for the result and marshals it into the message to be sent with the reply. A part of the skeleton is also called dispatcher. The dispatcher receives a request from the communication module, identifies the invoked method and directs the request to the corresponding method of the skeleton.
Implementation is as follows
Object A and Object B belong to the application. Remote reference module and communication module belong to the middleware. The proxy for B and the skeleton for B represent the so called RMI software. They are situated at the border between middleware and application and usually can be generated automatically with help of available tools that are delivered together with the middleware software.
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Communication module The communication modules on the client and server are responsible of carrying out the exchange of messages which implement the request/reply protocol needed to execute the remote invocation. The particular messages exchanged and the way errors are handled, depends on the RMI semantics which is implemented (see slide 40).
Remote reference module The remote reference module translates between local and remote object references. The correspondence between them is recorded in a remote object table. Remote object references are initially obtained by a client from a so called binder that is part of the global name service (it is not part of the remote reference module). Here servers register their remote objects and clients look up after services.
Remote Procedure Call (RPC)
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
2. Logical clocks
In many applications we are not interested in the physical time at which events occur; what is important is the relative order of events! The make- program is such an example. In such situations we don’t need synchronized physical clocks. Relative ordering is based on a virtual notion of time - logical time. Logical time is implemented using logical clocks.
Lamport’s Logical Clocks
1. The order of events occurring at different processes is critical for many distributed applications. Example: P.o_created and P.c_created. 2. Ordering can be based on two simple situations: I. If two events occurred in the same process then they occurred in the order observed following the respective process; II. Whenever a message is sent between processes, the event of sending the message occurred before the event of receiving it. 3. Ordering by Lamport is based on the happened before relation (denoted by →): I. a → b, if a and b are events in the same process and a occurred before b; II. a→b, if a is the event of sending a message m in a process, and b is the event of the same message m being received by another process; III. If a → b and b → c, then a → c (the relation is transitive). 4. Ifa→b, we say that event a causally affects event b. The two events are causally related. 5. There are events which are not related by the happened-before relation. If both a → e and e → a are false, then a and e are concurrent events; we can write a||e.
P1, P2, P3: processes; a, b, c, d, e, f: events; a → b, c → d, e → f, b → c, d → f a → c, a → d, a → f, b → d, b → f, ... a || e, c || e, ...
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
6. Using physical clocks, the happened before relation cannot be captured. It is possible that b → c and at the same time Tb > Tc (Tb is the physical time of b). 7. Logical clocks can be used in order to capture the happened-before relation. A logical clock is a monotonically increasing software counter. There is a logical clock CPi at each process Pi in the system. The value of the logical clock is used to assign timestamps to events. CPi(a) is the timestamp of event a in process Pi. There is no relationship between a logical clock and any physical clock.
To capture the happened-before relation, logical clocks have to be implemented so that if a → b, then C(a) < C(b)
Implementation Rules of Lamport’s Logical Clocks Implementation of logical clocks is performed using the following rules for updating the clocks and transmitting their values in messages:
[R1]: CPi is incremented before each event is issued at process Pi: CPi := CPi + 1. [R2]: a) When ‘a’ is the event of sending a message ‘m’ from process Pi, then the timestamp tm = CPi (a) is included in ‘m’. (CPi(a) is the logical clock value obtained after applying rule R1). b) On receiving message ‘m’ by process Pj, its logical clock CPj is updated as follows: CPj := max(CPj, tm). c) The new value of CPj is used to timestamp the event of receiving message ‘m’ by Pj (applying rule R1).
If ‘a’ and ‘b’ are events in the same process and ‘a’ occurred before ‘b’, then a→b, and (by R1) C(a) < C(b). If ‘a’ is the event of sending a message ‘m’ in a process, and ‘b’ is the event of the same message ‘m’ being received by another process, then a→b, and (by R2) C(a) < C(b). If a → b and b → c, then a → c, and (by induction) C(a) < C(c).
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
M1 has to be processed before M2 and M3. However P3 has not to wait for M3 in order to process it before M2 (although M3’s logical clock timestamp is smaller than M2’s).
Vector Clocks Vector clocks give the ability to decide whether two events are causally related or not by simply looking at their timestamp. Each process Pi has a clock CPi, which is an integer vector of length n (n is the number of processes). The value of CPi is used to assign timestamps to events in process Pi. CvPi(a) is the timestamp of event a in process Pi. CPi[i], the ith entry of CPi, corresponds to Pi’s own logical time. CPi*j+, j ≠ i, is Pi’s "best guess" of the logical time at Pj. CPi[j] indicates the (logical) time of occurrence of the last event at Pj which is in a happened before relation to the current event at Pi.
Implementation of Vector Clock Implementation of vector clocks is performed using the following rules for updating the clocks and transmitting their values in messages: [R1]: CPi is incremented before each event is issued at process Pi: CPi[i] := CPi[i] + 1. [R2]: a) When ‘a’ is the event of sending a message ‘m’ from process Pi, then the timestamp tm = CPi(a) is included in ‘m’ (CPi(a) is the vector clock value obtained after applying rule R1). b) On receiving message ‘m’ by process Pj, its vector clock CPj is updated as follows: ∀k ∈ {1, 2, .., n}, CPj[k] := max(CPj[k], tm[k]). c) The new value of CPj is used to timestamp the event of receiving message ‘m’ by Pj (applying rule R1).
For any two vector timestamps u and v, we have: u = v if and only if ∀i, u[i] = v[i] u ≤ v if and only if ∀i, ui+ ≤ vi+ u < v if and only if (u ≤ v ∧ u ≠ v) u || v if and only if ¬(u < v) ∧ ¬(v < u)
DEPARTMENT OF INFORMATION TECHNOLOGY
I.T.S ENGINEERING COLLEGE, GREATER NOIDA
Causal Order Two events a and b are causally related if and only if o C(a) < C(b) or C(b) < C(a). Otherwise the events are concurrent. With vector clocks we get the property which we missed for Lamport’s logical clocks: o a → b if and only if Cv(a) < Cv(b). Thus, by just looking at the timestamps of the events, we can say whether two events are causally related or not.
Causal Ordering of Messages Using Vector Clocks
We would like messages to be processed according to their causal order.
If Send(M1) → Send(M2), then every recipient of both messages M1 and M2 must receive M1 before M2.
Basic Idea: A message is delivered to a process only if the message immediately preceding it (considering the causal ordering) has been already delivered to the process. Otherwise, the message is buffered. We assume that processes communicate using broadcast messages. (There exist similar protocols for non-broadcast communication too.) The events which are of interest here are the sending of messages ⇒ vector clocks will be incremented only for message sending.
Implementation of Causal Order
BIRMAN-CHIPER-STEPHENSON PROTOCOL (BES) Implementation of the protocol is based on the following rules: [R1]: a) Before broadcasting a message m, a process Pi increments the vector clock: CPi[i] := CPi[i] + 1. b) The timestamp tm = CPi is included in m. [R2]: The receiving side, at process Pj, delays the delivery of message ‘m’ coming from Pi until both the following conditions are satisfied:
