--- 1/draft-ietf-mpls-ldp-ft-04.txt 2006-02-05 00:40:12.000000000 +0100 +++ 2/draft-ietf-mpls-ldp-ft-05.txt 2006-02-05 00:40:12.000000000 +0100 @@ -1,34 +1,18 @@ -MPLS Working Group Adrian Farrel -Internet Draft Movaz Networks, Inc. -Document: draft-ietf-mpls-ldp-ft-04.txt -Expiration Date: January 2003 Paul Brittain - Data Connection Ltd. - - Philip Matthews - Hyperchip - - Eric Gray - Celox Networks - - Toby Smith - Laurel Networks - - Andrew G. Malis - Jack Shaio - Vivace Networks - - July 2002 +MPLS Working Group Editor +Internet Draft Adrian Farrel +Document: draft-ietf-mpls-ldp-ft-05.txt Movaz Networks, Inc. +Expiration Date: March 2003 September 2002 Fault Tolerance for the Label Distribution Protocol (LDP) - draft-ietf-mpls-ldp-ft-04.txt + draft-ietf-mpls-ldp-ft-05.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [RFC2026]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute @@ -46,226 +30,237 @@ accessed at http://www.ietf.org/shadow.html. NOTE: The new TLV type numbers, bit values for flags specified in this draft, and new LDP status code values are preliminary suggested values and have yet to be approved by IANA or the MPLS WG. See the section "IANA Considerations" for further details. Abstract - MPLS systems will be used in core networks where system - downtime must be kept to an absolute minimum. Many MPLS - LSRs may, therefore, exploit Fault Tolerant (FT) hardware - or software to provide high availability of the core - networks. + Multiprotocol Label Switching (MPLS) systems will be used + in core networks where system downtime must be kept to an + absolute minimum. Many MPLS Label Switching Routers + (LSRs) may, therefore, exploit Fault Tolerant (FT) + hardware or software to provide high availability of the + core networks. The details of how FT is achieved for the various - components of an FT LSR, including LDP, the switching - hardware and TCP, are implementation specific. This - document identifies issues in the LDP specification - [RFC3036] that make it difficult to implement an FT LSR - using the current LDP protocols, and proposes - enhancements to the LDP specification to ease such FT LSR - implementations. + components of an FT LSR, including Label Distribution + Protocol (LDP), the switching hardware and TCP, are + implementation specific. This document identifies issues + in the LDP specification in RFC 3036 "LDP Specification" + that make it difficult to implement an FT LSR using the + current LDP protocols, and proposes enhancements to the + LDP specification to ease such FT LSR implementations. The issues and extensions described here are equally - applicable to CR-LDP [RFC3212]. + applicable to RFC 3212, "Constraint-Based LSP Setup Using + LDP" (CR-LDP). Contents 1. Conventions and Terminology used in this document 3 - 2. Introduction 4 - 2.1. Fault Tolerance for MPLS 5 - 2.2. Issues with LDP and CR-LDP 5 - 3. Overview of LDP FT Enhancements 7 - 3.1. Establishing an FT LDP Session 8 - 3.1.1 Interoperation with Non-FT LSRs 8 - 3.2. TCP Connection Failure 9 - 3.2.1 Detecting TCP Connection Failures 9 - 3.2.2 LDP Processing after Connection Failure 9 - 3.3. Data Forwarding During TCP Connection Failure 10 - 3.4. FT LDP Session Reconnection 10 - 3.5. Operations on FT Labels 11 - 3.6. Check-Pointing 12 - 3.6.1 Graceful Termination 13 - 3.7. Label Space Depletion and Replenishment 13 - 4. FT Operations 14 - 4.1. FT LDP Messages 14 - 4.1.1 Sequence Numbered FT Label Messages 14 - 4.1.2 FT Address Messages 15 - 4.1.3 Label Resources Available Notifications 15 - 4.2. FT Operation ACKs 17 - 4.3. Preservation of FT State 17 - 4.4. FT Procedure After TCP Failure 19 - 4.4.1 FT LDP Operations During TCP Failure 20 - 4.5. FT Procedure After TCP Re-connection 21 - 4.5.1 Re-Issuing FT Messages 22 - 4.5.2 Interaction with CR-LDP LSP Modification 22 - 5. Checkpointing Procedures 23 - 5.1. Checkpointing with the Keepalive Message 23 - 5.2. Quiesce and Keepalive 24 - 6. Changes to Existing Messages 24 - 6.1. LDP Initialization Message 24 - 6.2. LDP Keepalive Messages 25 - 6.3. All Other LDP Session Messages 25 - 7. New Fields and Values 26 - 7.1. Status Codes 26 - 7.2. FT Session TLV 27 - 7.3. FT Protection TLV 30 - 7.4. FT ACK TLV 32 - 7.5. FT Cork TLV 34 - 8. Example Use 35 - 8.1. Session Failure and Recovery - FT Procedures 36 - 8.2. Use of Check-Pointing With FT Procedures 38 - 8.3. Temporary Shutdown With FT Procedures 40 - 8.4. Temporary Shutdown With FT Procedures and Check-Pointing 42 - 8.5. Checkpointing Without FT Procedures 44 - 8.6. Graceful Shutdown With Checkpointing But No FT Procedures 46 - 9. Security Considerations 47 - 10. Implementation Notes 49 - 10.1. FT Recovery Support on Non-FT LSRs 49 - 10.2. ACK generation logic 49 - 10.2.1 Ack Generation Logic When Using Check-Pointing 49 - 11. Acknowledgments 50 - 12. Intellectual Property Consideration 50 - 13. Full Copyright Statement 51 - 14. IANA Considerations 51 - 14.1. New TLVs 52 - 14.2. New Status Codes 52 - 15. Authors' Addresses 53 - 16. References 53 - 16.1. Normative References 53 - 16.2. Informative References 53 + 2. Contributing Authors 4 + 3. Introduction 4 + 3.1. Fault Tolerance for MPLS 4 + 3.2. Issues with LDP 5 + 4. Overview of LDP FT Enhancements 7 + 4.1. Establishing an FT LDP Session 8 + 4.1.1 Interoperation with Non-FT LSRs 8 + 4.2. TCP Connection Failure 9 + 4.2.1 Detecting TCP Connection Failures 9 + 4.2.2 LDP Processing after Connection Failure 9 + 4.3. Data Forwarding During TCP Connection Failure 10 + 4.4. FT LDP Session Reconnection 10 + 4.5. Operations on FT Labels 11 + 4.6. Check-Pointing 11 + 4.6.1 Graceful Termination 12 + 4.7. Label Space Depletion and Replenishment 13 + 4.8. Tunneled LSPs 13 + 5. FT Operations 14 + 5.1. FT LDP Messages 14 + 5.1.1 Sequence Numbered FT Label Messages 14 + 5.1.2 FT Address Messages 15 + 5.1.3 Label Resources Available Notifications 15 + 5.2. FT Operation ACKs 17 + 5.3. Preservation of FT State 17 + 5.4. FT Procedure After TCP Failure 19 + 5.4.1 FT LDP Operations During TCP Failure 20 + 5.5. FT Procedure After TCP Re-connection 21 + 5.5.1 Re-Issuing FT Messages 22 + 6. Checkpointing Procedures 22 + 6.1 Checkpointing with the Keepalive Message 23 + 6.1 Quiesce and Keepalive 23 + 7. Changes to Existing Messages 24 + 7.1. LDP Initialization Message 24 + 7.2. LDP Keepalive Messages 24 + 7.3. All Other LDP Session Messages 25 + 8. New Fields and Values 25 + 8.1. Status Codes 25 + 8.2. FT Session TLV 26 + 8.3. FT Protection TLV 29 + 8.4. FT ACK TLV 31 + 8.5. FT Cork TLV 33 + 9. Example Use 34 + 9.1. Session Failure and Recovery - FT Procedures 35 + 9.2. Use of Check-Pointing With FT Procedures 37 + 9.3. Temporary Shutdown With FT Procedures 39 + 9.4. Temporary Shutdown With FT Procedures and Check-Pointing 41 + 9.5. Checkpointing Without FT Procedures 43 + 9.6. Graceful Shutdown With Checkpointing But No FT Procedures 45 + 10. Security Considerations 46 + 11. Implementation Notes 47 + 11.1. FT Recovery Support on Non-FT LSRs 47 + 11.2. ACK generation logic 48 + 11.2.1 Ack Generation Logic When Using Check-Pointing 48 + 12. Acknowledgments 49 + 13. Intellectual Property Consideration 49 + 14. Full Copyright Statement 49 + 15. IANA Considerations 50 + 15.1. New TLVs 50 + 15.2. New Status Codes 51 + 16. Authors' Addresses 51 + 17. References 52 + 17.1. Normative References 52 + 17.2. Informative References 52 1. Conventions and Terminology used in this document Definitions of key words and terms applicable to LDP and CR-LDP are inherited from [RFC3212] and [RFC3036]. The term "FT Label" is introduced in this document to indicated a label for which some fault tolerant operation is used. A "non-FT Label" is not fault tolerant and is - handled as specified in [RFC3212] and [RFC3036]. + handled as specified in [RFC3036]. The term "Sequence Numbered FT Label" is used to indicate an FT label which is secured using the sequence number in the FT Protection TLV described in this document. The term "Checkpointable FT Label" is used to indicate an FT label which is secured by using the checkpointing techniques described in this document. The extensions to LDP specified in this document are collectively referred to as the "LDP FT enhancements". Within the context of this draft, "Checkpointing" refers to a process of messages exchanges that confirm receipt and processing (or secure storage) of specific protocol messages. - In the examples quoted, the following notation is used: + When talking about the individual bits in the 16-bit FT + Flag Field, the words "bit" and "flag" is used + interchangeably. + In the examples quoted, the following notation is used: Ln : An LSP. For example L1. - Pn : An LDP peer. For example P1. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. -2. Introduction +2. Contributing Authors + + This document was the collective work of several + individuals over a period of several years. The text and + content of this document was contributed by the editor + and the co-authors listed in section 16. + +3. Introduction High Availability (HA) is typically claimed by equipment vendors when their hardware achieves availability levels of at least 99.999% (five 9s). To implement this, the equipment must be capable of recovering from local hardware and software failures through a process known as fault tolerance (FT). The usual approach to FT involves provisioning backup copies of hardware and/or software. When a primary copy fails, processing is switched to the backup copy. This process, called failover, should result in minimal disruption to the Data Plane. In an FT system, backup resources are sometimes - provisioned on a one-to-one basis (1:1), sometimes as - many-to-one (1:n), and occasionally as many-to-many - (m:n). Whatever backup provisioning is made, the system - must switch to the backup automatically on failure of the + provisioned on a one-to-one basis (1:1), sometimes as one- + to-many (1:n), and occasionally as many-to-many (m:n). + Whatever backup provisioning is made, the system must + switch to the backup automatically on failure of the primary, and the software and hardware state in the backup must be set to replicate the state in the primary at the point of failure. -2.1. Fault Tolerance for MPLS +3.1. Fault Tolerance for MPLS - MPLS will be used in core networks where system downtime - must be kept to an absolute minimum. Many MPLS LSRs may, - therefore, exploit FT hardware or software to provide - high availability of core networks. + MPLS is a technology that will be used in core networks + where system downtime must be kept to an absolute + minimum. Many MPLS LSRs may, therefore, exploit FT + hardware or software to provide high availability of core + networks. In order to provide HA, an MPLS system needs to be able to survive a variety of faults with minimal disruption to the Data Plane, including the following fault types: - failure/hot-swap of a physical connection between LSRs - - failure/hot-swap of the switching fabric in the LSR + - failure/hot-swap of the switching fabric in an LSR - failure of the TCP or LDP stack in an LSR - - software upgrade to the TCP or LDP stacks. + - software upgrade to the TCP or LDP stacks in an LSR. The first two examples of faults listed above are confined to the Data Plane. Such faults can be handled by providing redundancy in the Data Plane which is transparent to LDP operating in the Control Plane. The last two example types of fault require action in the Control Plane to recover from the fault without disrupting traffic in the Data Plane. This is possible because many recent router architectures separate the Control and Data Planes such that forwarding can continue unaffected by recovery action in the Control Plane. -2.2. Issues with LDP and CR-LDP +3.2. Issues with LDP - LDP and CR-LDP use TCP to provide reliable connections - between LSRs over which to exchange protocol messages to - distribute labels and to set up LSPs. A pair of LSRs that - have such a connection are referred to as LDP peers. + LDP uses TCP to provide reliable connections between LSRs + over which to exchange protocol messages to distribute + labels and to set up LSPs. A pair of LSRs that have such + a connection are referred to as LDP peers. - TCP enables LDP and CR-LDP to assume reliable transfer of - protocol messages. This means that some of the messages - do not need to be acknowledged (for example, Label - Release). + TCP enables LDP to assume reliable transfer of protocol + messages. This means that some of the messages do not + need to be acknowledged (for example, Label Release). - LDP and CR-LDP are defined such that if the TCP - connection fails, the LSR should immediately tear down - the LSPs associated with the session between the LDP - peers, and release any labels and resources assigned to - those LSPs. + LDP is defined such that if the TCP connection fails, the + LSR should immediately tear down the LSPs associated with + the session between the LDP peers, and release any labels + and resources assigned to those LSPs. It is notoriously hard to provide a Fault Tolerant implementation of TCP. To do so might involve making copies of all data sent and received. This is an issue familiar to implementers of other TCP applications such as BGP. During failover affecting the TCP or LDP stacks, therefore, the TCP connection may be lost. Recovery from - this position is made worse by the fact that LDP or CR- - LDP control messages may have been lost during the - connection failure. Since these messages are - unconfirmed, it is possible that LSP or label state - information will be lost. + this position is made worse by the fact that LDP control + messages may have been lost during the connection + failure. Since these messages are unconfirmed, it is + possible that LSP or label state information will be + lost. This draft describes a solution which involves - negotiation between LDP peers of the intent to support extensions to LDP that facilitate recovery from failover without loss of LSPs - selection of FT survival on a per LSP/label basis - acknowledgement of LDP messages to ensure that a full @@ -276,80 +270,80 @@ - solicitation of up-to-date acknowledgement (checkpointing) of previous LDP messages to ensure the current state is flushed to disk/NVRAM, with an additional option that allows an LDP partner to request that state is flushed in both directions if graceful shutdown is required. - re-issuing lost messages after failover to ensure that LSP/label state is correctly recovered after reconnection of the LDP session. + The issues and objectives described above are equally + applicable to CR-LDP. + Other objectives of this draft are to - - offer back-compatibility with LSRs that do not implement - these proposals + - offer backward-compatibility with LSRs that do not + implement these extensions to LDP - - preserve existing protocol rules described in [RFC3212] - and [RFC3036] for handling unexpected duplicate messages and - for processing unexpected messages referring to unknown + - preserve existing protocol rules described in [RFC3036] + for handling unexpected duplicate messages and for + processing unexpected messages referring to unknown LSPs/labels - - integrate with the LSP modification function described in - [RFC3214] - - avoid full state refresh solutions (such as those present in RSVP: see [RFC2205], [RFC2961], [RFC3209] and [LDP- - RESTART]) whether they be full-time, or limited to post- + RESTART]) whether they be continual, or limited to post- failover recovery. Note that this draft concentrates on the preservation of label state for labels exchanged between a pair of adjacent LSRs when the TCP connection between those LSRs is lost. This is a requirement for Fault Tolerant operation of LSPs, but a full implementation of end-to- end protection for LSPs requires that this is combined with other techniques that are outside the scope of this draft. In particular, this draft does not attempt to describe how to modify the routing of an LSP or the resources allocated to a label or LSP, which is covered by [RFC3214]. This draft also does not address how to - provide automatic layer 2/3 protection switching for a - label or LSP, which is a separate area for study. + provide automatic layer 2 or layer 3 protection switching + for a label or LSP, which is a separate area for study. This specification does not preclude an implementation from attempting (or require it to attempt) to use the FT behavior described here to recover from a preemptive failure of a connection on a non-FT system due to, for example, a partial system crash. Note, however, that there are potential issues too numerous to list here - not least the likelihood that the same crash will immediately occur when processing the restored data. -3. Overview of LDP FT Enhancements +4. Overview of LDP FT Enhancements The LDP FT enhancements consist of the following main elements, which are described in more detail in the sections that follow. - The presence of an FT Session TLV on the LDP Initialization message indicates that an LSR supports some form of protection or recovery from session failure. A flag bit within this TLV (the S bit) indicates that the LSR supports the LDP FT enhancements on this session. Another flag (the C bit) indicates that the checkpointing procedures are to be used. - - An FT Reconnect Flag in the FT Session TLV indicates - whether an LSR has preserved FT Label state across a failure - of the TCP connection. + - An FT Reconnect Flag in the FT Session TLV (the R bit) + indicates whether an LSR has preserved FT Label state across + a failure of the TCP connection. - An FT Reconnection Timeout, exchanged on the LDP Initialization message, that indicates the maximum time peer LSRs will preserve FT Label state after a failure of the TCP connection. - An FT Protection TLV used to identify operations that affect LDP labels. All LDP messages carrying the FT Protection TLV need to be secured (e.g. to NVRAM) and ACKed to the sending LDP peer in order that the state for Sequence @@ -368,27 +362,26 @@ very large number of addresses. - The FT Protection TLV may also be used on the Keepalive message to flush acknowledgement of all previous FT operations. This enables a check-point for future recovery, either in mid-session or prior to graceful shutdown of an LDP session. This procedure may also be used to checkpoint all (that is both FT and non-FT) operations for future recovery. -3.1. Establishing an FT LDP Session +4.1. Establishing an FT LDP Session - In order that the extensions to LDP [RFC3036] and CR-LDP - [RFC3212] described in this draft can be used - successfully on an LDP session between a pair of LDP - peers, they MUST negotiate that the LDP FT enhancements - are to be used on the LDP session. + In order that the extensions to LDP [RFC3036] described + in this draft can be used successfully on an LDP session + between a pair of LDP peers, they MUST negotiate that the + LDP FT enhancements are to be used on the LDP session. This is done on the LDP Initialization message exchange using a new FT Session TLV. Presence of this TLV indicates that the peer wants to support some form of protection or recovery processing. The S bit within this TLV indicates that the peer wants to support the LDP FT enhancements on this LDP session. The C bit indicates that the peer wants to support the checkpointing functions described in this draft. The S and C bits may be set independently. @@ -404,62 +397,61 @@ MUST NOT be used during this LDP session. Use of LDP FT enhancements by a sending LDP peer MUST be interpreted by the receiving LDP peer as a serious protocol error causing the session to be terminated. An LSR MAY present different FT/non-FT behavior on different TCP connections, even if those connections are successive instantiations of the LDP session between the same LDP peers. -3.1.1 Interoperation with Non-FT LSRs +4.1.1 Interoperation with Non-FT LSRs The FT Session TLV on the LDP Initialization message carries the U-bit. If an LSR does not support any protection or recovery mechanisms , it will ignore this TLV. Since such partners also do not include the FT Session TLV, all LDP sessions to such LSRs will not use the LDP FT enhancements. The rest of this draft assumes that the LDP sessions under discussion are between LSRs that do support the LDP FT enhancements, except where explicitly stated otherwise. -3.2. TCP Connection Failure +4.2. TCP Connection Failure -3.2.1 Detecting TCP Connection Failures +4.2.1 Detecting TCP Connection Failures TCP connection failures may be detected and reported to the LDP component in a variety of ways. These should all be treated in the same way by the LDP component. - Indication from the management component that a TCP connection or underlying resource is no longer active. - Notification from a hardware management component of an interface failure. - Sockets keepalive timeout. - Sockets send failure. - New (incoming) Socket opened. - LDP protocol timeout. -3.2.2 LDP Processing after Connection Failure +4.2.2 LDP Processing after Connection Failure If the LDP FT enhancements are not in use on an LDP session, the action of the LDP peers on failure of the - TCP connection is as specified in [RFC3212] and - [RFC3036]. + TCP connection is as specified in [RFC3036]. All state information and resources associated with non- FT Labels MUST be released on the failure of the TCP connection, including deprogramming the non-FT Label from the switching hardware. This is equivalent to the behavior specified in [RFC3036]. If the LDP FT enhancements are in use on an LDP session, both LDP peers SHOULD preserve state information and resources associated with FT Labels exchanged on the LDP @@ -471,35 +463,35 @@ described in [RFC3036]. The FT Reconnection Timeout each LDP peer intends to apply to the LDP session is carried in the FT Session TLV on the LDP Initialization messages. Both LDP peers MUST use the value that corresponds to the lesser timeout interval of the two proposed timeout values from the LDP Initialization exchange, where a value of zero is treated as positive infinity. -3.3. Data Forwarding During TCP Connection Failure +4.3. Data Forwarding During TCP Connection Failure An LSR that implements the LDP FT enhancements SHOULD preserve the programming of the switching hardware across a failover. This ensures that data forwarding is unaffected by the state of the TCP connection between LSRs. It is an integral part of FT failover processing in some hardware configurations that some data packets might be lost. If data loss is not acceptable to the applications using the MPLS network, the LDP FT enhancements described in this draft SHOULD NOT be used. -3.4. FT LDP Session Reconnection +4.4. FT LDP Session Reconnection When a new TCP connection is established, the LDP peers MUST exchange LDP Initialization messages. When a new TCP connection is established after failure, the LDP peers MUST re-exchange LDP Initialization messages. If an LDP peer includes the FT Session TLV with the S bit set in the LDP Initialization message for the new instantiation of the LDP session, it MUST also set the FT Reconnect Flag according to whether it has been able to @@ -540,40 +532,40 @@ with the FT Reconnect Flag set before it sends its own Initialization message, but has retained no information about the previous version of the session, it MUST respond with an Initialization message with the FT Reconnect Flag clear. If an LDP peer receives an LDP Initialization message with the FT Reconnect Flag set in response to an Initialization message that it has sent with the FT Reconnect Flag clear it MUST act as if no state was retained by either peer on the session. -3.5. Operations on FT Labels +4.5. Operations on FT Labels Label operations on Sequence Numbered FT Labels are made Fault Tolerant by providing acknowledgement of all LDP messages that affect Sequence Numbered FT Labels. Acknowledgements are achieved by means of sequence numbers on these LDP messages. The message exchanges used to achieve acknowledgement of label operations and the procedures used to complete interrupted label operations are detailed in the section "FT Operations". Using these acknowledgements and procedures, it is not necessary for LDP peers to perform a complete re- synchronization of state for all Sequence Numbered FT Labels, either on re-connection of the LDP session between the LDP peers or on a timed basis. -3.6. Check-Pointing +4.6. Check-Pointing Check-pointing is a useful feature that allows nodes to reduce the amount of processing that they need to do to acknowledge LDP messages. The C bit in the FT Session TLV is used to indicate that checkpointing is supported. Under the normal operation on Sequence Numbered FT Labels, acknowledgments may be deferred during normal processing and only sent periodically. Check-pointing may be used to flush acknowledgement from a peer by @@ -585,38 +577,38 @@ If the S bit is not agreed upon, checkpointing may still be used. In this case it is used to acknowledge all messages exchanged between the peers, and all labels are Checkpointable FT Labels. This offers an approach where acknowledgements need not be sent to every message or even frequently, but are only sent as check-points in response to requests carried on Keepalive messages. Such an approach may be considered optimal in systems that do not show a high degree of - change over time (such as targeted LDP session or CR-LDP - systems) and that are prepared to risk loss of state for - the most recent LDP exchanges. More dynamic systems - (such as LDP discovery sessions) are more likely to want - to acknowledge state changes more frequently so that the + change over time (such as targeted LDP sessions) and that + are prepared to risk loss of state for the most recent + LDP exchanges. More dynamic systems (such as LDP + discovery sessions) are more likely to want to + acknowledge state changes more frequently so that the maximum amount of state can be preserved over a failure. Note that an important consideration of this draft is that nodes acknowledging messages on a one-for-one basis, nodes deferring acknowledgements, and nodes relying on check-pointing should all interoperate seamlessly and without protocol negotiation beyond session initialization. Further discussion of this feature is provided in the section "FT Operations". -3.6.1 Graceful Termination +4.6.1 Graceful Termination A feature that builds on check-pointing is graceful termination. In some cases, such as controlled failover or software upgrade, it is possible for a node to know in advance that it is going to terminate its session with a peer. In these cases the node that intends terminating the session can flush acknowledgement using a check-point @@ -628,21 +620,21 @@ This, however, only provides for acknowledgement in one direction, and the node that is terminating also requires to know that it has secured all state sent by its peer. This is achieved by a three-way hand shake of the check- point which is requested by an additional TLV (the Cork TLV) in the Keepalive message. Further discussion of this feature is provided in the section "FT Operations". -3.7. Label Space Depletion and Replenishment +4.7. Label Space Depletion and Replenishment When an LDP peer is unable to satisfy a Label Request message because it has no more available labels, it sends a Notification message carrying the status code 'No label resources'. This warns the requesting LDP peer that subsequent Label Request messages are also likely to fail for the same reason. This message does not need to be acknowledged for FT purposes since Label Request messages sent after session recovery will receive the same response. However, the LDP peer that receives a 'No @@ -667,34 +659,69 @@ receiver of 'Label resources available' Notification message may choose to acknowledge the message without actually saving any state. This is an implementation choice made possible by making the FT parameters on the Notification message optional. Implementations will interoperate fully if they take opposite approaches, but additional LDP messages may be sent unnecessarily on session recovery. -4. FT Operations +4.8. Tunneled LSPs + + The procedures described in this document can be applied + to LSPs that are tunneled and to LSPs that are carried by + tunnels. Recall that tunneled LSPs are managed by a + single LDP session that runs end to end while the tunnel + is managed by a different LDP session for each hop along + the path. Nevertheless, a break in one of the sessions + that manages the tunnel is likely to correspond with a + break in the session that manages the tunneled LSP. This + is certainly the case when the LDP exchanges share a + failed link, but need not be the case if the LDP messages + have been routed along a path that is different from that + of the tunnel, or if the failure in the tunnel is caused + by an LDP software failure at a transit LSR. + + In order that the forwarding path of a tunneled LSP be + preserved, the forwarding path of the tunnel itself must + be preserved. This means that the tunnel must not be torn + down if there is any session failure along its path. To + achieve this the label exchanges between each pair of LDP + peers along the path of the tunnel must use one of the + procedures in this document or in [LDP-RESTART]. + + It is perfectly acceptable to mix the restart procedures + used for the tunnel and the tunneled LSP. For example, + the tunnel could be set up using just check-pointing + because it is a stable LSP, but the tunneled LSPs might + use full FT procedures so that they can recover active + state. + + Lastly, it is permissible to carry tunneled LSPs that do + not have FT protection on an LSP that does have FT + protection. + +5. FT Operations Once an FT LDP session has been established, using the S bit in the FT Session TLV on the Session Initialization message as described in the section "Establishing an FT LDP Session", both LDP peers MUST apply the procedures described in this section for FT LDP message exchanges. If the LDP session has been negotiated to not use the LDP FT enhancements, these procedures MUST NOT be used. -4.1. FT LDP Messages +5.1. FT LDP Messages -4.1.1 Sequence Numbered FT Label Messages +5.1.1 Sequence Numbered FT Label Messages A label is identified as being a Sequence Numbered FT Label if the initial Label Request or Label Mapping message relating to that label carries the FT Protection TLV. It is a valid implementation option to flag all labels as Sequence Numbered FT Labels. Indeed this may be a preferred option for implementations wishing to use Keepalive messages carrying the FT Protection TLV to @@ -704,63 +731,63 @@ If a label is a Sequence Numbered FT Label, all LDP messages affecting that label MUST carry the FT Protection TLV in order that the state of the label can be recovered after a failure of the LDP session. A valid option is for no labels to be Sequence Numbered FT Labels. In this case checkpointing using the Keepalive message applies to all messages exchanged on the session. -4.1.1.1 Scope of FT Labels +5.1.1.1 Scope of FT Labels The scope of the FT/non-FT status of a label is limited to the LDP message exchanges between a pair of LDP peers. In Ordered Control, when the message is forwarded downstream or upstream, the TLV may be present or absent according to the requirements of the LSR sending the message. If a platform-wide label space is used for FT Labels, an FT Label value MUST NOT be reused until all LDP FT peers to which the label was passed have acknowledged the withdrawal of the FT Label, either by an explicit LABEL WITHDRAW/LABEL RELEASE exchange or implicitly if the LDP session is reconnected after failure but without the FT Reconnect Flag set. In the event that a session is not re-established within the Reconnection Timeout, a label MAY become available for re-use if it is not still in use on some other session. -4.1.2 FT Address Messages +5.1.2 FT Address Messages If an LDP session uses the LDP FT enhancements, both LDP peers MUST secure Address and Address Withdraw messages using FT Operation ACKs, as described below. This avoids any ambiguity over whether an Address is still valid after the LDP session is reconnected. If an LSR determines that an Address message that it sent on a previous instantiation of a recovered LDP session is no longer valid, it MUST explicitly issue an Address Withdraw for that address when the session is reconnected. If the FT Reconnect Flag is not set by both LDP peers on reconnection of an LDP session (i.e. state has not been preserved), both LDP peers MUST consider all Addresses to have been withdrawn. The LDP peers SHOULD issue new Address messages for all their valid addresses as specified in [RFC3036]. -4.1.3 Label Resources Available Notifications +5.1.3 Label Resources Available Notifications In LDP, it is possible that a downstream LSR may not have labels available to respond to a Label Request. In this case, as specified in RFC3036, the downstream LSR must respond with a Notification - No Label Resources message. The upstream LSR then suspends asking for new labels until it receives a Notification - Label Resources Available message from the downstream LSR. When the FT extensions are used on a session, @@ -796,30 +823,30 @@ its peer thinks the LSR's resource state is, because the Notification may or may not have been delivered. Such an implementation MUST begin recovered sessions by sending an additional Notification - Label Resources Available to reset its peer. - The upstream node may choose not to secure information about its peer's resource state. It would acknowledge a Notification - Label Resources Available, but would not save the information. Such an implementation MUST assume that - its peer's resource satte has been reset to Label Resources + its peer's resource state has been reset to Label Resources Available when the session is re-established. If the FT Reconnect Flag is not set by both LDP peers on reconnection of an LDP session (i.e. state has not been preserved), both LDP peers MUST consider the label availability state to have been reset as if the session had been set up for the first time. -4.2. FT Operation ACKs +5.2. FT Operation ACKs Handshaking of FT LDP messages is achieved by use of ACKs. Correlation between the original message and the ACK is by means of the FT Sequence Number contained in the FT Protection TLV, and passed back in the FT ACK TLV. The FT ACK TLV may be carried on any LDP message that is sent on the TCP connection between LDP peers. An LDP peer maintains a separate FT sequence number for each LDP session it participates in. The FT Sequence @@ -850,29 +877,29 @@ the same sequence number) are acceptable. If an LDP peer discovers that its sequence number space for a specific session is full of un-acknowledged sequence numbers (because its partner on the session has not acknowledged them in a timely way) it cannot allocate a new sequence number for any further FT LPD message. It SHOULD send a Notification message with the status code "FT Seq Numbers Exhausted". -4.3. Preservation of FT State +5.3. Preservation of FT State If the LDP FT enhancements are in use on an LDP session, each LDP peer SHOULD NOT release the state information and resources associated with FT Labels exchanged on that LDP session when the TCP connection fails. This is - contrary to [RFC3212] and [RFC3036], but allows label - operations on FT Labels to be completed after re- - connection of the TCP connection. + contrary to [RFC3036], but allows label operations on FT + Labels to be completed after re-connection of the TCP + connection. Both LDP peers on an LDP session that is using the LDP FT enhancements SHOULD preserve the state information and resources they hold for that LDP session as described below. - An upstream LDP peer SHOULD release the resources (in particular bandwidth) associated with a Sequence Numbered FT Label when it initiates a Label Release or Label Abort message for the label. The upstream LDP peer MUST preserve @@ -912,50 +939,50 @@ operations (as described in section 4.4.1) during a TCP failure. That is, the peer is not required to wait for the duration of the FT Reconnection Timeout before releasing state; the timeout provides an upper limit on the persistence of state. However, in the event that a peer releases state before the expiration of the Reconnection Timeout it MUST NOT re-use any label that was in use on the session until the Reconnection Timeout has expired. - When an LSR receives a Status TLV with the E-bit set in + - When an LSR receives a Status TLV with the E-bit set in the status code, which causes it to close the TCP connection, the LSR MUST release all state information and resources associated with the session. This behavior is mandated because it is impossible for the LSR to predict the precise state and future behavior of the partner LSR that set the E-bit without knowledge of the implementation of that partner LSR. Note that the "Temporary Shutdown" status code does not have the E-bit set, and MAY be used during maintenance or upgrade operations to indicate that the LSR intends to preserve state across a closure and re-establishment of the TCP session. - If an LSR determines that it must release state for any single FT Label during a failure of the TCP connection on which that label was exchanged, it MUST release all state for all labels on the LDP session. The release of state information and resources associated - with non-FT labels is as described in [RFC3212] and - [RFC3036]. + with non-FT labels is as described in [RFC3036]. Note that a Label Release and the acknowledgement to a Label Withdraw may be received by a downstream LSR in any order. The downstream LSR MAY release its resources on receipt of the first message and MUST release its resources on receipt of the second message. -4.4. FT Procedure After TCP Failure +5.4. FT Procedure After TCP Failure When an LSR discovers or is notified of a TCP connection failure it SHOULD start an FT Reconnection Timer to allow a period for re-connection of the TCP connection between the LDP peers. The RECOMMENDED default value for this timer is 5 seconds. During this time, failure must be detected and reported, new hardware may need to be activated, software state must be audited, and a new TCP session must be set @@ -986,21 +1013,21 @@ issue LDP operations that were interrupted by (that is, un-acknowledged as a result of) the TCP connection failure. This procedure is described in section "FT Procedure After TCP Re-connection". The Hold Timer for an FT LDP Session (see [RFC3036] section 2.5.5) SHOULD be ignored while the FT Reconnection Timer is running. The hold timer SHOULD be restarted when the TCP connection is re-established. -4.4.1 FT LDP Operations During TCP Failure +5.4.1 FT LDP Operations During TCP Failure When the LDP FT enhancements are in use for an LDP session, it is possible that an LSR may determine that it needs to send an LDP message to an LDP peer but that the TCP connection to that peer is currently down. These label operations affect the state of FT Labels preserved for the failed TCP connection, so it is important that the state changes are passed to the LDP peer when the TCP connection is restored. @@ -1040,66 +1067,69 @@ It is RECOMMENDED that Label Request operations for new FT Labels are not pended awaiting the re-establishment of TCP connection that is awaiting recovery at the time the LSR determines that it needs to issue the Label Request message. Instead, such Label Request operations SHOULD be failed and, if necessary, a notification message containing the "No LDP Session" status code sent upstream. Label Requests for new non-FT Labels MUST be rejected - during TCP connection failure, as specified in [RFC3212] - and [RFC3036]. + during TCP connection failure, as specified in [RFC3036]. -4.5. FT Procedure After TCP Re-connection +5.5. FT Procedure After TCP Re-connection The FT operation handshaking described above means that all state changes for Sequence Numbered FT Labels and - Address messages are confirmed or reproducible at each LSR. + Address messages are confirmed or reproducible at each + LSR. If the TCP connection between LDP peers fails but is re- connected within the FT Reconnection Timeout, and both LSRs have indicated they will be re-establishing the previous LDP session, both LDP peers on the connection MUST complete any label operations for Sequence Numbered FT Labels that were interrupted by the failure and re- connection of the TCP connection. The procedures for FT Reconnection Timeout MAY have been invoked as a result of either LDP peer being unable (or unwilling) to pend operations which occurred during the TCP Failure (as described in section 4.4.1). - If, for any reason, an LSR has been unable to pend operations - with respect to an LDP peer, as described in section 4.4.1, the - LSR MUST set the FT Reconnect Flag to 0 on re-connection to that - LDP peer indicating that no FT state has been preserved. + If, for any reason, an LSR has been unable to pend + operations with respect to an LDP peer, as described in + section 4.4.1, the LSR MUST set the FT Reconnect Flag to + 0 on re-connection to that LDP peer indicating that no FT + state has been preserved. - Label operations are completed using the procedure described below. + Label operations are completed using the procedure + described below. -4.5.1 Re-Issuing FT Messages +5.5.1 Re-Issuing FT Messages On restoration of the TCP connection between LDP peers, any LDP messages for Sequence Numbered FT Labels that were lost because of the TCP connection failure are re- issued. The LDP peer that receives a re-issued message processes the message as if received for the first time. "Net-zero" combinations of messages need not be re-issued after re-establishment of the TCP connection between LDP peers. This leads to the following rules for re-issuing messages that are not ACKed by the LDP peer on the LDP Initialization message exchange after re-connection of the TCP session. - - A Label Request message MUST be re-issued unless a Label Abort - would be re-issued for the same Sequence Numbered FT Label. + - A Label Request message MUST be re-issued unless a Label + Abort would be re-issued for the same Sequence Numbered FT + Label. - A Label Mapping message MUST be re-issued unless a Label Withdraw message would be re-issued for the same Sequence Numbered FT Label. - All other messages on the LDP session that carried the FT Protection TLV MUST be re-issued if an acknowledgement had not previously been received. Any FT Label operations that were pended (see section @@ -1113,32 +1143,21 @@ messages prior to the re-establishment of the TCP connection in order to optimize the use of queue resources. Messages that were sent to the LDP peer before the TCP connection failure, or pended messages that are paired with them, MUST NOT be subject to such optimization until an FT ACK TLV is received from the LDP peer. This ACK allows the LSR to identify which messages were received by the LDP peer prior to the TCP connection failure. -4.5.2 Interaction with CR-LDP LSP Modification - - Re-issuing LDP messages for FT operation is orthogonal to - the use of duplicate messages marked with the Modify - ActFlg, as specified in [RFC3214]. Each time an LSR uses - the modification procedure for an FT LSP to issue a new - Label Request message, the FT label operation procedures - MUST be separately applied to the new Label Request - message to the same degree that they were applied to the - original Label Request. - -5. Checkpointing Procedures +6. Checkpointing Procedures Checkpointing can be selected independently from the FT procedures described above by using the C bit in the FT Session TLV on the Session Initialization message. Note, however, that checkpointing is an integral part of the FT procedures. Setting the S and the C bit will achieve the same function as setting just the S bit. If the C bit is set, but the S bit is not set, no label is aSequence Numbered FT Label. Instead, all labels are @@ -1153,21 +1172,21 @@ received messages and checkpoint requests to ensure that checkpoint acknowledgements are secured. If the S and C bits are both set, or only the S bit is set, checkpointing applies only to Sequence Numbered FT Labels and to address messages. The set of all messages that are checkpointed in this way is called the Checkpointable Messages. -5.1. Checkpointing with the Keepalive Message +6.1 Checkpointing with the Keepalive Message If an LSR receives a FT Protection TLV on a Keepalive message, this is a request to flush the acknowledgements for all previously received Checkpointable Messages on the session. As soon as the LSR has completed securing the Checkpointable Messages (or state changes consequent on those messages) received before the Keepalive, it MUST send an acknowledgement to the sequence number of the @@ -1177,127 +1196,127 @@ acknowledgements have been stored up, this may be immediately on receipt of the Keepalive. An example message flow showing this use of the Keepalive message to perform a periodic check-point of state is shown in section 8. An example message flow showing the use of checkpointing without the FT procedures is shown in section 8. -5.2. Quiesce and Keepalive +6.2 Quiesce and Keepalive If the Keepalive Message also contains the FT Cork TLV, this indicates that the peer LSR wishes to quiesce the session prior to a graceful restart. It is RECOMMENDED that on receiving a Keepalive with the FT CORK TLV, an LSR should cease to send any further label or address related messages on the session until it has been disconnected and reconnected, other than any messages generated while processing and securing any previously unacknowledged messages received from the peer requesting the quiesce. It should also attempt to complete this processing and return a Keepalive with the FT ACK TLV as soon as possible in order to allow the session to be quiesced. An example message flow showing this use of the FT Cork TLV to achieves three-way handshake of state synchronization between two LDP peers is given in section 8. -6. Changes to Existing Messages +7. Changes to Existing Messages -6.1. LDP Initialization Message +7.1. LDP Initialization Message The LDP FT enhancements add the following optional parameters to a LDP Initialization message Optional Parameter Length Value FT Session TLV 4 See below FT ACK TLV 4 See below The encoding for these TLVs is found in Section "New Fields and Values". - FT Session + FT Session TLV If present, specifies the FT behavior of the LDP session. FT ACK TLV If present, specifies the last FT message that the sending LDP peer was able to secure prior to the failure of the previous instantiation of the LDP session. This TLV is only present if the FT Reconnect flag is set in the FT Session TLV, in which case this TLV MUST be present. -6.2. LDP Keepalive Messages +7.2. LDP Keepalive Messages The LDP FT enhancements add the following optional parameters to a LDP Keepalive message Optional Parameter Length Value FT Protection TLV 4 See below FT Cork TLV 0 See below FT ACK TLV 4 See below The encoding for these TLVs is found in Section "New Fields and Values". - FT Protection + FT Protection TLV If present, specifies FT Sequence Number for the LDP message. When present on a Keepalive message, this indicates a solicited flush of the acknowledgements to all previous LDP messages containing sequence numbers and issued by the sender of the Keepalive on the same session. - FT Cork + FT Cork TLV Indicates that the remote LSR wishes to quiesce the LDP session. See section 5 for the recommended action in such cases. FT ACK TLV If present, specifies the most recent FT message that the sending LDP peer has been able to secure. -6.3. All Other LDP Session Messages +7.3. All Other LDP Session Messages The LDP FT enhancements add the following optional parameters to all other message types that flow on an LDP session after the LDP Initialization message Optional Parameter Length Value FT Protection TLV 4 See below FT ACK TLV 4 See below The encoding for these TLVs is found in the section "New Fields and Values". - FT Protection + FT Protection TLV If present, specifies FT Sequence Number for the LDP message. - FT ACK + FT ACK TLV If present, identifies the most recent FT LDP message ACKed by the sending LDP peer. -7. New Fields and Values +8. New Fields and Values -7.1. Status Codes +8.1. Status Codes The following new status codes are defined to indicate various conditions specific to the LDP FT enhancements. These status codes are carried in the Status TLV of a Notification message. The "E" column is the required setting of the Status Code E-bit; the "Status Data" column is the value of the 30- bit Status Data field in the Status Code TLV. @@ -1324,21 +1344,21 @@ changed Unexpected FT Cork TLV 1 0x000000xx The Temporary Shutdown status code SHOULD be used in place of the Shutdown status code (which has the E-bit set) if the LSR that is shutting down wishes to inform its LDP peer that it expects to be able to preserve FT Label state and to return to service before the FT Reconnection Timer expires. -7.2. FT Session TLV +8.2. FT Session TLV LDP peers can negotiate whether the LDP session between them supports FT extensions by using a new OPTIONAL parameter, the FT Session TLV, on LDP Initialization Messages. The FT Session TLV is encoded as follows. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 @@ -1373,31 +1390,29 @@ session between the same LDP peers, and set to 0 otherwise. See the section "FT LDP Session Reconnection" for details of how this flag is used. If the FT Reconnect Flag is set, the sending LSR MUST include an FT ACK TLV on the LDP Initialization message. S: Save State Flag. - Set to 1 if the use of the FT Protection TLV is supported on messages other than the KeepAlive message used for chekpointing (see the - C bit). I.e., the S bit indicates hat - some label on the session may be a - Sequence Numbered FT Label. + C bit). I.e., the S bit indicates + that some label on the session may be + a Sequence Numbered FT Label. A: All-Label Protection Required - Set to 1 if all labels on the session MUST be treated as Sequence Numbered FT Labels. This removes from a node the option of treating some labels as FT Labels and some labels as non-FT Labels. Passing this information may be considered helpful to a peer since it may allow it to make optimizations in @@ -1438,31 +1451,31 @@ All other bits in this field are currently reserved and SHOULD be set to zero on transmission and ignored on receipt. The following table summarizes the settings of these bits. S A C L Comments ========================= 0 x 0 0 Invalid - 0 x 0 1 See [LDP-RESTART] + 0 0 0 1 See [LDP-RESTART] + 0 1 0 1 Invalid 0 x 1 0 Checkpointing of all labels 0 x 1 1 Invalid 1 0 0 0 Full FT on selected labels 1 1 0 0 Full FT on all labels 1 x 0 1 Invalid 1 x 1 0 Same as (S=1,A=x,C=0,L=0) 1 x 1 1 Invalid. FT Reconnection Timeout - If the S bit or C bit in the FT Flags field is set this indicates the period of time the sending LSR will preserve state and resources for FT Labels exchanged on the previous instantiation of an FT LDP session that has currently failed. The timeout is encoded as a 32-bit unsigned integer number of milliseconds. A value of zero in this field means that @@ -1470,26 +1483,25 @@ resources indefinitely. See section 4.4 for details of how this field is used. If the L bit is set to 1 in the FT Flags field, the meaning of this field is defined in [LDP-RESTART]. Recovery Time - The Recovery Time only has meaning if the L bit is set in the FT Flags. The meaning is defined in [LDP-RESTART]. -7.3. FT Protection TLV +8.3. FT Protection TLV LDP peers use the FT Protection TLV to indicate that an LDP message contains an FT label operation. The FT Protection TLV MUST NOT be used in messages flowing on an LDP session that does not support the LDP FT enhancements. Its presence in such messages SHALL be treated as a protocol error by the receiving LDP peer which SHOULD send a Notification message with the 'Unexpected TLV Session Not FT' status code. LSRs that @@ -1527,25 +1539,24 @@ If a label is not to be a Sequence Numbered FT Label, then the Protection TLV MUST NOT be present on any of these messages that relate to the label. The presence of the FT TLV on a message relating to a non-FT Label SHALL be treated as a protocol error by the receiving LDP peer which SHOULD send a notification message with the 'Unexpected TLV Label Not FT' status code. Where a Label Withdraw or Label Release message contains only a FEC TLV and does not identify a single specific - label, the FT TLV should be included in the message if - any label affected by the message is a Sequence Numbered - FT Label. If there is any doubt as to whether an FT TLV - should be present, it is RECOMMENDED that the sender add - the TLV. + label, the FT TLV should be included in the message if any + label affected by the message is a Sequence Numbered FT + Label. If there is any doubt as to whether an FT TLVshould + be present, it is RECOMMENDED that the sender add the TLV. When an LDP peer receives a Label Withdraw Message or Label Release message that contains only a FEC, it SHALL accept the FT TLV if it is present regardless of the FT status of the labels which it affects. If an LDP session is an FT session as determined by the presence of the FT Session TLV with the S bit set on the LDP Initialization messages, the FT Protection TLV MUST be present on all Address messages on the session. @@ -1562,30 +1573,28 @@ 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0| FT Protection (0x0203) | Length (= 4) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FT Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ FT Sequence Number - - The sequence number for this Sequence - Numbered FT Label operation. The sequence - number is encoded as a 32-bit unsigned - integer. The initial value for this field - on a new LDP session is 0x00000001 and is - incremented by one for each FT LDP message - issued by the sending LSR on this LDP - session. This field may wrap from - 0xFFFFFFFF to 0x00000001. + The sequence number for this Sequence Numbered + FT Label operation. The sequence number is + encoded as a 32-bit unsigned integer. The + initial value for this field on a new LDP + session is 0x00000001 and is incremented by + one for each FT LDP message issued by the + sending LSR on this LDP session. This field + may wrap from 0xFFFFFFFF to 0x00000001. This field MUST be reset to 0x00000001 if either LDP peer does not set the FT Reconnect Flag on re-establishment of the TCP connection. See the section "FT Operation Acks" for details of how this field is used. The special use of 0x00000000 is discussed @@ -1608,21 +1617,21 @@ TLV affecting a label that it believes is a Sequence Numbered FT Label, it SHOULD send a Notification message to its LDP peer containing the "Missing FT Protection TLV" status code. If an LSR receives an FT Protection TLV containing a zero FT Sequence Number, it SHOULD send a Notification message to its LDP peer containing the "Zero FT Seqnum" status code. -7.4. FT ACK TLV +8.4. FT ACK TLV LDP peers use the FT ACK TLV to acknowledge FT Label operations. The FT ACK TLV MUST NOT be used in messages flowing on an LDP session that does not support the LDP FT enhancements. Its presence on such messages SHALL be treated as a protocol error by the receiving LDP peer. The FT ACK TLV MAY be present on any LDP message @@ -1634,22 +1643,22 @@ The FT ACK TLV is encoded as follows. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0| FT ACK (0x0504) | Length (= 4) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FT ACK Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - FT ACK Sequence Number + FT ACK Sequence Number The sequence number for the most recent FT label message that the sending LDP peer has received from the receiving LDP peer and secured against failure of the LDP session. It is not necessary for the sending peer to have fully processed the message before ACKing it. For example, an LSR MAY ACK a Label Request message as soon as it has securely recorded the message, without waiting until it can send the Label Mapping @@ -1679,37 +1688,38 @@ Notification message to its LDP peer containing the "Missing FT Protection TLV" status code. If an LSR receives an FT ACK TLV that contains an FT ACK Sequence Number that is less than the previously received FT ACK Sequence Number (remembering to take account of wrapping), it SHOULD send a Notification message to its LDP peer containing the "FT ACK Sequence Error" status code. -7.5. FT Cork TLV +8.5. FT Cork TLV LDP peers use the FT Cork TLV on FT Keepalive messages to indicate that they wish to quiesce the LDP session prior to a controlled shutdown and restart, for example during control-plane software upgrade. The FT Cork TLV is encoded as follows. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0| FT Cork (0x0505) | Length (= 0) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - On receipt of a Keepalive message with the FT Cork TLV and the FT - Protection TLV, an LSR SHOULD perform the following actions. + On receipt of a Keepalive message with the FT Cork TLV + and the FT Protection TLV, an LSR SHOULD perform the + following actions - Process and secure any messages from the peer LSR that have sequence numbers less than (accounting for wrap) that contained in the FT Protection TLV on the Keepalive message. - Send a Keepalive message back to the peer containing the FT Cork TLV and the FT ACK TLV specifying the FT ACK sequence number equal to that in the original Keepalive message (i.e. ACKing all messages up to that point). @@ -1735,21 +1745,21 @@ An example such 3-way handshake for controlled shutdown is given in section 8. If an LSR receives a message that should not carry the FT Cork TLV, or if the FT Cork TLV is used on a Keepalive message without one of the FT Protection or FT ACK TLVs present, , it SHOULD send a Notification message to its LDP peer containing the "Unexpected FR Cork TLV" status code. -8. Example Use +9. Example Use Consider two LDP peers, P1 and P2, implementing LDP over a TCP connection that connects them, and the message flow shown below. The parameters shown on each message shown below are as follows: message (label, senders FT sequence number, FT ACK number) @@ -1757,21 +1767,21 @@ A "-" for FT ACK number means that the FT ACK TLV is not included on that message. "n/a" means that the parameter in question is not applicable to that type of message. In the diagrams below, time flows from top to bottom. The relative position of each message shows when it is transmitted. See the notes for a description of when each message is received, secured for FT or processed. -8.1. Session Failure and Recovery - FT Procedures +9.1. Session Failure and Recovery - FT Procedures notes P1 P2 ===== == == (1) Label Request(L1,27,-) ---------------------------> Label Request(L2,28,-) ---------------------------> (2) Label Request(L3,93,27) <--------------------------- (3) Label Request(L1,123,-) @@ -1856,21 +1866,21 @@ to the failure. (13) From the LDP Init exchange, P1 determines that it needs to re-issue the Label request for L4. (14) Similarly, P2 determines that it needs to re-issue the Label Mapping for L2 and the Label Abort. (15) P2 issues the queued Label Withdraw to P1. -8.2. Use of Check-Pointing With FT Procedures +9.2. Use of Check-Pointing With FT Procedures notes P1 P2 ===== == == (1) Label Request(L1,27,-) ---------------------------> Label Request(L2,28,-) ---------------------------> (2) Label Request(L3,93,-) <--------------------------- (3) Label Request(L1,123,-) @@ -1908,30 +1918,31 @@ (8) P1 wishes to synchronize state with P2. It sends a Keepalive message containing a FT Protection TLV with sequence number 31. Since it is not interested in P2's perception of the state that it has stored, it does not include an FT ACK TLV. (9) P2 responds at once with a Keepalive acknowledging the sequence number on the received Keepalive. This tells P1 that P2 has preserved all state/messages previously received on this session. + (10) P3 wishes to synchronize state with P2. It sends a Keepalive message containing a FT Protection TLV with sequence number 59. P3 also takes this opportunity to get up to date with its acknowledgements to P2 by including an FT ACK TLV acknowledging up to sequence number 124. (11) P2 responds at once with a Keepalive acknowledging the sequence number on the received Keepalive. -8.3. Temporary Shutdown With FT Procedures +9.3. Temporary Shutdown With FT Procedures notes P1 P2 ===== == == (1) Label Request(L1,27,-) ---------------------------> Label Request(L2,28,-) ---------------------------> (2) Label Request(L3,93,27) <--------------------------- (3) Label Request(L1,123,-) @@ -1981,21 +1992,21 @@ (10) P1 needs to upgrade the software or hardware that it is running. It issues a Notification message to terminate the LDP session, but sets the status code as 'Temporary shutdown' to inform P2 that this is not a fatal error, and P2 should maintain FT state. The TCP connection may also fail during the period that the LDP session is down (in which case it will need to be re-established), but it is also possible that the TCP connection will be preserved. -8.4. Temporary Shutdown With FT Procedures and Check-Pointing +9.4. Temporary Shutdown With FT Procedures and Check-Pointing notes P1 P2 ===== == == (1) Label Request(L1,27,-) ---------------------------> Label Request(L2,28,-) ---------------------------> (2) Label Request(L3,93,-) <--------------------------- Label Request(L1,123,-) @@ -2051,25 +2062,25 @@ The Label abort at (7) crosses with this Keepalive, so at (8) P2 sends a Keepalive that acknowledges all messages received so far, but also including the FT Protection and FT Cork TLVs to indicate that there are still messages outstanding to be acknowledged. P1 is then able to complete the 3-way handshake at (9) and close the TCP session at (10). Upon recovery at (11) there are no messages to be re-sent - because the Keepalives flushed the acknowledgements. The + because the KeepAlives flushed the acknowledgements. The only messages sent after recovery is the Label Withdraw that was pended during the TCP session failure. -8.5. Checkpointing Without FT Procedures +9.5. Checkpointing Without FT Procedures notes P1 P2 ===== == == (1) Label Request(L1) ---------------------------> (2) Label Request(L2) <--------------------------- Label Request(L1) --------------------------> Label Mapping(L1) @@ -2121,42 +2132,42 @@ (7) The subsequent (un-acknowledged) label distribution completes. (8) The session fails and is restarted. Initialization messages confirm the sequence numbers of the secured checkpoints. (9) P1 recommences the unacknowledged label distribution request. (10) P2 recommences an unacknowledged label distribution request. -8.6. Graceful Shutdown With Checkpointing But No FT Procedures +9.6. Graceful Shutdown With Checkpointing But No FT Procedures notes P1 P2 ===== == == (1) Label Request(L1) ---------------------------> (2) Label Request(L2) <--------------------------- Label Request(L1) --------------------------> Label Mapping(L1) <-------------------------- (3) Label Mapping(L1) <--------------------------- - (4) Keepalive(n/a,12,23) * With Cork TLV + (4) Keepalive(n/a,12,23) * With Cork TLV * ---------------------------> (5) : : : - (6) Keepalive(n/a,24,12) * With Cork TLV + (6) Keepalive(n/a,24,12) * With Cork TLV * <--------------------------- - (7) Keepalive(n/a,-,24) * With Cork TLV + (7) Keepalive(n/a,-,24) * With Cork TLV * ---------------------------> (8) Notification(Temporary shutdown) ---------------------------> ===== TCP Session failure ===== : : : ===== TCP Session restored ===== (9) LDP Init(n/a,n/a,24) ---------------------------> @@ -2191,25 +2202,24 @@ (8) The session is gracefully shut down. (9) The session recovers and the peers exchange the sequence numbers of the last secured checkpoints. (10) P1 starts a new label distribution request. (11) P1 continues processing a previously received label distribution request. -9. Security Considerations +10. Security Considerations The LDP FT enhancements inherit similar security - considerations to those discussed in [RFC3212] and - [RFC3036]. + considerations to those discussed in [RFC3036]. The LDP FT enhancements allow the re-establishment of a TCP connection between LDP peers without a full re- exchange of the attributes of established labels, which renders LSRs that implement the extensions specified in this draft vulnerable to additional denial-of-service attacks as follows: - An intruder may impersonate an LDP peer in order to force a failure and reconnection of the TCP connection, but where @@ -2223,34 +2233,34 @@ - An intruder could intercept the traffic between LDP peers and override the setting of the FT Label Flag to be set to 0 for all labels. All of these attacks may be countered by use of an authentication scheme between LDP peers, such as the MD5- based scheme outlined in [RFC3036]. Alternative authentication schemes for LDP peers are outside the scope of this draft, but could be deployed to - provide enhanced security to implementations of LDP, CR- - LDP and the LDP FT enhancements. + provide enhanced security to implementations of LDP and + the LDP FT enhancements. - As with LDP and CR-LDP, a security issue may exist if an - LDP implementation continues to use labels after - expiration of the session that first caused them to be - used. This may arise if the upstream LSR detects the - session failure after the downstream LSR has released and - re-used the label. The problem is most obvious with the - platform-wide label space and could result in mis-routing - of data to other than intended destinations and it is - conceivable that these behaviors may be deliberately - exploited to either obtain services without authorization - or to deny services to others. + As with LDP, a security issue may exist if an LDP + implementation continues to use labels after expiration + of the session that first caused them to be used. This + may arise if the upstream LSR detects the session failure + after the downstream LSR has released and re-used the + label. The problem is most obvious with the platform- + wide label space and could result in mis-routing of data + to other than intended destinations and it is conceivable + that these behaviors may be deliberately exploited to + either obtain services without authorization or to deny + services to others. In this draft, the validity of the session may be extended by the FT Reconnection Timeout, and the session may be re-established in this period. After the expiry of the Reconnection Timeout the session must be considered to have failed and the same security issue applies as described above. However, the downstream LSR may declare the session as failed before the expiration of its Reconnection Timeout. @@ -2258,39 +2268,39 @@ might reallocate the label while the upstream LSR continues to transmit data using the old usage of the label. To reduce this issue, this draft requires that labels are not re-used until the Reconnection Timeout has expired. A further issue might apply if labels were re-used prior to the expiration of the FT Reconnection Timeout, but this is forbidden by this draft. -10. Implementation Notes +11. Implementation Notes -10.1. FT Recovery Support on Non-FT LSRs +11.1. FT Recovery Support on Non-FT LSRs In order to take full advantage of the FT capabilities of LSRs in the network, it may be that an LSR that does not itself contain the ability to recover from local hardware or software faults still needs to support the LDP FT enhancements described in this draft. Consider an LSR, P1, that is an LDP peer of a fully Fault Tolerant LSR, P2. If P2 experiences a fault in the hardware or software that serves an LDP session between P1 and P2, it may fail the TCP connection between the peers. When the connection is recovered, the LSPs/labels between P1 and P2 can only be recovered if both LSRs were applying the FT recovery procedures to the LDP session. -10.2. ACK generation logic +11.2. ACK generation logic FT ACKs SHOULD be returned to the sending LSR as soon as is practicable in order to avoid building up a large quantity of unacknowledged state changes at the LSR. However, immediate one-for-one acknowledgements would waste bandwidth unnecessarily. A possible implementation strategy for sending ACKs to FT LDP messages is as follows: @@ -2301,78 +2311,78 @@ FT ACK TLV listing Sr - Optionally, the LSR may attach an FT ACK TLV to any other LDP message sent between Keepalive messages if, for example, Sr has increased by more than a threshold value since the last ACK sent. This implementation combines the bandwidth benefits of accumulating ACKs while still providing timely ACKs. -10.2.1 Ack Generation Logic When Using Check-Pointing +11.2.1 Ack Generation Logic When Using Check-Pointing If check-pointing is in use, the LSRs need not be concerned to send ACKs in such a timely manner. Check-points are solicitations for acknowledgement conveyed as a sequence number in an FT Protection TLV on a Keepalive message. Such check-point requests could be issued on a timer, after a significant amount of change, or before controlled shutdown of a session. The use of check-pointing may considerably simplify an implementation since it does not need to track the sequence numbers of all received LDP messages. It must, however, still ensure that all received messages (or the consequent state changes) are secured before acknowledging the sequence number on the Keepalive. This approach may be considered optimal in systems that do not show a high degree of change over time (such as - targeted LDP session or CR-LDP systems) and that are - prepared to risk loss of state for the most recent LDP - exchanges. More dynamic systems (such as LDP discovery - sessions) are more likely to want to acknowledge state - changes more frequently so that the maximum amount of - state can be preserved over a failure. + targeted LDP sessions) and that are prepared to risk loss + of state for the most recent LDP exchanges. More dynamic + systems (such as LDP discovery sessions) are more likely + to want to acknowledge state changes more frequently so + that the maximum amount of state can be preserved over a + failure. -11. Acknowledgments +12. Acknowledgments - The work in this draft is based on the LDP and CR-LDP - ideas expressed by the authors of [RFC3212] and []. + The work in this draft is based on the LDP ideas + expressed by the authors of [RFC3036]. The ACK scheme used in this draft was inspired by the proposal by David Ward and John Scudder for restarting BGP sessions now included in [BGP-RESTART]. The authors would also like to acknowledge the careful review and comments of Nick Weeds, Piers Finlayson, Tim Harrison, Duncan Archer, Peter Ashwood-Smith, Bob Thomas, - S.Manikantan, Adam Sheppard, Alan Davey and Iftekhar - Hussain. + S.Manikantan, Adam Sheppard, Alan Davey, Iftekhar Hussain + and Loa Andersson. -12. Intellectual Property Consideration +13. Intellectual Property Consideration The IETF has been notified of intellectual property rights claimed in regard to some or all of the specification contained in this document. For more information, consult the online list of claimed rights. -13. Full Copyright Statement +14. Full Copyright Statement - Copyright (c) The Internet Society (2000, 2001, 2002). All - Rights Reserved. This document and translations of it may - be copied and furnished to others, and derivative works - that comment on or otherwise explain it or assist in its - implementation may be prepared, copied, published and - distributed, in whole or in part, without restriction of - any kind, provided that the above copyright notice and + Copyright (c) The Internet Society (2000, 2001, 2002). + All Rights Reserved. This document and translations of it + may be copied and furnished to others, and derivative + works that comment on or otherwise explain it or assist + in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction + of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. @@ -2381,64 +2391,68 @@ successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. -14. IANA Considerations +15. IANA Considerations This draft requires the use of a number of new TLVs and status codes from the number spaces within the LDP protocol. This section explains the logic used by the authors to choose the most appropriate number space for each new entity, and is intended to assist in the determination of any final values assigned by IANA or the MPLS WG in the event that the MPLS WG chooses to advance this draft on the standards track. This section will be removed when the TLV and status code values have been agreed with IANA. -14.1. New TLVs +15.1. New TLVs - The FT Protection TLV carries attributes that affect a single label - exchanged between LDP peers. It is taken from the 0x02xx range for - TLVs that is used in [RFC3036] by other TLVs carrying label - attributes. The next available value in this range is 0x0203. + The FT Protection TLV carries attributes that affect a + single label exchanged between LDP peers. It is taken + from the 0x02xx range for TLVs that is used in [RFC3036] + by other TLVs carrying label attributes. The next + available value in this range is 0x0203. - The FT Session TLV carries attributes that affect the entire LDP - session between LDP peers. It is taken from the 0x05xx range for - TLVs that is used in [RFC3036] by other TLVs carrying session-wide - attributes. The next available value in this range is 0x0503. + The FT Session TLV carries attributes that affect the + entire LDP session between LDP peers. It is taken from + the 0x05xx range for TLVs that is used in [RFC3036] by + other TLVs carrying session-wide attributes. The next + available value in this range is 0x0503. - The FT Protection TLV may ACK many label operations at once if - cumulative ACKS are used. It is taken from the 0x05xx range for - TLVs that is used in [RFC3036] by other TLVs carrying session-wide - attributes. The next available value in this range is 0x0504. + The FT Protection TLV may ACK many label operations at + once if cumulative ACKS are used. It is taken from the + 0x05xx range for TLVs that is used in [RFC3036] by other + TLVs carrying session-wide attributes. The next + available value in this range is 0x0504. - The FT Cork TLV carries attributes that apply to all labels exchanged - between LDP peers. It is taken from the 0x05xx range for TLVs that - is used in [RFC3036] by other TLVs carrying label attributes. The - next available value in this range is 0x0505. + The FT Cork TLV carries attributes that apply to all + labels exchanged between LDP peers. It is taken from the + 0x05xx range for TLVs that is used in [RFC3036] by other + TLVs carrying label attributes. The next available value + in this range is 0x0505. In summary: FT Protection TLV 0x0203 FT Session TLV 0x0503 FT Ack TLV 0x0504 FT Cork TLV 0x0505 -14.2. New Status Codes +15.2. New Status Codes LDP status codes are not sub-divided into specific ranges for different types of error. Hence, the numeric status code values are selected as the next available. Section 7.1 lists the new status codes required by this draft and gives interpretative information. The new codes are as follows. Status Code E Status Data @@ -2450,80 +2464,80 @@ Unexpected TLV / 1 0x0000001D Label Not FT Missing FT Protection TLV 1 0x0000001E FT ACK sequence error 1 0x0000001F Temporary Shutdown 0 0x00000020 FT Seq Numbers Exhausted 1 0x00000021 FT Session parameters / 1 0x00000022 changed Unexpected FT Cork TLV 1 0x00000023 -15. Authors' Addresses +16. Authors' Addresses Adrian Farrel (editor) Paul Brittain Movaz Networks, Inc. Data Connection Ltd. 7926 Jones Branch Drive, Suite 615 Windsor House, Pepper Street, McLean, VA 22102 Chester, Cheshire Phone: +1 703-847-1867 CH1 1DF, UK Email: afarrel@movaz.com Phone: +44-(0)20-8366-1177 Email: pjb@dataconnection.com Philip Matthews Eric Gray Hyperchip Celox Networks, Inc. 1800 Rene-Levesque Blvd W 2 Park Central Drive, Montreal, Quebec H3H 2H2 Southborough, MA 01772 - Canada Phone: +1 508-305-7214 + Canada Phone: +1 508 305 7214 Phone: +1 514-906-4965 Email: egray@celoxnetworks.com Email: pmatthews@hyperchip.com Jack Shaio Toby Smith Vivace Networks Laurel Networks, Inc. 2730 Orchard Parkway 1300 Omega Drive San Jose, CA 95134 Pittsburgh, PA 15205 Phone: +1 408 432 7623 Email: tob@laurelnetworks.com Email: jack.shaio@vivacenetworks.com Andrew G. Malis Vivace Networks 2730 Orchard Parkway San Jose, CA 95134 Phone: +1 408 383 7223 andy.malis@vivacenetworks.com -16. References - -16.1. Normative References - - [RFC3036] Andersson, L., et. al., LDP Specification, RFC 3036, - January 2001. - - [RFC3212] Jamoussi, B., et. al., Constraint-Based LSP Setup - using LDP, RFC 3212, January 2002. +17. References -16.2. Informative References +17.1. Normative References [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC3036] Andersson, L., et. al., LDP Specification, RFC 3036, + January 2001. + + [LDP-RESTART] Leelanivas, M., et al., Graceful Restart Mechanism for + LDP, draft-ietf-ldp-restart-02.txt, June 2002, work in + progress. + +17.2. Informative References + [RFC2205] Braden, R., et al., Resource ReSerVation Protocol (RSVP) -- Version 1, Functional Specification, RFC 2205, September 1997. [RFC2961] Berger, L., et al., RSVP Refresh Reduction Extensions, RFC 2961, April 2001. [RFC3209] Awduche, D., et al,. Extensions to RSVP for LSP Tunnels, RFC 3209, December 2001. + [RFC3212] Jamoussi, B., et. al., Constraint-Based LSP Setup + using LDP, RFC 3212, January 2002. + [RFC3214] Ash, G., et al., LSP Modification Using CR-LDP, RFC 3214, January 2001. [BGP-RESTART] Sangli, S., et al., Graceful Restart Mechanism for BGP, draft-ietf-idr-restart-05.txt, June 2002 (work in progress). - - [LDP-RESTART] Leelanivas, M., et al., Graceful Restart Mechanism for - LDP, draft-ietf-ldp-restart-02.txt, June 2002, work in - progress.