Network Working Group                                     Robert Thurlow

Document: draft-ietf-nfsv4-repl-mig-proto-00.txt draft-ietf-nfsv4-repl-mig-proto-01.txt

           A Server-to-Server Replication/Migration Protocol

Status of this Memo

   This document is an Internet-Draft and is subject to all provisions
   of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   Discussion and suggestions for improvement are requested.  This
   document will expire in April, November, 2003. Distribution of this draft is
   unlimited.

Abstract

   NFS Version 4 [RFC3010] [RFC3530] provided support for client/server
   interactions to support replication and migration, but left
   unspecified how replication and migration would be done.  This
   document is an initial draft of a protocol which could be used to
   transfer filesystem data and metadata for use with replication and
   migration services for NFS Version 4.

Title               A Replication/Migration Protocol        October 2002            May 2003

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
   1.1.  Shortcomings . . . . . . . .  Changes Since Last Revision  . . . . . . . . . . . . . . . 3
   1.2.  Rationale  .  Shortcomings . . . . . . . . . . . . . . . . . . . . . . . 3 4
   1.3.  Basic structure  Rationale  . . . . . . . . . . . . . . . . . . . . . . . . 4
   2.  Common data types
   1.4.  Basic structure  . . . . . . . . . . . . . . . . . . . . . 4
   2.1.  Security tokens
   2.  Common data types  . . . . . . . . . . . . . . . . . . . . . 4
   2.2. 5
   2.1.  Session, message, file and checkpoint IDs . . . . . . . . 4
   2.3. . . . . . 5
   2.2.  Offset, length and cookies . . . . . . . . . . . . . . . . 5
   2.4.
   2.3.  General status . . . . . . . . . . . . . . . . . . . . . . 5
   2.5.
   2.4.  From NFS Version 4 [RFC3010] . . . . . . . . . . . . . . . 5
   3.  Transfer protocol phases . . . [RFC3530] . . . . . . . . . . . . . . . 6
   4.  Initiation/restart phase . . . . . . . . . . . . . . . . . . 7
   4.1.  Initiation/restart phase messages  . . .
   3.  Session Management . . . . . . . . . 7
   4.2.  Initiation/restart phase overview . . . . . . . . . . . . 7
   4.3.
   3.1.  Capabilities negotiation . . . . . . . . . . . . . . . . . 7
   4.4.  OPEN_SESSION_NEW message . . . . .
   3.2.  Security Negotiation . . . . . . . . . . . . 7
   4.5.  OPEN_SESSION_RESUME message . . . . . . . 8
   3.3.  OPEN_SESSION call  . . . . . . . . 8
   4.6.  OPEN_SESSION_CONFIRM message . . . . . . . . . . . . 8
   3.4.  CLOSE_SESSION call . . . 9
   4.7.  OPEN_SESSION_DENY message . . . . . . . . . . . . . . . . 9
   5.  11
   4.  Data transfer phase  . . . . . . . . . . . . . . . . . . . . 9
   5.1. . .  12
   4.1.  Data transfer phase messages operations . . . . . . . . . . . . . . . 9
   5.2. .  12
   4.2.  Data transfer phase overview . . . . . . . . . . . . . .  10
   5.3.  SEND_OBJECT_METADATA message . . . . . . . . . . . . . .  11
   5.4.  SEND_FILE_DATA message . . . . . . . . . . . . . . . . .  12
   5.5.  SEND_LOCK_STATE message  . . . . . .
   4.3.  SEND call  . . . . . . . . . .  12
   5.6.  SEND_SHARE_STATE message . . . . . . . . . . . . .  13
   4.4.  Data transfer operation description  . . .  12
   5.7.  SEND_REMOVE message . . . . . . .  15
   4.4.1.  SEND_METADATA operation  . . . . . . . . . . .  13
   5.8.  SEND_RENAME message . . . .  15
   4.4.2.  SEND_FILE_DATA operation . . . . . . . . . . . . . .  13
   5.9.  SEND_DIRECTORY_CONTENTS message .  15
   4.4.3.  SEND_FILE_HOLE operation . . . . . . . . . . .  14
   5.10.  SEND_CHECKPOINT message . . . .  16
   4.4.4.  SEND_LOCK_STATE operation  . . . . . . . . . . . .  14
   5.11.  CLOSE_OBJECT message . .  16
   4.4.5.  SEND_SHARE_STATE operation . . . . . . . . . . . . . .  16
   4.4.6.  SEND_DELEG_STATE operation .  14
   5.12.  CONFIRM_MESSAGE message . . . . . . . . . . . . .  17
   4.4.7.  SEND_REMOVE operation  . . .  15
   6.  Termination phase . . . . . . . . . . . . .  17
   4.4.8.  SEND_RENAME operation  . . . . . . .  15
   6.1.  Termination phase messages . . . . . . . . .  18
   4.4.9.  SEND_LINK operation  . . . . . .  15
   6.2.  Termination phase overview . . . . . . . . . . .  18
   4.4.10.  SEND_SYMLINK operation  . . . .  15
   6.3.  ABORT_SESSION message . . . . . . . . . . .  18
   4.4.11.  SEND_DIR_CONTENTS operation . . . . . .  15
   6.4.  CLOSE_SESSION message . . . . . . .  19
   4.4.12.  SEND_CLOSE operation  . . . . . . . . . .  16
   7.  XDR protocol definition file . . . . . .  19
   5.  IANA Considerations  . . . . . . . . .  17
   8.  IANA Considerations . . . . . . . . . .  19
   6.  Security Considerations  . . . . . . . . .  24
   9.  Security Considerations . . . . . . . .  19
   7.  Appendix A: XDR Protocol Definition File . . . . . . . . .  24
   10.  20
   8.  Normative References . . . . . . . . . . . . . . . . . .  25
   11. .  28
   9.  Informative References . . . . . . . . . . . . . . . . .  25
   12. .  29
   10.  Author's Address  . . . . . . . . . . . . . . . . . . . .  26  30

Title               A Replication/Migration Protocol        October 2002            May 2003

1.  Introduction

   This document introduces describes a "strawman" proposed protocol to perform the data
   transfer involved with replication and migration, as the problem was
   described in [DESIGN]; familiarity with that document is assumed.  It
   is not yet proven by implementation experience, but is presented for
   collective work and discussion.

   Though data replication and transfer are needed in many areas, this
   document will focus primarily on solving the problem of providing
   replication and migration support between NFS Version 4 servers.  It
   is assumed that the reader has familiarity with NFS Version 4
   [RFC3010].
   [RFC3530].

1.1.  Shortcomings

   This draft has  Changes Since Last Revision

   Since the -00 version of this draft, the following known shortcomings: major changes have
   been made:

   o    it    The protocol no longer uses XDR-formatted messages sent via TCP;
        it now uses RPC calls and replies.

   o    The elements used to transfer data and metadata are now
        operations arguments to a unified SEND RPC, so that an array of
        information about a particular file may be sent in one RPC call.

   o    Session management has been simplified to a single OPEN_SESSION
        call and a single CLOSE_SESSION call.  Sessions may also be
        multiplexed over the same connection.

   o    The protocol should now work in a continuous replication mode,
        where a transfer session stays up indefinitely and changes can
        be passed rapidly to replicas.

   o    Support for transferring delegation state has been added.

   o    Support for transferring hard links and symbolic links has been
        added.

   o    Zero-filled regions or "holes" are now sent as separate
        operations, rather than being treated as a special case of data
        transfers.

   o    ACLs and the object type are handled as part of the RMattrs
        type, rather than being separate.

Title               A Replication/Migration Protocol            May 2003

1.2.  Shortcomings

   This draft has the following known shortcomings:

   o    it does not deal with [RSYNC]-like behaviour, which can compare
        source and destination files

   o    it does not define how security will be implemented

   o    it introduces a capabilities negotiation feature which is very
        incomplete

1.2. not
        complete enough to be useful

   o    it does not fully specify compression algorithms which can be
        used

   o    it does not specify how it works with minor revisions to NFS
        Version 4

1.3.  Rationale

   The protocol presented below is currently a very simple bulk-data transfer protocol
   with minimal traffic in the reverse direction.  It is believed that
   optimal performance is best achieved by a well-
   implemented well-implemented source
   server sending the smallest set of change information to the
   destination.  The advantages in this protocol over data formats such
   as tar/pax/cpio (as defined by IEEE 1003.1 or ISO/IEC 9945-1) are:

   o    NFSv4 Access Control Lists (ACLs) and named attributes can be
        transferred

   o    The richer NFSv4 metadata set can be transferred

   o    Restarting of transfers can be achieved.

Title               A Replication/Migration Protocol        October 2002

1.3. achieved

   o    The bandwidth requirements approach the smallest possible.

1.4.  Basic structure

   This replication/migration protocol is optimized for bulk data
   transfer with a minimum of overhead.  The ideal case is where the
   source server can stream filesystem data (or just the changes made)
   to the destination, without negotiations which can cause stalls. destination.  An alternate [RSYNC]-like mode which supports
   both servers comparing files to determine differences may be added at a later time, has been
   discussed, but is not present in this draft.  As discussed in [DESIGN],

   Unlike the previous version of this protocol draft draft, this version will not use specify
   RPC [RFC1831] but will use rather than just XDR [RFC1832] formatted messages over
   TCP.  Implementations MUST support operation over TCP
   to maximize efficiency. and MAY support

Title               A Replication/Migration Protocol            May 2003

   UDP and other transports supported by RPC.

   The use of XDR is also subject to change.

2.  Common data types

2.1.  Security tokens

   Security tokens are defined for each protocol message permits multiple "sessions" per TCP connection by using
   session identifiers in each RPC.  Sessions can be terminated and
   restarted at a later time.  Sessions used to update replicas can also
   be left in place continuously, so that it will
   be possible changes to implement security with GSS-API tokens.  Currently, we
   do not define this completely enough to permit a secure
   implementation.

   enum RMsec_mode {
           RM_NULLSEC = 0,
           RM_PERMESSAGE = 1
   };

   struct RMsecpayload {
           uint32_t length;
           opaque contents<>;
   };

   union RMsec_token switch (RMsec_mode mode) {
    case RM_PERMESSAGE:
           RMsecpayload payload;
    case RM_NULLSEC:
           void;
    default:
           void;
   };

2.2.  Session, message, file and checkpoint IDs

   These IDs are common to many messages.  All are simple 64-bit
   quantities except for RMcheckpoint, which is adds a time so that the
   earliest checkpoint master can be chosen;
   reflected on the id field is chosen such that replicas in near-real-time.

   The SEND RPC has been optimized by permitting an array of data and
   metadata updates to be sent in one RPC call, while the response
   permits the source server can easily use it to restart an aborted session. know how far the destination got in
   applying the updates.

2.  Common data types

2.1.  Session, file and checkpoint IDs

   RMsession_id permits multiplexing transfer sessions on a single
   authenticated connection; the value is chosen at the pleasure of arbitrarily by the
   source server.

Title               A Replication/Migration Protocol        October 2002

   RMmessage_id  RMcheckpoint is used to track the last RPC known to
   the destination so that restart can be done; a monotonically increasing message count chosen by timestamp is supplied
   to help choose the source server. earliest checkpoint.  RMfile_id is intended to be
   identical to the NFSv4 fileid attribute.

   typedef uint64_t RMsession_id;

   typedef uint64_t RMmessage_id;

   typedef uint64_t RMfile_id;

   struct RMcheckpoint {
           nfstime4 time;
           uint64_t id;
   };

2.3.

2.2.  Offset, length and cookies

   These variables are chosen for compatibility with NFSv4.

   typedef uint64_t        RMoffset;
   typedef uint64_t        RMlength;
   typedef uint64_t        RMcookie;

2.4.

2.3.  General status

   Status messages in OPEN_SESSION_DENY responses for OPEN_SESSION and SEND responses and ABORT_SESSION
   CLOSE_SESSION reasons shall return a value from this set.

Title               A Replication/Migration Protocol            May 2003

   enum RMstatus {
           RM_OK = 0,
           RMERR_PERM = 1,
           RMERR_IO = 5,
           RMERR_EXISTS = 17
   };

2.5.

2.4.  From NFS Version 4 [RFC3010] [RFC3530]

   The following definitions are imported from NFS Version 4.

   typedef uint32_t        bitmap4<>;
   typedef opaque          attrlist4<>;
   typedef opaque          utf8string<>;
   typedef opaque          utf8str_mixed<>;
   typedef opaque          utf8str_cis<>;

   struct nfstime4 {
           int64_t         seconds;
           uint32_t        nseconds;
   };

Title               A Replication/Migration Protocol        October 2002

   enum nfs_ftype4 {
           NF4REG          = 1,    /* Regular File */
           NF4DIR          = 2,    /* Directory */
           NF4BLK          = 3,    /* Special File - block device */
           NF4CHR          = 4,    /* Special File - character device */
           NF4LNK          = 5,    /* Symbolic Link */
           NF4SOCK         = 6,    /* Special File - socket */
           NF4FIFO         = 7,    /* Special File - fifo */
           NF4ATTRDIR      = 8,    /* Attribute Directory */
           NF4NAMEDATTR    = 9     /* Named Attribute */
   };

   typedef uint32_t        acetype4;
   typedef uint32_t        aceflag4;
   typedef uint32_t        acemask4;
   struct nfsace4 {
           acetype4        type;
           aceflag4        flag;
           acemask4        access_mask;
           utf8string      who;
   };

   typedef nfsace4         fattr4_acl<>;
   typedef nfs_acl         fattr4_acl;

Title               A Replication/Migration Protocol            May 2003

   struct fattr4 {
           bitmap4         attrmask;
           attrlist4       attr_vals;
   };
   typedef nfs_attr fattr4;

   const OPEN4_SHARE_ACCESS_READ   = 0x00000001;
   const OPEN4_SHARE_ACCESS_WRITE  = 0x00000002;
   const OPEN4_SHARE_ACCESS_BOTH   = 0x00000003;
   const OPEN4_SHARE_DENY_NONE     = 0x00000000;
   const OPEN4_SHARE_DENY_READ     = 0x00000001;
   const OPEN4_SHARE_DENY_WRITE    = 0x00000002;
   const OPEN4_SHARE_DENY_BOTH     = 0x00000003;

3.  Transfer protocol phases

   The servers using  Session Management

   Security flavors supported by the protocol can destination server may be known in
   advance, or may be discovered via an initial NULL RPC call which uses
   SNEGO GSS-API pseudo-mechanism as defined in [RFC2478].  A security
   flavor normally does not change through the following phases:

   o    Initiation Phase: authentication, negotiation, filesystem info

   o    Restart Phase [initiate when restarting]: add restart info to life of the above

   o    Data Transfer Phase: send attributes and data for each file

   o    Termination Phase: final handshake

   For simplicity, session.

   A transfer session is created or resumed with the protocol merges initiation OPEN_SESSION call
   and restart phases.

Title               A Replication/Migration Protocol        October 2002

4.  Initiation/restart phase

4.1.  Initiation/restart phase messages terminated normally or abnormally with the CLOSE_SESSION call.
   This is simpler than the previous draft of this protocol.  The following messages are used to set up
   OPEN_SESSION call permits negotiation of capabilities and authenticate a new
   transfer session:

   o    OPEN_SESSION_NEW - create new transfer session

   o    OPEN_SESSION_RESUME - resume previous transfer session at of the
   checkpoint

   o    OPEN_SESSION_CONFIRM - accept transfer session

   o    OPEN_SESSION_DENY - decline transfer session

4.2.  Initiation/restart phase overview

   The source server initiates a session by sending OPEN_SESSION_NEW
   (for a new transfer) or OPEN_SESSION_RESUME (to resume an old
   transfer).  The destination server responds with either
   OPEN_SESSION_CONFIRM to permit a session or OPEN_SESSION_DENY to
   refuse be used for a session.

4.3. restart, while CLOSE_SESSION permits
   abnormal as well as normal termination.

3.1.  Capabilities negotiation

   The OPEN_SESSION_NEW and OPEN_SESSION_RESUME messages

   Parameters in the OPEN_SESSION call express certain capabilities of
   the source server and provide an indication of properties of the data
   to be transferred.  The destination server is responsible for
   reacting to these capabilities.  If the desired capabilities are an issue, it can respond with OPEN_SESSION_DENY not
   acceptable to
   refuse a session or it the destination, the response can respond with OPEN_SESSION_CONFIRM with
   unsupportable capability bits cleared to bid down. down capabilities
   by clearing capabilities bits, or reject the session by failing the
   RPC.  If the lowered capabilities bid by the destination server are
   not acceptable to the source server, the session should be terminated
   with ABORT_SESSION.

4.4.  OPEN_SESSION_NEW message

   SYNOPSIS

   enum RMcomp_type {
           RM_NULLCOMP = 0,
           RM_COMPRESS = 1,
           RM_ZIP = 2
   };

Title               A Replication/Migration Protocol        October 2002

   typedef uint64_t RMcapability;
   const   RM_UTF8NAMES = 0x00000001;
   const   RM_FHPRESERVE = 0x00000002;

   struct OPEN_SESSION_NEW {
           RMsec_token sec_token;
           RMsession_id session_id;
           utf8string src_path;
           utf8string dest_path;
           uint64_t fs_size;
           uint64_t tr_size;
           uint64_t tr_objs;
           RMcomp_type comp_list<>;
           RMcapability capabilities;
   };

   OPEN_SESSION_NEW is a proposal CLOSE_SESSION.

   Currently, only three capabilities are specified; we expect to create a transfer session add
   more through working group effort.  Specified so far are the
   following:

   o    RM_UTF8NAMES - source server supports and expects to send
        filenames encoded in UTF-8 format.  If the full or incremental contents of one filesystem.  The session_id
   is a unique number assigned destination server
        does not support UTF-8 filenames, it should convey that by
        clearing the flag.

   o    RM_FHPRESERVE - source server to the transfer
   session.  src_path is the full path name willing to attempt to preserve
        filehandles by sending them as part of each SEND_METADATA
        operation.  If the destination can issue filehandles which it
        did not generate, and can work with the filehandle format used
        by the implementation identified by RMimplementation field in
        the OPEN_SESSION arguments, it can accept this offer; otherwise
        it should clear the bit to indicate refusal.  Since the source

Title               A Replication/Migration Protocol            May 2003

        server may be denied in attempting to preserve filehandles, it
        should either refuse to transfer data if the destination clears
        this flag, or should advise clients of the possibility that
        filehandles will change via the [RFC3530] FH4_VOL_MIGRATION bit.

   o    RM_FILEID - in combination with RM_FHPRESERVE, the source server
        is willing to attempt to preserve file_ids as well.  If the
        destination can issue file_ids which it did not generate, and
        can work with the file_id format used by the implementation
        identified by RMimplementation field in the OPEN_SESSION
        arguments, it can accept this offer; otherwise it should clear
        the bit to indicate refusal.

3.2.  Security Negotiation

   Security for this protocol is provided by the RPCSEC_GSS mechanism,
   defined in [RFC2203], with the same GSS-API mechanisms defined as
   mandatory-to-implement as [RFC3530], namely the Kerberos V5 and
   LIPKEY mechanisms defined in [RFC1964] and [RFC2847].  In the case of
   a client and server implementing more than one of these mechanisms,
   the first RPC call should be an RPC NULL procedure call with the
   RPCSEC_GSS auth flavor and the SNEGO GSS-API mechanism populated with
   the mechanisms acceptable to the client.  The server should respond
   with the preferred mechanism, if any, and this mechanism will be used
   for all sessions on this connection.

3.3.  OPEN_SESSION call

   SYNOPSIS

   OPEN_SESSIONargs -> OPEN_SESSIONres

   ARGUMENT

   struct RMnewsession {
           utf8string src_path;
           utf8string dest_path;
           uint64_t fs_size;
           uint64_t tr_size;
           uint64_t tr_objs;
   };

   struct RMoldsession {
           RMcheckpoint check_id;
           uint64_t rem_size;
           uint64_t rem_objs;
   };

Title               A Replication/Migration Protocol            May 2003

   union RMopeninfo switch (bool new) {
    case TRUE:
           RMnewsession newinfo;
    case FALSE:
           RMoldsession oldinfo;
   };

   typedef uint64_t RMcapability;
   typedef utf8str_cis RMimplementation<>;

   struct OPEN_SESSIONargs {
           RMsession_id session_id;
           RMcomp_type comp_list<>;
           RMcapability capabilities;
           RNimplementation implementation;
           RMopeninfo info;
   };

   RESULT

   struct RMopenok {
           RMcheckpoint check_id;
           RMcomp_type comp_alg;
           RMcapability capabilities;
   };

   union RMopenresp switch (RMstatus status) {
    case RM_OK:
           RMopenok info;
    default:
           void;
   };

   struct OPEN_SESSIONres {
           RMsession_id session_id;
           RMopenresp response;
   };

   OPEN_SESSION is a request to create or resume a transfer session to
   send the full or incremental contents of one filesystem.  For either
   new or resuming sessions, the source server supplies the following
   information:

   o    session_id - a unique number assigned by the source server to
        the transfer session, or the number of the session to be
        resumed.

Title               A Replication/Migration Protocol            May 2003

   o    comp_list - a list of compression types the filesystem on the source server, server can
        use to compress data.

   o    capabilities - the bitmask used to negotiate as described in
        Section 4.3.

   o    implementation - a descriptor of the operating system and dest_path
        filesystem implementation, with version information, used by the
        source server; this is to permit preservation of filehandles and
        fileids if the destination server runs a compatible version.
        This field is constructed at the pleasure of the source server
        and need only be parsed properly by a destination server running
        the same operating system code.

   For new sessions, the source server supplies the following
   information:

   o    src_path - full path name to the filesystem on source server

   o    dest_path - full path name to the destination.  fs_size and tr_size are filesystem on the approximate destination
        server

   o    fs_size - total size of the filesystem data and the

   o    tr_size - amount of filesystem data to be sent during this
        transfer session. session

   o    tr_objs is the approximate - number of objects to be sent or updated in this
        transfer session.  comp_list is a list of
   compression types session

   For resuming sessions, the source server can use to compress data.
   capabilities is supplies the bitmask used to negotiate as described in Section
   4.3.

4.5.  OPEN_SESSION_RESUME message

   SYNOPSIS

   struct OPEN_SESSION_RESUME {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           uint64_t rem_size;
           uint64_t rem_objs;
           RMcomp_type comp_list<>;
           RMcapability capabilities;
   };

   OPEN_SESSION_RESUME is a proposal to resume a transfer session which
   was not previously completed.  It sends a following
   information:

   o    check_id - checkpoint ID which is for the last message it believes it successfully sent.  If the
   destination has a checkpoint with an earlier timestamp, it will reply
   with that checkpoint ID as an alternate starting point.  The

Title               A Replication/Migration Protocol        October 2002

   approximate RPC believed sent

   o    rem_size - remaining number amount of bytes filesystem data to transfer and be sent

   o    rem_objs - remaining number of objects to
   update are passed in rem_size and rem_objs.  Other parameters are as
   defined in OPEN_SESSION_NEW.

4.6.  OPEN_SESSION_CONFIRM message

   SYNOPSIS

   struct OPEN_SESSION_CONFIRM {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMcomp_type comp_alg;
           RMcapability capabilities;
   };

   OPEN_SESSION_CONFIRM is used by be sent or updated

   The response from the destination server to agree to
   open a transfer may reject the session and agree
   proposal with an error code, may accept the proposal outright, or may
   bid down the capabilities or state that it needs to start from an earlier
   checkpoint than that proposed by the source server, and to source.  The destination will
   also choose a compression algorithm. algorithm from the list the source
   provided.  The source may issue a CLOSE_SESSION call if capabilities
   negotiated down are not acceptable to it.  Once the session OPEN_SESSION RPC
   has been confirmed, completed, SEND RPCs with data transfer messages operations will be
   sent until a CLOSE_SESSION or ABORT_SESSION message RPC is sent.

4.7.  OPEN_SESSION_DENY message

Title               A Replication/Migration Protocol            May 2003

3.4.  CLOSE_SESSION call

   SYNOPSIS

   CLOSE_SESSIONargs -> CLOSE_SESSIONres

   ARGUMENT

   struct RMbadclose {
           RMcheckpoint check_id;
           bool_t restartable;
   };

   union RMcloseinfo switch (RMstatus status) {
    case RM_OK:
           void;
    default:
           RMbadclose info;
   };

   struct OPEN_SESSION_DENY CLOSE_SESSIONargs {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMstatus status;
           RMcloseinfo info;
   };

   RESULT

   struct CLOSE_SESSIONres {
           RMsession_id session_id;
           RMcheckpoint check_id;
   };

   OPEN_SESSION_DENY

   CLOSE_SESSION is used to terminate the session normally or abnormally
   by the destination server source server.

   A normal close is handled by setting the RMcloseinfo status to reject an
   OPEN_SESSION_NEW or OPEN_SESSION_RESUME proposal for any reason.  The
   reason RM_OK.
   Upon a normal close, a migration event is considered complete and the
   source will be expressed begin to refer clients to the destination server.

   An abnormal close is handled by setting the status code.

5. to something other
   than RM_OK and supplying the last checkpoint the source server
   believes it sent plus an indication of whether it is possible to
   restart the transfer from that checkpoint.  The destination server
   responds with the last checkpoint it has successfully committed.  The
   destination server should attempt to save the state of the aborted
   session for a period of at least one hour.

Title               A Replication/Migration Protocol            May 2003

4.  Data transfer phase

5.1.

4.1.  Data transfer phase messages operations

   Data transfer is accomplished by the SEND RPC, which takes an array
   of unions to permit a variety of transfer operations to be sent in
   each RPC.  All operations must pertain to one filesystem object,
   since the RMfile_id is provided for each SEND RPC, not for each
   operation.  Each operation in the array has an RMstatus in the
   response, so the source server can track how much was done if the
   call failed.  Processingn stops at the first failure, and the SEND
   RPC response status is set to the first failure status.

   The following messages are used to transfer filesystem data during a transfer session:

Title               A Replication/Migration Protocol        October 2002 operations are supported:

   o    SEND_OBJECT_METADATA    SEND_METADATA - send metadata about object

   o    SEND_FILE_DATA - send file data

   o    SEND_FILE_HOLE - send file data

   o    SEND_LOCK_STATE - send file lock state

   o    SEND_SHARE_STATE - send share modes state

   o    SEND_DELEG_STATE - send delegation state

   o    SEND_REMOVE - send an object removal transaction

   o    SEND_RENAME - send an object rename transaction

   o    SEND_DIRECTORY_CONTENTS    SEND_LINK - send an object link transaction

   o    SEND_SYMLINK - send an object symlink transaction

   o    SEND_DIR_CONTENTS - send names of objects in a directory

   o    SEND_CHECKPOINT - send checkpoint info

   o    CLOSE_OBJECT    SEND_CLOSE - signal completion of object

   o    CONFIRM_MESSAGE - confirm any data transfer message

5.2.

4.2.  Data transfer phase overview

   The source server begins processing processes filesystem objects in some fixed known order
   which will permit checkpointing and restarting in case of some
   problem or operator abort.  SEND_OBJECT_METADATA  Full transfers should be done in order
   such that objects which are needed, such as directories and link
   targets, are present when referrals are made to them.  Incremental

Title               A Replication/Migration Protocol            May 2003

   transfers should be done in the order changes were made on the source
   server, if possible; if not possible, the order described for full
   transfers is acceptable.

   For files which are to be created or updated, SEND_METADATA is sent
   first, then SEND_FILE_DATA messages operations will be sent for non-directory objects. sent.  If outstanding lock
   lock, share or delegation state for an onject object exists on the source
   server, it will be sent via SEND_LOCK_STATE messages; SEND_LOCK_STATE, SEND_SHARE_STATE does or
   SEND_DELEG_STATE operations after all data has been transferred.
   SEND_CLOSE is used to signal that all changes to a file are complete.
   Directories are created with SEND_METADATA, but are not populated
   until its objects are created, so the
   equivalent for share modes state. SEND_METADATA is followed by
   SEND_CLOSE.

   Ideally, the source server will track all filesystem changes, changes via a
   mechanism such as [DMAPI], and will be able to reflect remove and remove, rename
   and link changes via SEND_REMOVE and SEND_REMOVE, SEND_RENAME messages. and SEND_LINK
   operations.  If the source server cannot capture all create and
   remove operations on a directory reliably,
   SEND_DIRECTORY_CONTENTS will permit the destination server to list
   its SEND_DIR_CONTENTS should
   be used.  This operation lists all directory entries for a source
   server, so that the destination server can compute what items should
   be removed.  This is less reliable than being able to send
   SEND_REMOVE, SEND_RENAME and SEND_LINK operations, and should be used
   only when the underlying filesystem cannot record changes as they
   happen.

   Named attributes for a filesystem object are handled with SEND_OBJECT_METADATA messages
   SEND_METADATA operations with
   IS_NAMED_ATTR set to true, and apply to file type NF4NAMEDATTR.  This will be
   "nested", i.e. it will be understood that the previous non-named- named attribute which was is
   associated with the parent object handled.  CLOSE_OBJECT  SEND_CLOSE is used to
   indicate that all data and named attributes metadata of an object the named attribute have been transferred.  At
   any time,
   transferred, and must be issued before another named attribute can be
   handled and before the SEND_CLOSE for the source server parent object is issued.
   Named attributes may set a checkpoint with
   SEND_CHECKPOINT.  All messages are confirmed by the destination
   server with CONFIRM_MESSAGE. not themselves have named attributes.

4.3.  SEND call

   SYNOPSIS

   SENDargs -> SENDres

   ARGUMENT

   union RMsendargs switch (RMoptype sendtype) {
    case OP_SEND_METADATA:
           SEND_METADATA metadata;
    case OP_SEND_FILE_DATA:
           SEND_FILE_DATA data;

Title               A Replication/Migration Protocol        October 2002

5.3.  SEND_OBJECT_METADATA message

   SYNOPSIS

   enum RMattrtype            May 2003

    case OP_SEND_FILE_HOLE:
           SEND_FILE_HOLE hole;
    case OP_SEND_LOCK_STATE:
           SEND_LOCK_STATE lock;
    case OP_SEND_SHARE_STATE:
           SEND_SHARE_STATE share;
    case OP_SEND_DELEG_STATE:
           SEND_DELEG_STATE deleg;
    case OP_SEND_REMOVE:
           SEND_REMOVE remove;
    case OP_SEND_RENAME:
           SEND_RENAME rename;
    case OP_SEND_LINK:
           SEND_LINK link;
    case OP_SEND_SYMLINK:
           SEND_SYMLINK symlink;
    case OP_SEND_DIR_CONTENTS:
           SEND_DIR_CONTENTS dirc;
    case OP_SEND_CLOSE:
           void;
   };

   struct SEND1args {
           RM_NFS_ATTR = 0,
           RM_CIFS_ATTR = 1
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMfile_id file_id;
           RMsendargs sendarray<>;
   };

   RESULT

   union RMattrs RMsendres switch (RMattrtype type) (RMoptype sendtype) {
    case RM_NFS_ATTR:
           nfs_attr        attr; OP_SEND_METADATA:
    case RM_CIFS_ATTR:
           void;
    default:
           void; OP_SEND_FILE_DATA:
    case OP_SEND_FILE_HOLE:
    case OP_SEND_LOCK_STATE:
    case OP_SEND_SHARE_STATE:
    case OP_SEND_DELEG_STATE:
    case OP_SEND_REMOVE:
    case OP_SEND_RENAME:
    case OP_SEND_LINK:
    case OP_SEND_SYMLINK:
    case OP_SEND_DIR_CONTENTS:
    case OP_SEND_CLOSE:
           RMstatus status;
   };

Title               A Replication/Migration Protocol            May 2003

   struct SEND_OBJECT_METADATA SEND1res {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMfile_id file_id;
           RMsendres resarray<>;
           RMstatus status;
   };

   The SEND RPC batches data transfer operations together and sends them
   to the destination server to operate on one file and with one
   checkpoint.  The destination server may fail a call in the middle of
   the array by setting the return status for that operation to
   something other than RM_OK, and will not process further operations.
   The call will be failed with that status as well.

4.4.  Data transfer operation description

4.4.1.  SEND_METADATA operation

   SYNOPSIS

   struct SEND_METADATA {
           utf8string obj_name;
           nfs_ftype4 obj_type;
           RMattrs attrs;
           nfs_acl obj_acl;
           bool is_named_attr;
   };

   SEND_OBJECT_METADATA

   SEND_METADATA announces that we are about to transfer information
   about a particular filesystem object.  If an object does not exist on
   the destination, it will be created with the given
   obj_name, obj_type, attributes, ACL obj_name and file_id (if supported).
   attributes supplied.  If the object exists and is is the correct
   type, its attributes and ACL will be updated.  If an object of the same name
   but a different type exists, it will be removed and recreated with
   this information.  If a
   SEND_OBJECT_METADATA SEND_METADATA has not followed a CLOSE_OBJECT, SEND_CLOSE,
   it may have the is_named_attr flag set, in which case the object is a
   named attribute of the most recent object identified by a SEND_OBJECT_METADATA.

Title               A Replication/Migration Protocol        October 2002

5.4.
   SEND_METADATA.

4.4.2.  SEND_FILE_DATA message operation

   SYNOPSIS

   struct SEND_FILE_DATA {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMoffset offset;
           RMlength length;
           bool is_hole;
           opaque data<>;
   };

Title               A Replication/Migration Protocol            May 2003

   SEND_FILE_DATA sends a block of data for a regular file, or, if the
   is_hole flag is set, an indication that a block of data has been
   zeroed. file.  The range
   is identified by the offset, length pair as starting at seek position
   'offset' and extending through 'offset+length-1', inclusive.

5.5.  SEND_LOCK_STATE message

4.4.3.  SEND_FILE_HOLE operation

   SYNOPSIS

   struct RMlock_desc SEND_FILE_HOLE {
           RMowner owner;
           RMoffset offset;
           RMlength length;
   };
   SEND_FILE_HOLE sends a description of a "hole", or a zero-filled and
   usually unallocated block of data.  A source server which does sparse
   allocation and which can learn via APIs what parts of a file are
   unallocated can use this to describe the hole without transferring
   the block of zeros.

4.4.4.  SEND_LOCK_STATE operation

   SYNOPSIS

   enum RMlocktype {
           RM_NOLOCK = 0,
           RM_READLOCK = 1,
           RM_WRITELOCK = 2
   };

   struct SEND_LOCK_STATE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMlock_desc lock_desc<>;
           RMowner owner;
           RMclientid clientid;
           RMoffset offset;
           RMlength length;
           RMlocktype type;
           RMstateid id;
   };

   SEND_LOCK_STATE transfers ownership and range information about
   outstanding byte-range locks to the destination server.

5.6.  The lock
   stateid is transferred so that the client need not reestablish the
   lock after migration.  RM_NOLOCK is included to support continuous
   replication by permitting locks on replicas to be cleared.

4.4.5.  SEND_SHARE_STATE message operation

   SYNOPSIS

   typedef uint32_t RMaccess;
   typedef uint32_t RMdeny;

Title               A Replication/Migration Protocol        October 2002            May 2003

   typedef uint32_t RMdeny;

   struct RMshare_desc SEND_SHARE_STATE {
           RMowner owner;
           RMclientid client;
           RMaccess mode; accmode;
           RMdeny mode;
   };

   struct SEND_SHARE_STATE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMshare_desc share_desc<>; denymode;
   };

   SEND_SHARE_STATE transfers ownership and mode information about
   outstanding share reservations to the destination server.

5.7.

4.4.6.  SEND_DELEG_STATE operation

   SYNOPSIS

   enum RMdelegtype {
           RM_NODELEG = 0,
           RM_READDELEG = 1,
           RM_WRITEDELEG = 2
   };

   struct SEND_DELEG_STATE {
           RMclientid client;
           RMdelegtype type;
           RMstateid id;
   };

   SEND_DELEG_STATE transfers ownership and type information about
   outstanding file delegations to the destination server.  RM_NODELEG
   is included to support continuous replication by permitting
   delegations on replicas to be cleared.

4.4.7.  SEND_REMOVE message operation

   SYNOPSIS

   struct SEND_REMOVE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           utf8string name;
   };

   SEND_REMOVE documents a remove event on the object identified; upon
   receipt, the destination server will remove the object as well.

5.8.

Title               A Replication/Migration Protocol            May 2003

4.4.8.  SEND_RENAME message operation

   SYNOPSIS

   struct SEND_RENAME {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           utf8string old_name;
           utf8string new_name;
   };

   SEND_RENAME documents a rename event on the object identified by
   old_name; upon receipt, the destination server will rename the object
   as well.
   in the destination filesystem.  Full paths may be used relative to
   the root of the source filesystem.

4.4.9.  SEND_LINK operation

   SYNOPSIS

   struct SEND_LINK {
           utf8string old_name;
           utf8string new_name;
   };

   SEND_LINK documents the creation of a hard link from the old_name to
   the new_name; upon receipt, the destination server will link the
   objects in the destination filesystem.  Full paths may be used
   relative to the root of the source filesystem.

4.4.10.  SEND_SYMLINK operation

   SYNOPSIS

   struct SEND_SYMLINK {
           utf8string old_name;
           utf8string new_name;
   };

   SEND_SYMLINK documents the creation of a symbolic link from the
   old_name to the new_name; upon receipt, the destination server will
   symlink the objects in the destination filesystem.  The old_name
   value is not checked in any way and can be arbitrary textual data.

Title               A Replication/Migration Protocol        October 2002

5.9.  SEND_DIRECTORY_CONTENTS message            May 2003

4.4.11.  SEND_DIR_CONTENTS operation

   SYNOPSIS

   struct SEND_DIRECTORY_CONTENTS SEND_DIR_CONTENTS {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMcookie cookie;
           bool eof;
           utf8string names<>;
   };

   SEND_DIRECTORY_CONTENTS

   SEND_DIR_CONTENTS is used to account for removals and renames when
   source servers cannot record the events such that they may be sent
   with SEND_REMOVE and SEND_RENAME.  The contents are listed in no
   predictable order so that the destination can what entries it has
   which are no longer found on the source.  Each
   SEND_DIRECTORY_CONTENTS SEND_DIR_CONTENTS
   includes an opaque directory cookie to represent starting location of
   the block on the source server, and the eof flag is set on the last
   block.  Any item existing on the destination that is not listed in a SEND_DIRECTORY_CONTENTS message
   SEND_DIR_CONTENTS operation will be removed.

5.10.  SEND_CHECKPOINT message

   SYNOPSIS

   struct SEND_CHECKPOINT {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMcheckpoint check_id;
   };

   SEND_CHECKPOINT is used periodically by the source server to indicate
   a point from which a restart can be done.  The destination server
   will track the last checkpoint it has received and be prepared to
   examine it upon restart.

5.11.  CLOSE_OBJECT message

4.4.12.  SEND_CLOSE operation

   SYNOPSIS

   struct CLOSE_OBJECT {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
   };

Title               A Replication/Migration Protocol        October 2002

   CLOSE_OBJECT

   void;

   SEND_CLOSE is used to announce that all data and metadata changes for
   a particular object have been completed.

5.12.  CONFIRM_MESSAGE message

   SYNOPSIS

   struct CONFIRM_MESSAGE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
   };

   CONFIRM_MESSAGE is used by the destination server to acknowledge
   every data transfer message.

6.  Termination phase

6.1.  Termination phase messages

5.  IANA Considerations

   The following messages are used to terminate replication/migration protocol will use a transfer session:

   o    ABORT_SESSION - end transfer session before natural end

   o    CLOSE_SESSION - complete transfer session normally

6.2.  Termination phase overview

   ABORT_SESSION may be used well-known RPC program
   number at any time by either server to terminate which destination servers will register.  The author will
   acquire an RPC program number for this purpose.

6.  Security Considerations

   NFS Version 4 is the primary impetus behind a
   session prematurely; replication/migration
   protocol, so this protocol should mandate a checkpoint ID is recommended to permit restart
   if possible.  CLOSE_SESSION is strong security scheme in
   a manner comparable with NFS Version 4.  Implementations of this
   protocol MUST support the normal termination message, RPCSEC_GSS security flavor as defined in
   [RFC2203] and must be issued by also support the source server first Kerberos V5 and then issued by the
   destination server LIPKEY mechanisms
   as a confirmation.

6.3.  ABORT_SESSION message

   SYNOPSIS

   struct ABORT_SESSION {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMstatus status;
   };

   ABORT_SESSION terminates a data transfer session immediately.  If may
   be used by either the source or destination server, and records the

Title               A Replication/Migration Protocol        October 2002

   last checkpoint known defined in [RFC1964] and a status code to explain the termination.

6.4.  CLOSE_SESSION message

   SYNOPSIS

   struct CLOSE_SESSION {
           RMsec_token sec_token;
           RMsession_id session_id;
   };

   CLOSE_SESSION terminates a data transfer session normally; it [RFC2847].  The particular mechanism
   chosen for sessions is used
   first determined by the source and is then used by use of SNEGO on the destination server to
   confirm it. initial
   call, which should be a NULL RPC.

Title               A Replication/Migration Protocol        October 2002            May 2003

7.  Appendix A: XDR protocol definition file Protocol Definition File

   /*
    * Copyright (C) The Internet Society (1998,1999,2000,2001,2002).
    *  All Rights Reserved.
    */

   /*
    *      repl-mig.x
    */

   %#pragma ident  "@(#)repl-mig.x        1.1"

   /*
    * Derived types for clarity
    */
   typedef int             int32_t;
   typedef unsigned int    uint32_t;
   typedef hyper           int64_t;
   typedef unsigned hyper  uint64_t; 1.4     03/05/27"

   /*
    * From RFC3010 RFC3530
    */
   typedef uint32_t        bitmap4<>;
   typedef opaque          attrlist4<>;
   typedef opaque          utf8string<>;
   typedef opaque          utf8str_mixed<>;
   typedef opaque          utf8str_cis<>;

   struct nfstime4 {
           int64_t         seconds;
           uint32_t        nseconds;
   };

   enum nfs_ftype4 {
           NF4REG          = 1,    /* Regular File */
           NF4DIR          = 2,    /* Directory */
           NF4BLK          = 3,    /* Special File - block device */
           NF4CHR          = 4,    /* Special File - character device */
           NF4LNK          = 5,    /* Symbolic Link */
           NF4SOCK         = 6,    /* Special File - socket */
           NF4FIFO         = 7,    /* Special File - fifo */
           NF4ATTRDIR      = 8,    /* Attribute Directory */
           NF4NAMEDATTR    = 9     /* Named Attribute */
   };

   typedef uint32_t        acetype4;
   typedef uint32_t        aceflag4;
   typedef uint32_t        acemask4;

   struct nfsace4 {
           acetype4        type;
           aceflag4        flag;
           acemask4        access_mask;
           utf8string      who;
   };

Title               A Replication/Migration Protocol        October 2002            May 2003

           utf8str_mixed   who;
   };

   typedef nfsace4         fattr4_acl<>;
   typedef nfs_acl         fattr4_acl;

   struct fattr4 {
           bitmap4         attrmask;
           attrlist4       attr_vals;
   };
   typedef nfs_attr fattr4;
   const OPEN4_SHARE_ACCESS_READ   = 0x00000001;
   const OPEN4_SHARE_ACCESS_WRITE  = 0x00000002;
   const OPEN4_SHARE_ACCESS_BOTH   = 0x00000003;
   const OPEN4_SHARE_DENY_NONE     = 0x00000000;
   const OPEN4_SHARE_DENY_READ     = 0x00000001;
   const OPEN4_SHARE_DENY_WRITE    = 0x00000002;
   const OPEN4_SHARE_DENY_BOTH     = 0x00000003;

   /*
    * For security tokens
    */
   enum RMsec_mode {
           RM_NULLSEC = 0,
           RM_PERMESSAGE = 1
   };

   struct RMsecpayload {
           uint32_t length;
           opaque contents<>;
   };

   union RMsec_token switch (RMsec_mode mode) {
    case RM_PERMESSAGE:
           RMsecpayload payload;
    case RM_NULLSEC:
           void;
    default:
           void;
   };

   /*
    * For session, message, file and checkpoint IDs
    */
   typedef uint64_t RMsession_id;

   typedef uint64_t RMmessage_id;

   typedef uint64_t RMfile_id;

   struct RMcheckpoint {
           nfstime4 time;

Title               A Replication/Migration Protocol        October 2002
           uint64_t id;
   };

   /*
    * For compression algorithm negotiation
    */
   enum RMcomp_type {
           RM_NULLCOMP = 0,
           RM_COMPRESS = 1,
           RM_ZIP = 2
   };

   /*
    * For capabilities negotiation
    */
   typedef utf8str_cis RMimplementation<>;
   typedef uint64_t RMcapability;
   const   RM_UTF8NAMES = 0x00000001;
   const   RM_FHPRESERVE = 0x00000002;

   /*
    * For general status
    */
   enum RMstatus {
           RM_OK = 0,
           RMERR_PERM = 1,
           RMERR_IO = 5,
           RMERR_EXISTS = 17
   };

Title               A Replication/Migration Protocol            May 2003

   /*
    * For generalized attributes Attributes
    */
   enum RMattrtype {
           RM_NFS_ATTR = 0,
           RM_CIFS_ATTR = 1
   };

   union
   struct RMattrs switch (RMattrtype type) {
    case RM_NFS_ATTR:
           nfs_attr
           fattr4  attr;
    case RM_CIFS_ATTR:
           void;
    default:
           void;
           nfs_ftype4 obj_type;
           fattr4_acl obj_acl;
           bool is_named_attr;
   };

   /*
    * Offset, length and cookies

Title               A Replication/Migration Protocol        October 2002
    */
   typedef uint64_t        RMoffset;
   typedef uint64_t        RMlength;
   typedef uint64_t        RMcookie;

   /*
    * Owner
    */
   typedef utf8str_mixed   RMowner;

   /*
    * Lock and share supporting definitions
    */
   struct RMlock_desc RMclientid {
           RMowner owner;
           RMoffset offset;
           RMlength length;
           utf8string name;
           opaque address<>;
   };

   struct RMstateid {
           uint32_t        seqid;
           opaque          other[12];
   };

   enum RMlocktype {
           RM_NOLOCK = 0,
           RM_READLOCK = 1,
           RM_WRITELOCK = 2
   };

   typedef uint32_t RMaccess;
   typedef uint32_t RMdeny;

   struct RMshare_desc

   enum RMdelegtype {
           RMowner owner;
           RMaccess mode;
           RMdeny mode;
           RM_NODELEG = 0,
           RM_READDELEG = 1,
           RM_WRITEDELEG = 2

Title               A Replication/Migration Protocol            May 2003

   };

   /*
    * Protocol messages elements - session control
    */
   struct OPEN_SESSION_NEW RMnewsession {
           RMsec_token sec_token;
           RMsession_id session_id;
           utf8string src_path;
           utf8string dest_path;
           uint64_t fs_size;
           uint64_t tr_size;
           uint64_t tr_objs;
           RMcomp_type comp_list<>;
           RMcapability capabilities;
   };

   struct OPEN_SESSION_RESUME RMoldsession {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           uint64_t rem_size;
           uint64_t rem_objs;
   };

   union RMopeninfo switch (bool new) {
    case TRUE:
           RMnewsession newinfo;
    case FALSE:
           RMoldsession oldinfo;
   };

   struct OPEN_SESSIONargs {
           RMsession_id session_id;
           RMcomp_type comp_list<>;
           RMcapability capabilities;
           RNimplementation impl;
           RMopeninfo info;
   };

Title               A Replication/Migration Protocol        October 2002

   struct OPEN_SESSION_CONFIRM RMopenok {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMcomp_type comp_alg;
           RMcapability capabilities;
   };

   union RMopenresp switch (RMstatus status) {
    case RM_OK:
           RMopenok info;
    default:
           void;
   };

   struct OPEN_SESSION_DENY OPEN_SESSIONres {
           RMsec_token sec_token;

Title               A Replication/Migration Protocol            May 2003

           RMsession_id session_id;
           RMstatus status;
           RMopenresp response;
   };

   struct SEND_OBJECT_METADATA RMbadclose {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMcheckpoint check_id;
           bool_t restartable;
   };

   union RMcloseinfo switch (RMstatus status) {
    case RM_OK:
           void;
    default:
           RMbadclose info;
   };

   struct CLOSE_SESSIONargs {
           RMsession_id session_id;
           RMcloseinfo info;
   };

   struct CLOSE_SESSIONres {
           RMsession_id session_id;
           RMcheckpoint check_id;
   };

   /*
    * Protocol elements - data transfer
    */
   enum RMoptype {
           OP_SEND_METADATA = 1,
           OP_SEND_FILE_DATA = 2,
           OP_SEND_FILE_HOLE = 3,
           OP_SEND_LOCK_STATE = 4,
           OP_SEND_SHARE_STATE = 5,
           OP_SEND_DELEG_STATE = 6,
           OP_SEND_REMOVE = 7,
           OP_SEND_RENAME = 8,
           OP_SEND_LINK = 9,
           OP_SEND_SYMLINK = 10,
           OP_SEND_DIR_CONTENTS = 11,
           OP_SEND_CLOSE = 12
   };

   /*
    * Data and metadata send items
    */
   struct SEND_METADATA {

Title               A Replication/Migration Protocol            May 2003

           utf8string obj_name;
           nfs_ftype4 obj_type;
           RMattrs attrs;
           nfs_acl obj_acl;
           bool is_named_attr;
   };

   struct SEND_FILE_DATA {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMoffset offset;
           RMlength length;
           bool is_hole;
           opaque data<>;
   };

   struct SEND_FILE_HOLE {
           RMoffset offset;
           RMlength length;
   };

   struct SEND_LOCK_STATE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMlock_desc lock_desc<>;
           RMowner owner;
           RMclientid client;
           RMoffset offset;
           RMlength length;
           RMlocktype type;
           RMstateid id;
   };

   struct SEND_SHARE_STATE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMshare_desc share_desc<>;
           RMowner owner;
           RMclientid client;
           RMaccess accmode;
           RMdeny denymode;
   };

   struct SEND_DELEG_STATE {
           RMclientid client;
           RMdelegtype type;
           RMstateid id;
   };

Title               A Replication/Migration Protocol        October 2002

   struct SEND_REMOVE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           utf8string name;
   };

   struct SEND_RENAME {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           utf8string old_name;
           utf8string new_name;
   };

   struct SEND_DIRECTORY_CONTENTS

   struct SEND_LINK {
           utf8string old_name;

Title               A Replication/Migration Protocol            May 2003

           utf8string new_name;
   };

   struct SEND_SYMLINK {
           utf8string old_name;
           utf8string new_name;
   };

   struct SEND_DIR_CONTENTS {
           RMcookie cookie;
           bool eof;
           utf8string names<>;
   };

   /* no parameters for SEND_CLOSE */

   union RMsendargs switch (RMoptype sendtype) {
    case OP_SEND_METADATA:
           SEND_METADATA metadata;
    case OP_SEND_FILE_DATA:
           SEND_FILE_DATA data;
    case OP_SEND_FILE_HOLE:
           SEND_FILE_HOLE hole;
    case OP_SEND_LOCK_STATE:
           SEND_LOCK_STATE lock;
    case OP_SEND_SHARE_STATE:
           SEND_SHARE_STATE share;
    case OP_SEND_DELEG_STATE:
           SEND_DELEG_STATE deleg;
    case OP_SEND_REMOVE:
           SEND_REMOVE remove;
    case OP_SEND_RENAME:
           SEND_RENAME rename;
    case OP_SEND_LINK:
           SEND_LINK link;
    case OP_SEND_SYMLINK:
           SEND_SYMLINK symlink;
    case OP_SEND_DIR_CONTENTS:
           SEND_DIR_CONTENTS dirc;
    case OP_SEND_CLOSE:
           void;
   };

   union RMsendres switch (RMoptype sendtype) {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMcookie cookie;
           bool eof;
           utf8string names<>;
    case OP_SEND_METADATA:
    case OP_SEND_FILE_DATA:
    case OP_SEND_FILE_HOLE:
    case OP_SEND_LOCK_STATE:

Title               A Replication/Migration Protocol            May 2003

    case OP_SEND_SHARE_STATE:
    case OP_SEND_DELEG_STATE:
    case OP_SEND_REMOVE:
    case OP_SEND_RENAME:
    case OP_SEND_LINK:
    case OP_SEND_SYMLINK:
    case OP_SEND_DIR_CONTENTS:
    case OP_SEND_CLOSE:
           RMstatus status;
   };

   struct SEND_CHECKPOINT SEND1args {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMsession_id session_id;
           RMcheckpoint check_id;
   };

   struct CLOSE_OBJECT {
           RMsec_token sec_token;
           RMmessage_id msg_id;
           RMfile_id file_id;
           RMsendargs sendarray<>;
   };

   struct CONFIRM_MESSAGE {
           RMsec_token sec_token;
           RMmessage_id msg_id;
   };

   struct ABORT_SESSION SEND1res {
           RMsec_token sec_token;
           RMsession_id session_id;
           RMcheckpoint check_id;
           RMfile_id file_id;
           RMsendres resarray<>;
           RMstatus status;
   };

Title               A Replication/Migration Protocol        October 2002

   struct CLOSE_SESSION

   program RM_PROGRAM {
           RMsec_token sec_token;
           RMsession_id session_id;
   };
           version RM_V1 {
                   void
                           RMPROC1_NULL(void) = 0;
                   OPEN_SESSIONres
                           RMPROC1_OPEN_SESSION(OPEN_SESSIONargs) = 1;
                   CLOSE_SESSIONres
                           RMPROC1_CLOSE_SESSION(CLOSE_SESSIONargs) = 2;
                   SEND1res
                           RMPROC1_SEND(SEND1args) = 3;
           } = 1;
   } = 100273;

Title               A Replication/Migration Protocol        October 2002            May 2003

8.  IANA Considerations

   The replication/migration protocol will define a well-known port at
   which destination servers will listen for connection requests.  The
   author will apply to IANA for a port number for this purpose.

9.  Security Considerations

   NFS Version 4 is the primary impetus behind a replication/migration
   protocol, so this protocol should mandate a strong security scheme in
   a manner comparable with NFS Version 4.  At the time of this draft,
   it is unclear whether this protocol should make use of a strong
   host-based security mechanism such as SSH and IPSec or strong per-
   message security based on GSS-API [RFC2078] tokens and mechanisms
   including Kerberos Version 5 used as described in [RFC1964] and
   LIPKEY as described in [RFC2847].  Pending further work in this area,
   this protocol draft defines a per-message security payload which may
   be NULL to permit prototyping, without specifying the messages for
   security negotiations and mechanism negotiation needed to use per-
   message security.

Title               A Replication/Migration Protocol        October 2002

10.  Normative References

   [RFC1831]
   R. Srinivasan, "RPC: Remote Procedure Call Protocol Specification
   Version 2", RFC1831, August 1995.

   [RFC1832]
   R. Srinivasan, "XDR: External Data Representation Standard", RFC1832,
   August 1995.

   [RFC3010]

   [RFC1964]
   J. Linn, "Kerberos Version 5 GSS-API Mechanism", RFC1964, June 1996

   [RFC2203]
   M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol Specification",
   RFC2203, September 1997

   [RFC2478]
   E. Baize, D. Pinkas, "The Simple and Protected GSS-API Negotiation
   Mechanism", RFC2478, December 1998.

   [RFC2847]
   M. Eisler, "LIPKEY - A Low Infrastructure Public Key Mechanism Using
   SPKM", RFC2847, June 2000

   [RFC3530]
   S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M.
   Eisler, D. Noveck, "NFS version "Network File System (NFS) Version 4 Protocol", RFC3010, December 2000.

11.
   RFC3530, April 2003.

Title               A Replication/Migration Protocol            May 2003

9.  Informative References

   [RFC1831]
   R. Srinivasan, "RPC: Remote Procedure Call Protocol Specification
   Version 2", RFC1831, August 1995.

   [RDIST]
   MagniComp, Inc., "The RDist "RDist Home Page", http://www.magnicomp.com/rdist.

   [RSYNC]
   The Samba Team, "The rsync "rsync web pages", http://samba.anu.edu.au/rsync.

   [DESIGN]
   R. Thurlow, "Server-to-Server Replication/Migration Protocol Design
   Principles" (work in progress), http://www.ietf.org/internet-
   drafts/draft-thurlow-nfsv4-repl-mig-design-00.txt, June
   drafts/draft-ietf-nfsv4-repl-mig-design-00.txt, December 2002.

   [DMAPI]
   P. Lawthers, "The Data Management Applications Programming
   Interface",
   http://www.computer.org/conferences/mss95/lawthers/lawthers.htm, July
   1995.

Title               A Replication/Migration Protocol        October 2002

12.            May 2003

10.  Author's Address

   Address comments related to this memorandum to:

        nfsv4-wg@sunroof.eng.sun.com

   Robert Thurlow
   Sun Microsystems, Inc.
   500 Eldorado Boulevard, UBRM05-171
   Broomfield, CO 80021

   Phone: 877-718-3419
   E-mail: robert.thurlow@sun.com