draft-ietf-nfsv4-repl-mig-proto-00.txt		draft-ietf-nfsv4-repl-mig-proto-01.txt
	Network Working Group Robert Thurlow		Network Working Group Robert Thurlow
	Document: draft-ietf-nfsv4-repl-mig-proto-00.txt		Document: draft-ietf-nfsv4-repl-mig-proto-01.txt
	A Server-to-Server Replication/Migration Protocol		A Server-to-Server Replication/Migration Protocol
	Status of this Memo		Status of this Memo
	This document is an Internet-Draft and is subject to all provisions		This document is an Internet-Draft and is subject to all provisions
	of Section 10 of RFC2026.		of Section 10 of RFC2026.
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF), its areas, and its working groups. Note that		Task Force (IETF), its areas, and its working groups. Note that
	skipping to change at page 1, line 31		skipping to change at page 1, line 31
	time. It is inappropriate to use Internet- Drafts as reference		time. It is inappropriate to use Internet- Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	The list of current Internet-Drafts can be accessed at		The list of current Internet-Drafts can be accessed at
	http://www.ietf.org/1id-abstracts.html		http://www.ietf.org/1id-abstracts.html
	The list of Internet-Draft Shadow Directories can be accessed at		The list of Internet-Draft Shadow Directories can be accessed at
	http://www.ietf.org/shadow.html		http://www.ietf.org/shadow.html
	Discussion and suggestions for improvement are requested. This		Discussion and suggestions for improvement are requested. This
	document will expire in April, 2003. Distribution of this draft is		document will expire in November, 2003. Distribution of this draft is
	unlimited.		unlimited.
	Abstract		Abstract
	NFS Version 4 [RFC3010] provided support for client/server		NFS Version 4 [RFC3530] provided support for client/server
	interactions to support replication and migration, but left		interactions to support replication and migration, but left
	unspecified how replication and migration would be done. This		unspecified how replication and migration would be done. This
	document is an initial draft of a protocol which could be used to		document is an initial draft of a protocol which could be used to
	transfer filesystem data and metadata for use with replication and		transfer filesystem data and metadata for use with replication and
	migration services for NFS Version 4.		migration services for NFS Version 4.
	Title A Replication/Migration Protocol October 2002		Title A Replication/Migration Protocol May 2003
	Table of Contents		Table of Contents
	1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3		1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
	1.1. Shortcomings . . . . . . . . . . . . . . . . . . . . . . . 3		1.1. Changes Since Last Revision . . . . . . . . . . . . . . . 3
	1.2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . 3		1.2. Shortcomings . . . . . . . . . . . . . . . . . . . . . . . 4
	1.3. Basic structure . . . . . . . . . . . . . . . . . . . . . 4		1.3. Rationale . . . . . . . . . . . . . . . . . . . . . . . . 4
	2. Common data types . . . . . . . . . . . . . . . . . . . . . 4		1.4. Basic structure . . . . . . . . . . . . . . . . . . . . . 4
	2.1. Security tokens . . . . . . . . . . . . . . . . . . . . . 4		2. Common data types . . . . . . . . . . . . . . . . . . . . . 5
	2.2. Session, message, file and checkpoint IDs . . . . . . . . 4		2.1. Session, file and checkpoint IDs . . . . . . . . . . . . . 5
	2.3. Offset, length and cookies . . . . . . . . . . . . . . . . 5		2.2. Offset, length and cookies . . . . . . . . . . . . . . . . 5
	2.4. General status . . . . . . . . . . . . . . . . . . . . . . 5		2.3. General status . . . . . . . . . . . . . . . . . . . . . . 5
	2.5. From NFS Version 4 [RFC3010] . . . . . . . . . . . . . . . 5		2.4. From NFS Version 4 [RFC3530] . . . . . . . . . . . . . . . 6
	3. Transfer protocol phases . . . . . . . . . . . . . . . . . . 6		3. Session Management . . . . . . . . . . . . . . . . . . . . . 7
	4. Initiation/restart phase . . . . . . . . . . . . . . . . . . 7		3.1. Capabilities negotiation . . . . . . . . . . . . . . . . . 7
	4.1. Initiation/restart phase messages . . . . . . . . . . . . 7		3.2. Security Negotiation . . . . . . . . . . . . . . . . . . . 8
	4.2. Initiation/restart phase overview . . . . . . . . . . . . 7		3.3. OPEN_SESSION call . . . . . . . . . . . . . . . . . . . . 8
	4.3. Capabilities negotiation . . . . . . . . . . . . . . . . . 7		3.4. CLOSE_SESSION call . . . . . . . . . . . . . . . . . . . 11
	4.4. OPEN_SESSION_NEW message . . . . . . . . . . . . . . . . . 7		4. Data transfer . . . . . . . . . . . . . . . . . . . . . . 12
	4.5. OPEN_SESSION_RESUME message . . . . . . . . . . . . . . . 8		4.1. Data transfer operations . . . . . . . . . . . . . . . . 12
	4.6. OPEN_SESSION_CONFIRM message . . . . . . . . . . . . . . . 9		4.2. Data transfer phase overview . . . . . . . . . . . . . . 12
	4.7. OPEN_SESSION_DENY message . . . . . . . . . . . . . . . . 9		4.3. SEND call . . . . . . . . . . . . . . . . . . . . . . . 13
	5. Data transfer phase . . . . . . . . . . . . . . . . . . . . 9		4.4. Data transfer operation description . . . . . . . . . . 15
	5.1. Data transfer phase messages . . . . . . . . . . . . . . . 9		4.4.1. SEND_METADATA operation . . . . . . . . . . . . . . . 15
	5.2. Data transfer phase overview . . . . . . . . . . . . . . 10		4.4.2. SEND_FILE_DATA operation . . . . . . . . . . . . . . . 15
	5.3. SEND_OBJECT_METADATA message . . . . . . . . . . . . . . 11		4.4.3. SEND_FILE_HOLE operation . . . . . . . . . . . . . . . 16
	5.4. SEND_FILE_DATA message . . . . . . . . . . . . . . . . . 12		4.4.4. SEND_LOCK_STATE operation . . . . . . . . . . . . . . 16
	5.5. SEND_LOCK_STATE message . . . . . . . . . . . . . . . . 12		4.4.5. SEND_SHARE_STATE operation . . . . . . . . . . . . . . 16
	5.6. SEND_SHARE_STATE message . . . . . . . . . . . . . . . . 12		4.4.6. SEND_DELEG_STATE operation . . . . . . . . . . . . . . 17
	5.7. SEND_REMOVE message . . . . . . . . . . . . . . . . . . 13		4.4.7. SEND_REMOVE operation . . . . . . . . . . . . . . . . 17
	5.8. SEND_RENAME message . . . . . . . . . . . . . . . . . . 13		4.4.8. SEND_RENAME operation . . . . . . . . . . . . . . . . 18
	5.9. SEND_DIRECTORY_CONTENTS message . . . . . . . . . . . . 14		4.4.9. SEND_LINK operation . . . . . . . . . . . . . . . . . 18
	5.10. SEND_CHECKPOINT message . . . . . . . . . . . . . . . . 14		4.4.10. SEND_SYMLINK operation . . . . . . . . . . . . . . . 18
	5.11. CLOSE_OBJECT message . . . . . . . . . . . . . . . . . 14		4.4.11. SEND_DIR_CONTENTS operation . . . . . . . . . . . . . 19
	5.12. CONFIRM_MESSAGE message . . . . . . . . . . . . . . . . 15		4.4.12. SEND_CLOSE operation . . . . . . . . . . . . . . . . 19
	6. Termination phase . . . . . . . . . . . . . . . . . . . . 15		5. IANA Considerations . . . . . . . . . . . . . . . . . . . 19
	6.1. Termination phase messages . . . . . . . . . . . . . . . 15		6. Security Considerations . . . . . . . . . . . . . . . . . 19
	6.2. Termination phase overview . . . . . . . . . . . . . . . 15		7. Appendix A: XDR Protocol Definition File . . . . . . . . . 20
	6.3. ABORT_SESSION message . . . . . . . . . . . . . . . . . 15		8. Normative References . . . . . . . . . . . . . . . . . . . 28
	6.4. CLOSE_SESSION message . . . . . . . . . . . . . . . . . 16		9. Informative References . . . . . . . . . . . . . . . . . . 29
	7. XDR protocol definition file . . . . . . . . . . . . . . . 17		10. Author's Address . . . . . . . . . . . . . . . . . . . . 30
	8. IANA Considerations . . . . . . . . . . . . . . . . . . . 24
	9. Security Considerations . . . . . . . . . . . . . . . . . 24
	10. Normative References . . . . . . . . . . . . . . . . . . 25
	11. Informative References . . . . . . . . . . . . . . . . . 25
	12. Author's Address . . . . . . . . . . . . . . . . . . . . 26
	Title A Replication/Migration Protocol October 2002		Title A Replication/Migration Protocol May 2003
	1. Introduction		1. Introduction
	This document introduces a "strawman" protocol to perform the data		This document describes a proposed protocol to perform the data
	transfer involved with replication and migration, as the problem was		transfer involved with replication and migration, as the problem was
	described in [DESIGN]; familiarity with that document is assumed. It		described in [DESIGN]; familiarity with that document is assumed. It
	is not yet proven by implementation experience, but is presented for		is not yet proven by implementation experience, but is presented for
	collective work and discussion.		collective work and discussion.
	Though data replication and transfer are needed in many areas, this		Though data replication and transfer are needed in many areas, this
	document will focus primarily on solving the problem of providing		document will focus primarily on solving the problem of providing
	replication and migration support between NFS Version 4 servers. It		replication and migration support between NFS Version 4 servers. It
	is assumed that the reader has familiarity with NFS Version 4		is assumed that the reader has familiarity with NFS Version 4
	[RFC3010].		[RFC3530].
	1.1. Shortcomings		1.1. Changes Since Last Revision
			Since the -00 version of this draft, the following major changes have
			been made:
			o The protocol no longer uses XDR-formatted messages sent via TCP;
			it now uses RPC calls and replies.
			o The elements used to transfer data and metadata are now
			operations arguments to a unified SEND RPC, so that an array of
			information about a particular file may be sent in one RPC call.
			o Session management has been simplified to a single OPEN_SESSION
			call and a single CLOSE_SESSION call. Sessions may also be
			multiplexed over the same connection.
			o The protocol should now work in a continuous replication mode,
			where a transfer session stays up indefinitely and changes can
			be passed rapidly to replicas.
			o Support for transferring delegation state has been added.
			o Support for transferring hard links and symbolic links has been
			added.
			o Zero-filled regions or "holes" are now sent as separate
			operations, rather than being treated as a special case of data
			transfers.
			o ACLs and the object type are handled as part of the RMattrs
			type, rather than being separate.
			Title A Replication/Migration Protocol May 2003
			1.2. Shortcomings
	This draft has the following known shortcomings:		This draft has the following known shortcomings:
	o it does not deal with [RSYNC]-like behaviour, which can compare		o it does not deal with [RSYNC]-like behaviour, which can compare
	source and destination files		source and destination files
	o it does not define how security will be implemented		o it introduces a capabilities negotiation feature which is not
			complete enough to be useful
	o it introduces a capabilities negotiation feature which is very		o it does not fully specify compression algorithms which can be
	incomplete		used
	1.2. Rationale		o it does not specify how it works with minor revisions to NFS
			Version 4
	The protocol presented below is currently a very simple bulk-data		1.3. Rationale
	transfer protocol with minimal traffic in the reverse direction. It
	is believed that optimal performance is best achieved by a well-
	implemented source server sending the smallest set of change
	information to the destination. The advantages in this protocol over
	data formats such as tar/pax/cpio (as defined by IEEE 1003.1 or
	ISO/IEC 9945-1) are:
	o Access Control Lists (ACLs) and named attributes can be		The protocol presented below is a simple bulk-data transfer protocol
			with minimal traffic in the reverse direction. It is believed that
			optimal performance is best achieved by a well-implemented source
			server sending the smallest set of change information to the
			destination. The advantages in this protocol over data formats such
			as tar/pax/cpio (as defined by IEEE 1003.1 or ISO/IEC 9945-1) are:
			o NFSv4 Access Control Lists (ACLs) and named attributes can be
	transferred		transferred
	o The richer NFSv4 metadata set can be transferred		o The richer NFSv4 metadata set can be transferred
	o Restarting of transfers can be achieved.		o Restarting of transfers can be achieved
	Title A Replication/Migration Protocol October 2002		o The bandwidth requirements approach the smallest possible.
	1.3. Basic structure		1.4. Basic structure
	This replication/migration protocol is optimized for bulk data		This replication/migration protocol is optimized for bulk data
	transfer with a minimum of overhead. The ideal case is where the		transfer with a minimum of overhead. The ideal case is where the
	source server can stream filesystem data (or just the changes made)		source server can stream filesystem data (or just the changes made)
	to the destination, without negotiations which can cause stalls. An		to the destination. An alternate [RSYNC]-like mode which supports
	alternate mode which supports servers comparing files to determine		both servers comparing files to determine differences has been
	differences may be added at a later time, but is not present in this		discussed, but is not present in this draft.
	draft. As discussed in [DESIGN], this protocol draft will not use
	RPC [RFC1831] but will use XDR [RFC1832] formatted messages over TCP
	to maximize efficiency. The use of XDR is also subject to change.
	2. Common data types
	2.1. Security tokens
	Security tokens are defined for each protocol message so that it will		Unlike the previous version of this draft, this version will specify
	be possible to implement security with GSS-API tokens. Currently, we		RPC [RFC1831] rather than just XDR [RFC1832] formatted messages over
	do not define this completely enough to permit a secure		TCP. Implementations MUST support operation over TCP and MAY support
	implementation.
	enum RMsec_mode {		Title A Replication/Migration Protocol May 2003
	RM_NULLSEC = 0,
	RM_PERMESSAGE = 1
	};
	struct RMsecpayload {		UDP and other transports supported by RPC.
	uint32_t length;
	opaque contents<>;
	};
	union RMsec_token switch (RMsec_mode mode) {		The protocol permits multiple "sessions" per TCP connection by using
	case RM_PERMESSAGE:		session identifiers in each RPC. Sessions can be terminated and
	RMsecpayload payload;		restarted at a later time. Sessions used to update replicas can also
	case RM_NULLSEC:		be left in place continuously, so that changes to the master can be
	void;		reflected on the replicas in near-real-time.
	default:
	void;
	};
	2.2. Session, message, file and checkpoint IDs		The SEND RPC has been optimized by permitting an array of data and
			metadata updates to be sent in one RPC call, while the response
			permits the source server to know how far the destination got in
			applying the updates.
	These IDs are common to many messages. All are simple 64-bit		2. Common data types
	quantities except for RMcheckpoint, which is adds a time so that the
	earliest checkpoint can be chosen; the id field is chosen such that
	the source server can easily use it to restart an aborted session.
	RMsession_id is chosen at the pleasure of the source server.
	Title A Replication/Migration Protocol October 2002		2.1. Session, file and checkpoint IDs
	RMmessage_id is a monotonically increasing message count chosen by		RMsession_id permits multiplexing transfer sessions on a single
	the source server. RMfile_id is intended to be identical to the		authenticated connection; the value is chosen arbitrarily by the
	NFSv4 fileid attribute.		source server. RMcheckpoint is used to track the last RPC known to
			the destination so that restart can be done; a timestamp is supplied
			to help choose the earliest checkpoint. RMfile_id is intended to be
			identical to the NFSv4 fileid attribute.
	typedef uint64_t RMsession_id;		typedef uint64_t RMsession_id;
	typedef uint64_t RMmessage_id;
	typedef uint64_t RMfile_id;		typedef uint64_t RMfile_id;
	struct RMcheckpoint {		struct RMcheckpoint {
	nfstime4 time;		nfstime4 time;
	uint64_t id;		uint64_t id;
	};		};
	2.3. Offset, length and cookies		2.2. Offset, length and cookies
	These variables are chosen for compatibility with NFSv4.		These variables are chosen for compatibility with NFSv4.
	typedef uint64_t RMoffset;		typedef uint64_t RMoffset;
	typedef uint64_t RMlength;		typedef uint64_t RMlength;
	typedef uint64_t RMcookie;		typedef uint64_t RMcookie;
	2.4. General status		2.3. General status
	Status messages in OPEN_SESSION_DENY and ABORT_SESSION shall return a		Status responses for OPEN_SESSION and SEND responses and
	value from this set.		CLOSE_SESSION reasons shall return a value from this set.
			Title A Replication/Migration Protocol May 2003
	enum RMstatus {		enum RMstatus {
	RM_OK = 0,		RM_OK = 0,
	RMERR_PERM = 1,		RMERR_PERM = 1,
	RMERR_IO = 5,		RMERR_IO = 5,
	RMERR_EXISTS = 17		RMERR_EXISTS = 17
	};		};
	2.5. From NFS Version 4 [RFC3010]		2.4. From NFS Version 4 [RFC3530]
	The following definitions are imported from NFS Version 4.		The following definitions are imported from NFS Version 4.
	typedef uint32_t bitmap4<>;		typedef uint32_t bitmap4<>;
			typedef opaque attrlist4<>;
	typedef opaque utf8string<>;		typedef opaque utf8string<>;
			typedef opaque utf8str_mixed<>;
			typedef opaque utf8str_cis<>;
	struct nfstime4 {		struct nfstime4 {
	int64_t seconds;		int64_t seconds;
	uint32_t nseconds;		uint32_t nseconds;
	};		};
	Title A Replication/Migration Protocol October 2002
	enum nfs_ftype4 {		enum nfs_ftype4 {
	NF4REG = 1, /* Regular File */		NF4REG = 1, /* Regular File */
	NF4DIR = 2, /* Directory */		NF4DIR = 2, /* Directory */
	NF4BLK = 3, /* Special File - block device */		NF4BLK = 3, /* Special File - block device */
	NF4CHR = 4, /* Special File - character device */		NF4CHR = 4, /* Special File - character device */
	NF4LNK = 5, /* Symbolic Link */		NF4LNK = 5, /* Symbolic Link */
	NF4SOCK = 6, /* Special File - socket */		NF4SOCK = 6, /* Special File - socket */
	NF4FIFO = 7, /* Special File - fifo */		NF4FIFO = 7, /* Special File - fifo */
	NF4ATTRDIR = 8, /* Attribute Directory */		NF4ATTRDIR = 8, /* Attribute Directory */
	NF4NAMEDATTR = 9 /* Named Attribute */		NF4NAMEDATTR = 9 /* Named Attribute */
	};		};
			typedef uint32_t acetype4;
			typedef uint32_t aceflag4;
			typedef uint32_t acemask4;
	struct nfsace4 {		struct nfsace4 {
	acetype4 type;		acetype4 type;
	aceflag4 flag;		aceflag4 flag;
	acemask4 access_mask;		acemask4 access_mask;
	utf8string who;		utf8string who;
	};		};
	typedef nfsace4 fattr4_acl<>;		typedef nfsace4 fattr4_acl<>;
	typedef nfs_acl fattr4_acl;
			Title A Replication/Migration Protocol May 2003
	struct fattr4 {		struct fattr4 {
	bitmap4 attrmask;		bitmap4 attrmask;
	attrlist4 attr_vals;		attrlist4 attr_vals;
	};		};
	typedef nfs_attr fattr4;
	const OPEN4_SHARE_ACCESS_READ = 0x00000001;
	const OPEN4_SHARE_ACCESS_WRITE = 0x00000002;
	const OPEN4_SHARE_ACCESS_BOTH = 0x00000003;
	const OPEN4_SHARE_DENY_NONE = 0x00000000;
	const OPEN4_SHARE_DENY_READ = 0x00000001;
	const OPEN4_SHARE_DENY_WRITE = 0x00000002;
	const OPEN4_SHARE_DENY_BOTH = 0x00000003;
	3. Transfer protocol phases
	The servers using the protocol can be in the following phases:		3. Session Management
	o Initiation Phase: authentication, negotiation, filesystem info		Security flavors supported by the destination server may be known in
			advance, or may be discovered via an initial NULL RPC call which uses
			SNEGO GSS-API pseudo-mechanism as defined in [RFC2478]. A security
			flavor normally does not change through the life of the session.
	o Restart Phase [initiate when restarting]: add restart info to		A transfer session is created or resumed with the OPEN_SESSION call
	the above		and terminated normally or abnormally with the CLOSE_SESSION call.
			This is simpler than the previous draft of this protocol. The
			OPEN_SESSION call permits negotiation of capabilities and of the
			checkpoint to be used for a restart, while CLOSE_SESSION permits
			abnormal as well as normal termination.
	o Data Transfer Phase: send attributes and data for each file		3.1. Capabilities negotiation
	o Termination Phase: final handshake		Parameters in the OPEN_SESSION call express certain capabilities of
			the source server and provide an indication of properties of the data
			to be transferred. The destination server is responsible for
			reacting to these capabilities. If the desired capabilities are not
			acceptable to the destination, the response can bid down capabilities
			by clearing capabilities bits, or reject the session by failing the
			RPC. If the lowered capabilities bid by the destination server are
			not acceptable to the source server, the session should be terminated
			with CLOSE_SESSION.
	For simplicity, the protocol merges initiation and restart phases.		Currently, only three capabilities are specified; we expect to add
			more through working group effort. Specified so far are the
			following:
	Title A Replication/Migration Protocol October 2002		o RM_UTF8NAMES - source server supports and expects to send
			filenames encoded in UTF-8 format. If the destination server
			does not support UTF-8 filenames, it should convey that by
			clearing the flag.
	4. Initiation/restart phase		o RM_FHPRESERVE - source server is willing to attempt to preserve
			filehandles by sending them as part of each SEND_METADATA
			operation. If the destination can issue filehandles which it
			did not generate, and can work with the filehandle format used
			by the implementation identified by RMimplementation field in
			the OPEN_SESSION arguments, it can accept this offer; otherwise
			it should clear the bit to indicate refusal. Since the source
	4.1. Initiation/restart phase messages		Title A Replication/Migration Protocol May 2003
	The following messages are used to set up and authenticate a new		server may be denied in attempting to preserve filehandles, it
	transfer session:		should either refuse to transfer data if the destination clears
			this flag, or should advise clients of the possibility that
			filehandles will change via the [RFC3530] FH4_VOL_MIGRATION bit.
	o OPEN_SESSION_NEW - create new transfer session		o RM_FILEID - in combination with RM_FHPRESERVE, the source server
			is willing to attempt to preserve file_ids as well. If the
			destination can issue file_ids which it did not generate, and
			can work with the file_id format used by the implementation
			identified by RMimplementation field in the OPEN_SESSION
			arguments, it can accept this offer; otherwise it should clear
			the bit to indicate refusal.
	o OPEN_SESSION_RESUME - resume previous transfer session at		3.2. Security Negotiation
	checkpoint
	o OPEN_SESSION_CONFIRM - accept transfer session		Security for this protocol is provided by the RPCSEC_GSS mechanism,
			defined in [RFC2203], with the same GSS-API mechanisms defined as
			mandatory-to-implement as [RFC3530], namely the Kerberos V5 and
			LIPKEY mechanisms defined in [RFC1964] and [RFC2847]. In the case of
			a client and server implementing more than one of these mechanisms,
			the first RPC call should be an RPC NULL procedure call with the
			RPCSEC_GSS auth flavor and the SNEGO GSS-API mechanism populated with
			the mechanisms acceptable to the client. The server should respond
			with the preferred mechanism, if any, and this mechanism will be used
			for all sessions on this connection.
	o OPEN_SESSION_DENY - decline transfer session		3.3. OPEN_SESSION call
	4.2. Initiation/restart phase overview		SYNOPSIS
	The source server initiates a session by sending OPEN_SESSION_NEW		OPEN_SESSIONargs -> OPEN_SESSIONres
	(for a new transfer) or OPEN_SESSION_RESUME (to resume an old
	transfer). The destination server responds with either
	OPEN_SESSION_CONFIRM to permit a session or OPEN_SESSION_DENY to
	refuse a session.
	4.3. Capabilities negotiation		ARGUMENT
	The OPEN_SESSION_NEW and OPEN_SESSION_RESUME messages express		struct RMnewsession {
	capabilities of the source server and provide an indication of		utf8string src_path;
	properties of the data to be transferred. The destination server is		utf8string dest_path;
	responsible for reacting to these capabilities. If the desired		uint64_t fs_size;
	capabilities are an issue, it can respond with OPEN_SESSION_DENY to		uint64_t tr_size;
	refuse a session or it can respond with OPEN_SESSION_CONFIRM with		uint64_t tr_objs;
	unsupportable capability bits cleared to bid down. If the lowered		};
	capabilities are not acceptable to the source server, the session
	should be terminated with ABORT_SESSION.
	4.4. OPEN_SESSION_NEW message		struct RMoldsession {
			RMcheckpoint check_id;
			uint64_t rem_size;
			uint64_t rem_objs;
			};
	SYNOPSIS		Title A Replication/Migration Protocol May 2003
	enum RMcomp_type {		union RMopeninfo switch (bool new) {
	RM_NULLCOMP = 0,		case TRUE:
	RM_COMPRESS = 1,		RMnewsession newinfo;
	RM_ZIP = 2		case FALSE:
			RMoldsession oldinfo;
	};		};
	Title A Replication/Migration Protocol October 2002
	typedef uint64_t RMcapability;		typedef uint64_t RMcapability;
	const RM_UTF8NAMES = 0x00000001;		typedef utf8str_cis RMimplementation<>;
	const RM_FHPRESERVE = 0x00000002;
	struct OPEN_SESSION_NEW {		struct OPEN_SESSIONargs {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
	utf8string src_path;
	utf8string dest_path;
	uint64_t fs_size;
	uint64_t tr_size;
	uint64_t tr_objs;
	RMcomp_type comp_list<>;		RMcomp_type comp_list<>;
	RMcapability capabilities;		RMcapability capabilities;
			RNimplementation implementation;
			RMopeninfo info;
	};		};
	OPEN_SESSION_NEW is a proposal to create a transfer session to send		RESULT
	the full or incremental contents of one filesystem. The session_id
	is a unique number assigned by the source server to the transfer
	session. src_path is the full path name to the filesystem on the
	source server, and dest_path is the full path name to the filesystem
	on the destination. fs_size and tr_size are the approximate total
	size of the filesystem data and the amount to be sent during this
	transfer session. tr_objs is the approximate number of objects to be
	sent or updated in this transfer session. comp_list is a list of
	compression types the source server can use to compress data.
	capabilities is the bitmask used to negotiate as described in Section
	4.3.
	4.5. OPEN_SESSION_RESUME message		struct RMopenok {
			RMcheckpoint check_id;
			RMcomp_type comp_alg;
			RMcapability capabilities;
			};
	SYNOPSIS		union RMopenresp switch (RMstatus status) {
			case RM_OK:
			RMopenok info;
			default:
			void;
			};
	struct OPEN_SESSION_RESUME {		struct OPEN_SESSIONres {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
	RMcheckpoint check_id;		RMopenresp response;
	uint64_t rem_size;
	uint64_t rem_objs;
	RMcomp_type comp_list<>;
	RMcapability capabilities;
	};		};
	OPEN_SESSION_RESUME is a proposal to resume a transfer session which		OPEN_SESSION is a request to create or resume a transfer session to
	was not previously completed. It sends a checkpoint ID which is for		send the full or incremental contents of one filesystem. For either
	the last message it believes it successfully sent. If the		new or resuming sessions, the source server supplies the following
	destination has a checkpoint with an earlier timestamp, it will reply		information:
	with that checkpoint ID as an alternate starting point. The
	Title A Replication/Migration Protocol October 2002		o session_id - a unique number assigned by the source server to
			the transfer session, or the number of the session to be
			resumed.
	approximate remaining number of bytes to transfer and objects to		Title A Replication/Migration Protocol May 2003
	update are passed in rem_size and rem_objs. Other parameters are as
	defined in OPEN_SESSION_NEW.
	4.6. OPEN_SESSION_CONFIRM message		o comp_list - a list of compression types the source server can
			use to compress data.
			o capabilities - the bitmask used to negotiate as described in
			Section 4.3.
			o implementation - a descriptor of the operating system and
			filesystem implementation, with version information, used by the
			source server; this is to permit preservation of filehandles and
			fileids if the destination server runs a compatible version.
			This field is constructed at the pleasure of the source server
			and need only be parsed properly by a destination server running
			the same operating system code.
			For new sessions, the source server supplies the following
			information:
			o src_path - full path name to the filesystem on source server
			o dest_path - full path name to the filesystem on the destination
			server
			o fs_size - total size of the filesystem data
			o tr_size - amount of filesystem data to be sent during this
			transfer session
			o tr_objs - number of objects to be sent or updated in this
			transfer session
			For resuming sessions, the source server supplies the following
			information:
			o check_id - checkpoint ID for the last RPC believed sent
			o rem_size - remaining amount of filesystem data to be sent
			o rem_objs - remaining number of objects to be sent or updated
			The response from the destination server may reject the session
			proposal with an error code, may accept the proposal outright, or may
			bid down capabilities or state that it needs to start from an earlier
			checkpoint than that proposed by the source. The destination will
			also choose a compression algorithm from the list the source
			provided. The source may issue a CLOSE_SESSION call if capabilities
			negotiated down are not acceptable to it. Once the OPEN_SESSION RPC
			has been completed, SEND RPCs with data transfer operations will be
			sent until a CLOSE_SESSION RPC is sent.
			Title A Replication/Migration Protocol May 2003
			3.4. CLOSE_SESSION call
	SYNOPSIS		SYNOPSIS
	struct OPEN_SESSION_CONFIRM {		CLOSE_SESSIONargs -> CLOSE_SESSIONres
	RMsec_token sec_token;
	RMsession_id session_id;		ARGUMENT
			struct RMbadclose {
	RMcheckpoint check_id;		RMcheckpoint check_id;
	RMcomp_type comp_alg;		bool_t restartable;
	RMcapability capabilities;
	};		};
	OPEN_SESSION_CONFIRM is used by the destination server to agree to		union RMcloseinfo switch (RMstatus status) {
	open a transfer session and agree with or bid down the capabilities		case RM_OK:
	proposed by the source server, and to choose a compression algorithm.		void;
	Once the session has been confirmed, data transfer messages will be		default:
	sent until a CLOSE_SESSION or ABORT_SESSION message is sent.		RMbadclose info;
			};
	4.7. OPEN_SESSION_DENY message		struct CLOSE_SESSIONargs {
			RMsession_id session_id;
			RMcloseinfo info;
			};
	SYNOPSIS		RESULT
	struct OPEN_SESSION_DENY {		struct CLOSE_SESSIONres {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
	RMstatus status;		RMcheckpoint check_id;
	};		};
	OPEN_SESSION_DENY is used by the destination server to reject an		CLOSE_SESSION is used to terminate the session normally or abnormally
	OPEN_SESSION_NEW or OPEN_SESSION_RESUME proposal for any reason. The		by the source server.
	reason will be expressed by the status code.
	5. Data transfer phase		A normal close is handled by setting the RMcloseinfo status to RM_OK.
			Upon a normal close, a migration event is considered complete and the
			source will begin to refer clients to the destination server.
	5.1. Data transfer phase messages		An abnormal close is handled by setting the status to something other
			than RM_OK and supplying the last checkpoint the source server
			believes it sent plus an indication of whether it is possible to
			restart the transfer from that checkpoint. The destination server
			responds with the last checkpoint it has successfully committed. The
			destination server should attempt to save the state of the aborted
			session for a period of at least one hour.
	The following messages are used to transfer filesystem data during a		Title A Replication/Migration Protocol May 2003
	transfer session:
	Title A Replication/Migration Protocol October 2002		4. Data transfer
	o SEND_OBJECT_METADATA - send metadata about object		4.1. Data transfer operations
			Data transfer is accomplished by the SEND RPC, which takes an array
			of unions to permit a variety of transfer operations to be sent in
			each RPC. All operations must pertain to one filesystem object,
			since the RMfile_id is provided for each SEND RPC, not for each
			operation. Each operation in the array has an RMstatus in the
			response, so the source server can track how much was done if the
			call failed. Processingn stops at the first failure, and the SEND
			RPC response status is set to the first failure status.
			The following transfer operations are supported:
			o SEND_METADATA - send metadata about object
	o SEND_FILE_DATA - send file data		o SEND_FILE_DATA - send file data
			o SEND_FILE_HOLE - send file data
	o SEND_LOCK_STATE - send file lock state		o SEND_LOCK_STATE - send file lock state
	o SEND_SHARE_STATE - send share modes state		o SEND_SHARE_STATE - send share modes state
			o SEND_DELEG_STATE - send delegation state
	o SEND_REMOVE - send an object removal transaction		o SEND_REMOVE - send an object removal transaction
	o SEND_RENAME - send an object rename transaction		o SEND_RENAME - send an object rename transaction
	o SEND_DIRECTORY_CONTENTS - send names of objects in a directory		o SEND_LINK - send an object link transaction
	o SEND_CHECKPOINT - send checkpoint info		o SEND_SYMLINK - send an object symlink transaction
	o CLOSE_OBJECT - signal completion of object		o SEND_DIR_CONTENTS - send names of objects in a directory
	o CONFIRM_MESSAGE - confirm any data transfer message		o SEND_CLOSE - signal completion of object
	5.2. Data transfer phase overview		4.2. Data transfer phase overview
	The source server begins processing filesystem objects in some fixed		The source server processes filesystem objects in some known order
	order which will permit checkpointing and restarting in case of some		which will permit checkpointing and restarting in case of some
	problem or operator abort. SEND_OBJECT_METADATA is sent first, then		problem or operator abort. Full transfers should be done in order
	SEND_FILE_DATA messages will be sent for non-directory objects. If		such that objects which are needed, such as directories and link
	outstanding lock state for an onject exists on the source server, it		targets, are present when referrals are made to them. Incremental
	will be sent via SEND_LOCK_STATE messages; SEND_SHARE_STATE does the
	equivalent for share modes state.
	Ideally, the source server will track all filesystem changes, and		Title A Replication/Migration Protocol May 2003
	will be able to reflect remove and rename changes via SEND_REMOVE and
	SEND_RENAME messages. If the source server cannot capture all create
	and remove operations on a directory reliably,
	SEND_DIRECTORY_CONTENTS will permit the destination server to list
	its directory entries so that the destination can compute what items
	should be removed.
	Named attributes are handled with SEND_OBJECT_METADATA messages with		transfers should be done in the order changes were made on the source
	IS_NAMED_ATTR set to true, and apply to the previous non-named-		server, if possible; if not possible, the order described for full
	attribute which was handled. CLOSE_OBJECT is used to indicate that		transfers is acceptable.
	all data and named attributes of an object have been transferred. At
	any time, the source server may set a checkpoint with
	SEND_CHECKPOINT. All messages are confirmed by the destination
	server with CONFIRM_MESSAGE.
	Title A Replication/Migration Protocol October 2002		For files which are to be created or updated, SEND_METADATA is sent
			first, then SEND_FILE_DATA operations will be sent. If outstanding
			lock, share or delegation state for an object exists on the source
			server, it will be sent via SEND_LOCK_STATE, SEND_SHARE_STATE or
			SEND_DELEG_STATE operations after all data has been transferred.
			SEND_CLOSE is used to signal that all changes to a file are complete.
			Directories are created with SEND_METADATA, but are not populated
			until its objects are created, so the SEND_METADATA is followed by
			SEND_CLOSE.
	5.3. SEND_OBJECT_METADATA message		Ideally, the source server will track all filesystem changes via a
			mechanism such as [DMAPI], and will be able to reflect remove, rename
			and link changes via SEND_REMOVE, SEND_RENAME and SEND_LINK
			operations. If the source server cannot capture all create and
			remove operations on a directory reliably, SEND_DIR_CONTENTS should
			be used. This operation lists all directory entries for a source
			server, so that the destination server can compute what items should
			be removed. This is less reliable than being able to send
			SEND_REMOVE, SEND_RENAME and SEND_LINK operations, and should be used
			only when the underlying filesystem cannot record changes as they
			happen.
			Named attributes for a filesystem object are handled with
			SEND_METADATA operations with file type NF4NAMEDATTR. This will be
			"nested", i.e. it will be understood that the named attribute is
			associated with the parent object handled. SEND_CLOSE is used to
			indicate that all data and metadata of the named attribute have been
			transferred, and must be issued before another named attribute can be
			handled and before the SEND_CLOSE for the parent object is issued.
			Named attributes may not themselves have named attributes.
			4.3. SEND call
	SYNOPSIS		SYNOPSIS
	enum RMattrtype {		SENDargs -> SENDres
	RM_NFS_ATTR = 0,
	RM_CIFS_ATTR = 1
	};
	union RMattrs switch (RMattrtype type) {		ARGUMENT
	case RM_NFS_ATTR:
	nfs_attr attr;		union RMsendargs switch (RMoptype sendtype) {
	case RM_CIFS_ATTR:		case OP_SEND_METADATA:
	void;		SEND_METADATA metadata;
	default:		case OP_SEND_FILE_DATA:
			SEND_FILE_DATA data;
			Title A Replication/Migration Protocol May 2003
			case OP_SEND_FILE_HOLE:
			SEND_FILE_HOLE hole;
			case OP_SEND_LOCK_STATE:
			SEND_LOCK_STATE lock;
			case OP_SEND_SHARE_STATE:
			SEND_SHARE_STATE share;
			case OP_SEND_DELEG_STATE:
			SEND_DELEG_STATE deleg;
			case OP_SEND_REMOVE:
			SEND_REMOVE remove;
			case OP_SEND_RENAME:
			SEND_RENAME rename;
			case OP_SEND_LINK:
			SEND_LINK link;
			case OP_SEND_SYMLINK:
			SEND_SYMLINK symlink;
			case OP_SEND_DIR_CONTENTS:
			SEND_DIR_CONTENTS dirc;
			case OP_SEND_CLOSE:
	void;		void;
	};		};
	struct SEND_OBJECT_METADATA {		struct SEND1args {
	RMsec_token sec_token;		RMsession_id session_id;
	RMmessage_id msg_id;		RMcheckpoint check_id;
	RMfile_id file_id;		RMfile_id file_id;
			RMsendargs sendarray<>;
			};
			RESULT
			union RMsendres switch (RMoptype sendtype) {
			case OP_SEND_METADATA:
			case OP_SEND_FILE_DATA:
			case OP_SEND_FILE_HOLE:
			case OP_SEND_LOCK_STATE:
			case OP_SEND_SHARE_STATE:
			case OP_SEND_DELEG_STATE:
			case OP_SEND_REMOVE:
			case OP_SEND_RENAME:
			case OP_SEND_LINK:
			case OP_SEND_SYMLINK:
			case OP_SEND_DIR_CONTENTS:
			case OP_SEND_CLOSE:
			RMstatus status;
			};
			Title A Replication/Migration Protocol May 2003
			struct SEND1res {
			RMsession_id session_id;
			RMcheckpoint check_id;
			RMfile_id file_id;
			RMsendres resarray<>;
			RMstatus status;
			};
			The SEND RPC batches data transfer operations together and sends them
			to the destination server to operate on one file and with one
			checkpoint. The destination server may fail a call in the middle of
			the array by setting the return status for that operation to
			something other than RM_OK, and will not process further operations.
			The call will be failed with that status as well.
			4.4. Data transfer operation description
			4.4.1. SEND_METADATA operation
			SYNOPSIS
			struct SEND_METADATA {
	utf8string obj_name;		utf8string obj_name;
	nfs_ftype4 obj_type;
	RMattrs attrs;		RMattrs attrs;
	nfs_acl obj_acl;
	bool is_named_attr;
	};		};
	SEND_OBJECT_METADATA announces that we are about to transfer		SEND_METADATA announces that we are about to transfer information
	information about a particular filesystem object. If an object does		about a particular filesystem object. If an object does not exist on
	not exist on the destination, it will be created with the given		the destination, it will be created with the given obj_name and
	obj_name, obj_type, attributes, ACL and file_id (if supported). If		attributes supplied. If the object exists and is is the correct
	the object exists and is is the correct type, its attributes and ACL		type, its attributes will be updated. If an object of the same name
	will be updated. If an object of the same name but a different type		but a different type exists, it will be removed and recreated with
	exists, it will be removed and recreated with this information. If a		this information. If a SEND_METADATA has not followed a SEND_CLOSE,
	SEND_OBJECT_METADATA has not followed a CLOSE_OBJECT, it may have the		it may have the is_named_attr flag set, in which case the object is a
	is_named_attr flag set, in which case the object is a named attribute		named attribute of the most recent object identified by a
	of the most recent object identified by a SEND_OBJECT_METADATA.		SEND_METADATA.
	Title A Replication/Migration Protocol October 2002
	5.4. SEND_FILE_DATA message		4.4.2. SEND_FILE_DATA operation
	SYNOPSIS		SYNOPSIS
	struct SEND_FILE_DATA {		struct SEND_FILE_DATA {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	RMoffset offset;		RMoffset offset;
	RMlength length;		RMlength length;
	bool is_hole;
	opaque data<>;		opaque data<>;
	};		};
	SEND_FILE_DATA sends a block of data for a regular file, or, if the		Title A Replication/Migration Protocol May 2003
	is_hole flag is set, an indication that a block of data has been
	zeroed. The range is identified by the offset, length pair as
	starting at seek position 'offset' and extending through
	'offset+length-1', inclusive.
	5.5. SEND_LOCK_STATE message		SEND_FILE_DATA sends a block of data for a regular file. The range
			is identified by the offset, length pair as starting at seek position
			'offset' and extending through 'offset+length-1', inclusive.
			4.4.3. SEND_FILE_HOLE operation
	SYNOPSIS		SYNOPSIS
	struct RMlock_desc {		struct SEND_FILE_HOLE {
	RMowner owner;
	RMoffset offset;		RMoffset offset;
	RMlength length;		RMlength length;
	};		};
			SEND_FILE_HOLE sends a description of a "hole", or a zero-filled and
			usually unallocated block of data. A source server which does sparse
			allocation and which can learn via APIs what parts of a file are
			unallocated can use this to describe the hole without transferring
			the block of zeros.
			4.4.4. SEND_LOCK_STATE operation
			SYNOPSIS
			enum RMlocktype {
			RM_NOLOCK = 0,
			RM_READLOCK = 1,
			RM_WRITELOCK = 2
			};
	struct SEND_LOCK_STATE {		struct SEND_LOCK_STATE {
	RMsec_token sec_token;		RMowner owner;
	RMmessage_id msg_id;		RMclientid clientid;
	RMfile_id file_id;		RMoffset offset;
	RMlock_desc lock_desc<>;		RMlength length;
			RMlocktype type;
			RMstateid id;
	};		};
	SEND_LOCK_STATE transfers ownership and range information about		SEND_LOCK_STATE transfers ownership and range information about
	outstanding byte-range locks to the destination server.		outstanding byte-range locks to the destination server. The lock
			stateid is transferred so that the client need not reestablish the
			lock after migration. RM_NOLOCK is included to support continuous
			replication by permitting locks on replicas to be cleared.
	5.6. SEND_SHARE_STATE message		4.4.5. SEND_SHARE_STATE operation
	SYNOPSIS		SYNOPSIS
	typedef uint32_t RMaccess;		typedef uint32_t RMaccess;
	typedef uint32_t RMdeny;
	Title A Replication/Migration Protocol October 2002		Title A Replication/Migration Protocol May 2003
	struct RMshare_desc {		typedef uint32_t RMdeny;
	RMowner owner;
	RMaccess mode;
	RMdeny mode;
	};
	struct SEND_SHARE_STATE {		struct SEND_SHARE_STATE {
	RMsec_token sec_token;		RMowner owner;
	RMmessage_id msg_id;		RMclientid client;
	RMfile_id file_id;		RMaccess accmode;
	RMshare_desc share_desc<>;		RMdeny denymode;
	};		};
	SEND_SHARE_STATE transfers ownership and mode information about		SEND_SHARE_STATE transfers ownership and mode information about
	outstanding share reservations to the destination server.		outstanding share reservations to the destination server.
	5.7. SEND_REMOVE message		4.4.6. SEND_DELEG_STATE operation
			SYNOPSIS
			enum RMdelegtype {
			RM_NODELEG = 0,
			RM_READDELEG = 1,
			RM_WRITEDELEG = 2
			};
			struct SEND_DELEG_STATE {
			RMclientid client;
			RMdelegtype type;
			RMstateid id;
			};
			SEND_DELEG_STATE transfers ownership and type information about
			outstanding file delegations to the destination server. RM_NODELEG
			is included to support continuous replication by permitting
			delegations on replicas to be cleared.
			4.4.7. SEND_REMOVE operation
	SYNOPSIS		SYNOPSIS
	struct SEND_REMOVE {		struct SEND_REMOVE {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	utf8string name;		utf8string name;
	};		};
	SEND_REMOVE documents a remove event on the object identified; upon		SEND_REMOVE documents a remove event on the object identified; upon
	receipt, the destination server will remove the object as well.		receipt, the destination server will remove the object as well.
	5.8. SEND_RENAME message		Title A Replication/Migration Protocol May 2003
			4.4.8. SEND_RENAME operation
	SYNOPSIS		SYNOPSIS
	struct SEND_RENAME {		struct SEND_RENAME {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	utf8string old_name;		utf8string old_name;
	utf8string new_name;		utf8string new_name;
	};		};
	SEND_RENAME documents a rename event on the object identified by		SEND_RENAME documents a rename event on the object identified by
	old_name; upon receipt, the destination server will rename the object		old_name; upon receipt, the destination server will rename the object
	as well.		in the destination filesystem. Full paths may be used relative to
			the root of the source filesystem.
	Title A Replication/Migration Protocol October 2002
	5.9. SEND_DIRECTORY_CONTENTS message
	SYNOPSIS
	struct SEND_DIRECTORY_CONTENTS {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	RMcookie cookie;
	bool eof;
	utf8string names<>;
	};
	SEND_DIRECTORY_CONTENTS is used to account for removals and renames
	when source servers cannot record the events such that they may be
	sent with SEND_REMOVE and SEND_RENAME. The contents are listed in no
	predictable order so that the destination can what entries it has
	which are no longer found on the source. Each
	SEND_DIRECTORY_CONTENTS includes an opaque directory cookie to
	represent starting location of the block on the server, and the eof
	flag is set on the last block. Any item existing on the destination
	that is not listed in a SEND_DIRECTORY_CONTENTS message will be
	removed.
	5.10. SEND_CHECKPOINT message		4.4.9. SEND_LINK operation
	SYNOPSIS		SYNOPSIS
	struct SEND_CHECKPOINT {		struct SEND_LINK {
	RMsec_token sec_token;		utf8string old_name;
	RMmessage_id msg_id;		utf8string new_name;
	RMcheckpoint check_id;
	};		};
	SEND_CHECKPOINT is used periodically by the source server to indicate		SEND_LINK documents the creation of a hard link from the old_name to
	a point from which a restart can be done. The destination server		the new_name; upon receipt, the destination server will link the
	will track the last checkpoint it has received and be prepared to		objects in the destination filesystem. Full paths may be used
	examine it upon restart.		relative to the root of the source filesystem.
	5.11. CLOSE_OBJECT message		4.4.10. SEND_SYMLINK operation
	SYNOPSIS		SYNOPSIS
	struct CLOSE_OBJECT {		struct SEND_SYMLINK {
	RMsec_token sec_token;		utf8string old_name;
	RMmessage_id msg_id;		utf8string new_name;
	RMfile_id file_id;
	};		};
	Title A Replication/Migration Protocol October 2002		SEND_SYMLINK documents the creation of a symbolic link from the
			old_name to the new_name; upon receipt, the destination server will
			symlink the objects in the destination filesystem. The old_name
			value is not checked in any way and can be arbitrary textual data.
	CLOSE_OBJECT is used to announce that all data and metadata changes		Title A Replication/Migration Protocol May 2003
	for a particular object have been completed.
	5.12. CONFIRM_MESSAGE message		4.4.11. SEND_DIR_CONTENTS operation
	SYNOPSIS		SYNOPSIS
	struct CONFIRM_MESSAGE {		struct SEND_DIR_CONTENTS {
	RMsec_token sec_token;		RMcookie cookie;
	RMmessage_id msg_id;		bool eof;
			utf8string names<>;
	};		};
	CONFIRM_MESSAGE is used by the destination server to acknowledge		SEND_DIR_CONTENTS is used to account for removals and renames when
	every data transfer message.		source servers cannot record the events such that they may be sent
			with SEND_REMOVE and SEND_RENAME. The contents are listed in no
	6. Termination phase		predictable order so that the destination can what entries it has
			which are no longer found on the source. Each SEND_DIR_CONTENTS
	6.1. Termination phase messages		includes an opaque directory cookie to represent starting location of
			the block on the source server, and the eof flag is set on the last
	The following messages are used to terminate a transfer session:		block. Any item existing on the destination that is not listed in a
			SEND_DIR_CONTENTS operation will be removed.
	o ABORT_SESSION - end transfer session before natural end
	o CLOSE_SESSION - complete transfer session normally
	6.2. Termination phase overview
	ABORT_SESSION may be used at any time by either server to terminate a
	session prematurely; a checkpoint ID is recommended to permit restart
	if possible. CLOSE_SESSION is the normal termination message, and
	must be issued by the source server first and then issued by the
	destination server as a confirmation.
	6.3. ABORT_SESSION message		4.4.12. SEND_CLOSE operation
	SYNOPSIS		SYNOPSIS
	struct ABORT_SESSION {		void;
	RMsec_token sec_token;
	RMsession_id session_id;
	RMcheckpoint check_id;
	RMstatus status;
	};
	ABORT_SESSION terminates a data transfer session immediately. If may
	be used by either the source or destination server, and records the
	Title A Replication/Migration Protocol October 2002
	last checkpoint known and a status code to explain the termination.		SEND_CLOSE is used to announce that all data and metadata changes for
			a particular object have been completed.
	6.4. CLOSE_SESSION message		5. IANA Considerations
	SYNOPSIS		The replication/migration protocol will use a well-known RPC program
			number at which destination servers will register. The author will
			acquire an RPC program number for this purpose.
	struct CLOSE_SESSION {		6. Security Considerations
	RMsec_token sec_token;
	RMsession_id session_id;
	};
	CLOSE_SESSION terminates a data transfer session normally; it is used		NFS Version 4 is the primary impetus behind a replication/migration
	first by the source and is then used by the destination server to		protocol, so this protocol should mandate a strong security scheme in
	confirm it.		a manner comparable with NFS Version 4. Implementations of this
			protocol MUST support the RPCSEC_GSS security flavor as defined in
			[RFC2203] and must also support the Kerberos V5 and LIPKEY mechanisms
			as defined in [RFC1964] and [RFC2847]. The particular mechanism
			chosen for sessions is determined by the use of SNEGO on the initial
			call, which should be a NULL RPC.
	Title A Replication/Migration Protocol October 2002		Title A Replication/Migration Protocol May 2003
	7. XDR protocol definition file		7. Appendix A: XDR Protocol Definition File
	/*		/*
	* Copyright (C) The Internet Society (1998,1999,2000,2001,2002).		* Copyright (C) The Internet Society (1998,1999,2000,2001,2002).
	* All Rights Reserved.		* All Rights Reserved.
	*/		*/
	/*		/*
	* repl-mig.x		* repl-mig.x
	*/		*/
	%#pragma ident "@(#)repl-mig.x 1.1"		%#pragma ident "@(#)repl-mig.x 1.4 03/05/27"
	/*
	* Derived types for clarity
	*/
	typedef int int32_t;
	typedef unsigned int uint32_t;
	typedef hyper int64_t;
	typedef unsigned hyper uint64_t;
	/*		/*
	* From RFC3010		* From RFC3530
	*/		*/
	typedef uint32_t bitmap4<>;		typedef uint32_t bitmap4<>;
			typedef opaque attrlist4<>;
	typedef opaque utf8string<>;		typedef opaque utf8string<>;
			typedef opaque utf8str_mixed<>;
			typedef opaque utf8str_cis<>;
	struct nfstime4 {		struct nfstime4 {
	int64_t seconds;		int64_t seconds;
	uint32_t nseconds;		uint32_t nseconds;
	};		};
	enum nfs_ftype4 {		enum nfs_ftype4 {
	NF4REG = 1, /* Regular File */		NF4REG = 1, /* Regular File */
	NF4DIR = 2, /* Directory */		NF4DIR = 2, /* Directory */
	NF4BLK = 3, /* Special File - block device */		NF4BLK = 3, /* Special File - block device */
	NF4CHR = 4, /* Special File - character device */		NF4CHR = 4, /* Special File - character device */
	NF4LNK = 5, /* Symbolic Link */		NF4LNK = 5, /* Symbolic Link */
	NF4SOCK = 6, /* Special File - socket */		NF4SOCK = 6, /* Special File - socket */
	NF4FIFO = 7, /* Special File - fifo */		NF4FIFO = 7, /* Special File - fifo */
	NF4ATTRDIR = 8, /* Attribute Directory */		NF4ATTRDIR = 8, /* Attribute Directory */
	NF4NAMEDATTR = 9 /* Named Attribute */		NF4NAMEDATTR = 9 /* Named Attribute */
	skipping to change at page 17, line 48		skipping to change at page 20, line 45
	NF4REG = 1, /* Regular File */		NF4REG = 1, /* Regular File */
	NF4DIR = 2, /* Directory */		NF4DIR = 2, /* Directory */
	NF4BLK = 3, /* Special File - block device */		NF4BLK = 3, /* Special File - block device */
	NF4CHR = 4, /* Special File - character device */		NF4CHR = 4, /* Special File - character device */
	NF4LNK = 5, /* Symbolic Link */		NF4LNK = 5, /* Symbolic Link */
	NF4SOCK = 6, /* Special File - socket */		NF4SOCK = 6, /* Special File - socket */
	NF4FIFO = 7, /* Special File - fifo */		NF4FIFO = 7, /* Special File - fifo */
	NF4ATTRDIR = 8, /* Attribute Directory */		NF4ATTRDIR = 8, /* Attribute Directory */
	NF4NAMEDATTR = 9 /* Named Attribute */		NF4NAMEDATTR = 9 /* Named Attribute */
	};		};
			typedef uint32_t acetype4;
			typedef uint32_t aceflag4;
			typedef uint32_t acemask4;
	struct nfsace4 {		struct nfsace4 {
	acetype4 type;		acetype4 type;
	aceflag4 flag;		aceflag4 flag;
	acemask4 access_mask;		acemask4 access_mask;
	utf8string who;
	};
	Title A Replication/Migration Protocol October 2002		Title A Replication/Migration Protocol May 2003
			utf8str_mixed who;
			};
	typedef nfsace4 fattr4_acl<>;		typedef nfsace4 fattr4_acl<>;
	typedef nfs_acl fattr4_acl;
	struct fattr4 {		struct fattr4 {
	bitmap4 attrmask;		bitmap4 attrmask;
	attrlist4 attr_vals;		attrlist4 attr_vals;
	};		};
	typedef nfs_attr fattr4;
	const OPEN4_SHARE_ACCESS_READ = 0x00000001;
	const OPEN4_SHARE_ACCESS_WRITE = 0x00000002;
	const OPEN4_SHARE_ACCESS_BOTH = 0x00000003;
	const OPEN4_SHARE_DENY_NONE = 0x00000000;
	const OPEN4_SHARE_DENY_READ = 0x00000001;
	const OPEN4_SHARE_DENY_WRITE = 0x00000002;
	const OPEN4_SHARE_DENY_BOTH = 0x00000003;
	/*
	* For security tokens
	*/
	enum RMsec_mode {
	RM_NULLSEC = 0,
	RM_PERMESSAGE = 1
	};
	struct RMsecpayload {
	uint32_t length;
	opaque contents<>;
	};
	union RMsec_token switch (RMsec_mode mode) {
	case RM_PERMESSAGE:
	RMsecpayload payload;
	case RM_NULLSEC:
	void;
	default:
	void;
	};
	/*		/*
	* For session, message, file and checkpoint IDs		* For session, message, file and checkpoint IDs
	*/		*/
	typedef uint64_t RMsession_id;		typedef uint64_t RMsession_id;
	typedef uint64_t RMmessage_id;
	typedef uint64_t RMfile_id;		typedef uint64_t RMfile_id;
	struct RMcheckpoint {		struct RMcheckpoint {
	nfstime4 time;		nfstime4 time;
	Title A Replication/Migration Protocol October 2002
	uint64_t id;		uint64_t id;
	};		};
	/*		/*
	* For compression algorithm negotiation		* For compression algorithm negotiation
	*/		*/
	enum RMcomp_type {		enum RMcomp_type {
	RM_NULLCOMP = 0,		RM_NULLCOMP = 0,
	RM_COMPRESS = 1,		RM_COMPRESS = 1,
	RM_ZIP = 2		RM_ZIP = 2
	};		};
	/*		/*
	* For capabilities negotiation		* For capabilities negotiation
	*/		*/
			typedef utf8str_cis RMimplementation<>;
	typedef uint64_t RMcapability;		typedef uint64_t RMcapability;
	const RM_UTF8NAMES = 0x00000001;		const RM_UTF8NAMES = 0x00000001;
	const RM_FHPRESERVE = 0x00000002;		const RM_FHPRESERVE = 0x00000002;
	/*		/*
	* For general status		* For general status
	*/		*/
	enum RMstatus {		enum RMstatus {
	RM_OK = 0,		RM_OK = 0,
	RMERR_PERM = 1,		RMERR_PERM = 1,
	RMERR_IO = 5,		RMERR_IO = 5,
	RMERR_EXISTS = 17		RMERR_EXISTS = 17
	};		};
			Title A Replication/Migration Protocol May 2003
	/*		/*
	* For generalized attributes		* Attributes
	*/		*/
	enum RMattrtype {		struct RMattrs {
	RM_NFS_ATTR = 0,		fattr4 attr;
	RM_CIFS_ATTR = 1		nfs_ftype4 obj_type;
	};		fattr4_acl obj_acl;
			bool is_named_attr;
	union RMattrs switch (RMattrtype type) {
	case RM_NFS_ATTR:
	nfs_attr attr;
	case RM_CIFS_ATTR:
	void;
	default:
	void;
	};		};
	/*		/*
	* Offset, length and cookies		* Offset, length and cookies
	Title A Replication/Migration Protocol October 2002
	*/		*/
	typedef uint64_t RMoffset;		typedef uint64_t RMoffset;
	typedef uint64_t RMlength;		typedef uint64_t RMlength;
	typedef uint64_t RMcookie;		typedef uint64_t RMcookie;
	/*		/*
	* Lock and share definitions		* Owner
	*/		*/
	struct RMlock_desc {		typedef utf8str_mixed RMowner;
	RMowner owner;
	RMoffset offset;		/*
	RMlength length;		* Lock and share supporting definitions
			*/
			struct RMclientid {
			utf8string name;
			opaque address<>;
			};
			struct RMstateid {
			uint32_t seqid;
			opaque other[12];
			};
			enum RMlocktype {
			RM_NOLOCK = 0,
			RM_READLOCK = 1,
			RM_WRITELOCK = 2
	};		};
	typedef uint32_t RMaccess;		typedef uint32_t RMaccess;
	typedef uint32_t RMdeny;		typedef uint32_t RMdeny;
	struct RMshare_desc {		enum RMdelegtype {
	RMowner owner;		RM_NODELEG = 0,
	RMaccess mode;		RM_READDELEG = 1,
	RMdeny mode;		RM_WRITEDELEG = 2
			Title A Replication/Migration Protocol May 2003
	};		};
	/*		/*
	* Protocol messages		* Protocol elements - session control
	*/		*/
	struct OPEN_SESSION_NEW {		struct RMnewsession {
	RMsec_token sec_token;
	RMsession_id session_id;
	utf8string src_path;		utf8string src_path;
	utf8string dest_path;		utf8string dest_path;
	uint64_t fs_size;		uint64_t fs_size;
	uint64_t tr_size;		uint64_t tr_size;
	uint64_t tr_objs;		uint64_t tr_objs;
	RMcomp_type comp_list<>;
	RMcapability capabilities;
	};		};
	struct OPEN_SESSION_RESUME {		struct RMoldsession {
	RMsec_token sec_token;
	RMsession_id session_id;
	RMcheckpoint check_id;		RMcheckpoint check_id;
	uint64_t rem_size;		uint64_t rem_size;
	uint64_t rem_objs;		uint64_t rem_objs;
	RMcomp_type comp_list<>;
	RMcapability capabilities;
	};		};
	Title A Replication/Migration Protocol October 2002		union RMopeninfo switch (bool new) {
			case TRUE:
			RMnewsession newinfo;
			case FALSE:
			RMoldsession oldinfo;
			};
	struct OPEN_SESSION_CONFIRM {		struct OPEN_SESSIONargs {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
			RMcomp_type comp_list<>;
			RMcapability capabilities;
			RNimplementation impl;
			RMopeninfo info;
			};
			struct RMopenok {
	RMcheckpoint check_id;		RMcheckpoint check_id;
	RMcomp_type comp_alg;		RMcomp_type comp_alg;
	RMcapability capabilities;		RMcapability capabilities;
	};		};
	struct OPEN_SESSION_DENY {		union RMopenresp switch (RMstatus status) {
	RMsec_token sec_token;		case RM_OK:
			RMopenok info;
			default:
			void;
			};
			struct OPEN_SESSIONres {
			Title A Replication/Migration Protocol May 2003
	RMsession_id session_id;		RMsession_id session_id;
	RMstatus status;		RMopenresp response;
	};		};
	struct SEND_OBJECT_METADATA {		struct RMbadclose {
	RMsec_token sec_token;		RMcheckpoint check_id;
	RMmessage_id msg_id;		bool_t restartable;
	RMfile_id file_id;		};
			union RMcloseinfo switch (RMstatus status) {
			case RM_OK:
			void;
			default:
			RMbadclose info;
			};
			struct CLOSE_SESSIONargs {
			RMsession_id session_id;
			RMcloseinfo info;
			};
			struct CLOSE_SESSIONres {
			RMsession_id session_id;
			RMcheckpoint check_id;
			};
			/*
			* Protocol elements - data transfer
			*/
			enum RMoptype {
			OP_SEND_METADATA = 1,
			OP_SEND_FILE_DATA = 2,
			OP_SEND_FILE_HOLE = 3,
			OP_SEND_LOCK_STATE = 4,
			OP_SEND_SHARE_STATE = 5,
			OP_SEND_DELEG_STATE = 6,
			OP_SEND_REMOVE = 7,
			OP_SEND_RENAME = 8,
			OP_SEND_LINK = 9,
			OP_SEND_SYMLINK = 10,
			OP_SEND_DIR_CONTENTS = 11,
			OP_SEND_CLOSE = 12
			};
			/*
			* Data and metadata send items
			*/
			struct SEND_METADATA {
			Title A Replication/Migration Protocol May 2003
	utf8string obj_name;		utf8string obj_name;
	nfs_ftype4 obj_type;
	RMattrs attrs;		RMattrs attrs;
	nfs_acl obj_acl;
	bool is_named_attr;
	};		};
	struct SEND_FILE_DATA {		struct SEND_FILE_DATA {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	RMoffset offset;		RMoffset offset;
	RMlength length;		RMlength length;
	bool is_hole;
	opaque data<>;		opaque data<>;
	};		};
			struct SEND_FILE_HOLE {
			RMoffset offset;
			RMlength length;
			};
	struct SEND_LOCK_STATE {		struct SEND_LOCK_STATE {
	RMsec_token sec_token;		RMowner owner;
	RMmessage_id msg_id;		RMclientid client;
	RMfile_id file_id;		RMoffset offset;
	RMlock_desc lock_desc<>;		RMlength length;
			RMlocktype type;
			RMstateid id;
	};		};
	struct SEND_SHARE_STATE {		struct SEND_SHARE_STATE {
	RMsec_token sec_token;		RMowner owner;
	RMmessage_id msg_id;		RMclientid client;
	RMfile_id file_id;		RMaccess accmode;
	RMshare_desc share_desc<>;		RMdeny denymode;
	};		};
	Title A Replication/Migration Protocol October 2002		struct SEND_DELEG_STATE {
			RMclientid client;
			RMdelegtype type;
			RMstateid id;
			};
	struct SEND_REMOVE {		struct SEND_REMOVE {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	utf8string name;		utf8string name;
	};		};
	struct SEND_RENAME {		struct SEND_RENAME {
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMfile_id file_id;
	utf8string old_name;		utf8string old_name;
	utf8string new_name;		utf8string new_name;
	};		};
	struct SEND_DIRECTORY_CONTENTS {		struct SEND_LINK {
	RMsec_token sec_token;		utf8string old_name;
	RMmessage_id msg_id;
	RMfile_id file_id;		Title A Replication/Migration Protocol May 2003
			utf8string new_name;
			};
			struct SEND_SYMLINK {
			utf8string old_name;
			utf8string new_name;
			};
			struct SEND_DIR_CONTENTS {
	RMcookie cookie;		RMcookie cookie;
	bool eof;		bool eof;
	utf8string names<>;		utf8string names<>;
	};		};
	struct SEND_CHECKPOINT {		/* no parameters for SEND_CLOSE */
	RMsec_token sec_token;
	RMmessage_id msg_id;
	RMcheckpoint check_id;
	};
	struct CLOSE_OBJECT {		union RMsendargs switch (RMoptype sendtype) {
	RMsec_token sec_token;		case OP_SEND_METADATA:
	RMmessage_id msg_id;		SEND_METADATA metadata;
	RMfile_id file_id;		case OP_SEND_FILE_DATA:
			SEND_FILE_DATA data;
			case OP_SEND_FILE_HOLE:
			SEND_FILE_HOLE hole;
			case OP_SEND_LOCK_STATE:
			SEND_LOCK_STATE lock;
			case OP_SEND_SHARE_STATE:
			SEND_SHARE_STATE share;
			case OP_SEND_DELEG_STATE:
			SEND_DELEG_STATE deleg;
			case OP_SEND_REMOVE:
			SEND_REMOVE remove;
			case OP_SEND_RENAME:
			SEND_RENAME rename;
			case OP_SEND_LINK:
			SEND_LINK link;
			case OP_SEND_SYMLINK:
			SEND_SYMLINK symlink;
			case OP_SEND_DIR_CONTENTS:
			SEND_DIR_CONTENTS dirc;
			case OP_SEND_CLOSE:
			void;
	};		};
	struct CONFIRM_MESSAGE {		union RMsendres switch (RMoptype sendtype) {
	RMsec_token sec_token;		case OP_SEND_METADATA:
	RMmessage_id msg_id;		case OP_SEND_FILE_DATA:
			case OP_SEND_FILE_HOLE:
			case OP_SEND_LOCK_STATE:
			Title A Replication/Migration Protocol May 2003
			case OP_SEND_SHARE_STATE:
			case OP_SEND_DELEG_STATE:
			case OP_SEND_REMOVE:
			case OP_SEND_RENAME:
			case OP_SEND_LINK:
			case OP_SEND_SYMLINK:
			case OP_SEND_DIR_CONTENTS:
			case OP_SEND_CLOSE:
			RMstatus status;
	};		};
	struct ABORT_SESSION {		struct SEND1args {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
	RMcheckpoint check_id;		RMcheckpoint check_id;
	RMstatus status;		RMfile_id file_id;
			RMsendargs sendarray<>;
	};		};
	Title A Replication/Migration Protocol October 2002		struct SEND1res {
	struct CLOSE_SESSION {
	RMsec_token sec_token;
	RMsession_id session_id;		RMsession_id session_id;
			RMcheckpoint check_id;
			RMfile_id file_id;
			RMsendres resarray<>;
			RMstatus status;
	};		};
	Title A Replication/Migration Protocol October 2002		program RM_PROGRAM {
			version RM_V1 {
	8. IANA Considerations		void
			RMPROC1_NULL(void) = 0;
	The replication/migration protocol will define a well-known port at		OPEN_SESSIONres
	which destination servers will listen for connection requests. The		RMPROC1_OPEN_SESSION(OPEN_SESSIONargs) = 1;
	author will apply to IANA for a port number for this purpose.		CLOSE_SESSIONres
			RMPROC1_CLOSE_SESSION(CLOSE_SESSIONargs) = 2;
	9. Security Considerations		SEND1res
			RMPROC1_SEND(SEND1args) = 3;
			} = 1;
			} = 100273;
	NFS Version 4 is the primary impetus behind a replication/migration		Title A Replication/Migration Protocol May 2003
	protocol, so this protocol should mandate a strong security scheme in
	a manner comparable with NFS Version 4. At the time of this draft,
	it is unclear whether this protocol should make use of a strong
	host-based security mechanism such as SSH and IPSec or strong per-
	message security based on GSS-API [RFC2078] tokens and mechanisms
	including Kerberos Version 5 used as described in [RFC1964] and
	LIPKEY as described in [RFC2847]. Pending further work in this area,
	this protocol draft defines a per-message security payload which may
	be NULL to permit prototyping, without specifying the messages for
	security negotiations and mechanism negotiation needed to use per-
	message security.
	Title A Replication/Migration Protocol October 2002		8. Normative References
	10. Normative References		[RFC1831]
			R. Srinivasan, "RPC: Remote Procedure Call Protocol Specification
			Version 2", RFC1831, August 1995.
	[RFC1832]		[RFC1832]
	R. Srinivasan, "XDR: External Data Representation Standard", RFC1832,		R. Srinivasan, "XDR: External Data Representation Standard", RFC1832,
	August 1995.		August 1995.
	[RFC3010]		[RFC1964]
			J. Linn, "Kerberos Version 5 GSS-API Mechanism", RFC1964, June 1996
			[RFC2203]
			M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol Specification",
			RFC2203, September 1997
			[RFC2478]
			E. Baize, D. Pinkas, "The Simple and Protected GSS-API Negotiation
			Mechanism", RFC2478, December 1998.
			[RFC2847]
			M. Eisler, "LIPKEY - A Low Infrastructure Public Key Mechanism Using
			SPKM", RFC2847, June 2000
			[RFC3530]
	S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M.		S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M.
	Eisler, D. Noveck, "NFS version 4 Protocol", RFC3010, December 2000.		Eisler, D. Noveck, "Network File System (NFS) Version 4 Protocol",
			RFC3530, April 2003.
	11. Informative References		Title A Replication/Migration Protocol May 2003
	[RFC1831]		9. Informative References
	R. Srinivasan, "RPC: Remote Procedure Call Protocol Specification
	Version 2", RFC1831, August 1995.
	[RDIST]		[RDIST]
	MagniComp, Inc., "The RDist Home Page",		MagniComp, Inc., "RDist Home Page", http://www.magnicomp.com/rdist.
	http://www.magnicomp.com/rdist.
	[RSYNC]		[RSYNC]
	The Samba Team, "The rsync web pages", http://samba.anu.edu.au/rsync.		The Samba Team, "rsync web pages", http://samba.anu.edu.au/rsync.
	[DESIGN]		[DESIGN]
	R. Thurlow, "Server-to-Server Replication/Migration Protocol Design		R. Thurlow, "Server-to-Server Replication/Migration Protocol Design
	Principles" (work in progress), http://www.ietf.org/internet-		Principles" (work in progress), http://www.ietf.org/internet-
	drafts/draft-thurlow-nfsv4-repl-mig-design-00.txt, June 2002.		drafts/draft-ietf-nfsv4-repl-mig-design-00.txt, December 2002.
	Title A Replication/Migration Protocol October 2002		[DMAPI]
			P. Lawthers, "The Data Management Applications Programming
			Interface",
			http://www.computer.org/conferences/mss95/lawthers/lawthers.htm, July
			1995.
	12. Author's Address		Title A Replication/Migration Protocol May 2003
			10. Author's Address
	Address comments related to this memorandum to:		Address comments related to this memorandum to:
	nfsv4-wg@sunroof.eng.sun.com		nfsv4-wg@sunroof.eng.sun.com
	Robert Thurlow		Robert Thurlow
	Sun Microsystems, Inc.		Sun Microsystems, Inc.
	500 Eldorado Boulevard, UBRM05-171		500 Eldorado Boulevard, UBRM05-171
	Broomfield, CO 80021		Broomfield, CO 80021
End of changes.
This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/