Here we update the modular tcp to be able to switch to an

alternate TCP stack in other then the closed state (pre-listen/connect).
The idea is that *if* that is supported by the alternate stack, it
is asked if its ok to switch. If it approves the "handoff" then we
allow the switch to happen. Also the fini() function now gets a flag
to tell if you are switching away *or* the tcb is destroyed. The
init() call into the alternate stack is moved to the end so the
tcb is more fully formed before the init transpires.

Sponsored by:	Netflix Inc.
Differential Revision:	D6790

2 files changed, 51 insertions, 4 deletions

diff --git a/share/man/man4/tcp.4 b/share/man/man4/tcp.4
index f275a4da45f9..54f503f5c2c8 100644
--- a/share/man/man4/tcp.4
+++ b/share/man/man4/tcp.4
@@ -633,7 +633,8 @@ when trying to use a TCP function block that is not available;
 .Xr mod_cc 4 ,
 .Xr siftr 4 ,
 .Xr syncache 4 ,
-.Xr setkey 8
+.Xr setkey 8 ,
+.Xr tcp_functions 9
 .Rs
 .%A "V. Jacobson"
 .%A "R. Braden"
diff --git a/share/man/man9/tcp_functions.9 b/share/man/man9/tcp_functions.9
index 06c42ff0fa60..b998dd0dd7df 100644
--- a/share/man/man9/tcp_functions.9
+++ b/share/man/man9/tcp_functions.9
@@ -114,14 +114,17 @@ struct tcp_function_block {
 			    struct inpcb *inp, struct tcpcb *tp);
 	/* Optional memory allocation/free routine */
 	void	(*tfb_tcp_fb_init)(struct tcpcb *);
-	void	(*tfb_tcp_fb_fini)(struct tcpcb *);
+	void	(*tfb_tcp_fb_fini)(struct tcpcb *, int);
 	/* Optional timers, must define all if you define one */
 	int	(*tfb_tcp_timer_stop_all)(struct tcpcb *);
 	void	(*tfb_tcp_timer_activate)(struct tcpcb *,
 			    uint32_t, u_int);
 	int	(*tfb_tcp_timer_active)(struct tcpcb *, uint32_t);
 	void	(*tfb_tcp_timer_stop)(struct tcpcb *, uint32_t);
+	/* Optional functions */
 	void	(*tfb_tcp_rexmit_tmr)(struct tcpcb *);
+	void	(*tfb_tcp_handoff_ok)(struct tcpcb *);
+	/* System use */
 	volatile uint32_t tfb_refcnt;
 	uint32_t  tfb_flags;
 };
@@ -157,6 +160,16 @@ in the
 .Va tfb_tcp_fb_fini
 field.
 .Pp
+If the
+.Va tfb_tcp_fb_fini
+argument is non-NULL, the function to which it points is called when the
+kernel is destroying the TCP control block or when the socket is transitioning
+to use a different TCP stack.
+The function is called with arguments of the TCP control block and an integer
+flag.
+The flag will be zero if the socket is transitioning to use another TCP stack
+or one if the TCP control block is being destroyed.
+.Pp
 If the TCP stack implements additional timers, the TCP stack should set a
 non-NULL pointer in the
 .Va tfb_tcp_timer_stop_all ,
@@ -193,6 +206,37 @@ However, care must be taken to ensure the retransmit timer leaves the
 TCP control block in a valid state for the remainder of the retransmit
 timer logic.
 .Pp
+A user may select a new TCP stack before calling
+.Xr connect 2
+or
+.Xr listen 2 .
+Optionally, a TCP stack may also allow a user to begin using the TCP stack for
+a connection that is in a later state by setting a non-NULL function pointer in
+the
+.Va tfb_tcp_handoff_ok
+field.
+If this field is non-NULL and a user attempts to select that TCP stack after
+calling
+.Xr connect 2
+or
+.Xr listen 2
+for that socket, the kernel will call the function pointed to by the
+.Va tfb_tcp_handoff_ok
+field.
+The function should return 0 if the user is allowed to switch the socket to use
+the TCP stack. Otherwise, the function should return an error code, which will
+be returned to the user.
+If the
+.Va tfb_tcp_handoff_ok
+field is
+.Dv NULL
+and a user attempts to select the TCP stack after calling
+.Xr connect 2
+or
+.Xr listen 2
+for that socket, the operation will fail and the kernel will return
+.Er EINVAL .
+.Pp
 The
 .Va tfb_refcnt
 and
@@ -269,8 +313,10 @@ The
 .Fa blk
 argument references a function block that is not currently registered.
 .Sh SEE ALSO
-.Xr malloc 9 ,
-.Xr tcp 4
+.Xr connect 2 ,
+.Xr listen 2 ,
+.Xr tcp 4 ,
+.Xr malloc 9
 .Sh HISTORY
 This framework first appeared in
 .Fx 11.0 .