The transport layer is one of the key layers of the Internet protocol stack. It enrichs the network layer service to make it suitable for applications. Almost 40 years after its initial design, TCP remains the most widely used transport protocol. In the early 2000s, SCTP was proposed as an alternative to TCP. Despite a clean and extensible design and many useful features, it did not reach wide deployment. This failure is mainly caused by middleboxes. We'll describe their operation and explain why Multipath TCP, which is a backward compatible evolution to TCP, has better chances of being deployed. We'll explain the main principles behind Multipath TCP and the lessons that can be drawn from its design. We'll then analyse why Internet giants like Google and Microsoft now consider application-layer solutions like QUIC to replace standard protocols like TCP.
8. Performance issues
• TCP considered to be too complex by many
– Software implementation cannot cope with
increasing network bandwidth
• For high performance, transport should be
implemented in hardware
– Transputers
– Simpler transport protocols
9. More limitations of TCP
• Issues with the TCP pipe model
– Only supports a single bytestream
• Some applications need several streams with priorities
– No support for messages
– Connections are attached to one IP address on
client and one IP address on server
• No failover even if hosts have multiple interfaces
• No support for mobility
• No load balancing for multihomed hosts
11. SCTP in two slides
• Modern transport protocol
– Cleaner connection establishment
• Four-way handshake to counter SYN flooding attacks
– Cleaner protocol
• Flexible TLV packet format that is easy to extend
• Selective acknowledgements from the start
– Richer semantics
• Messages, multiple streams, unreliable delivery
• Advanced API to replace socket API
– Failover support
• Connection can move from one IP address to another one
13. What went wrong with SCTP ?
• Replacing a transport protocol
Physical
Datalink
Network
TCP
Application
SCTP
Applications must be
rewritten with new API
IP protocol=132
For SCTP packets
15. The Internet architecture
that we explain to our students
Physical
Datalink
Network
Transport
Application
O. Bonaventure, Computer networking : Principles, Protocols and Practice, open ebook, http://inl.info.ucl.ac.be/cnp3
Physical
Physical
Datalink
Physical
Datalink
Network
16. In reality
– almost as many middleboxes as routers
– various types of middleboxes are deployed
Sherry, Justine, et al. "Making middleboxes someone else's problem: Network processing as a cloud service."
Proceedings of the ACM SIGCOMM 2012 conference. ACM, 2012.
17. Internet devices according to Cisco
http://www.cisco.com/web/about/ac50/ac47/2.html
Web Security
Appliance
NAC Appliance
ACE XML
Gateway
Streamer
VPN Concentrator
SSL
Terminator
Cisco IOS Firewall
IP Telephony
Router
PIX Firewall
Right and Left
Voice
GatewayVVVV
Content
Engine
NAT
18. Middleboxes in the architecture
• In the official architecture, they do not exist
• In reality...
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
TCP
Physical
Datalink
Network
Transport
Application
19. TCP segments processed by a router
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
IP
TCP
20. TCP segments processed by a NAT
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
22. End-to-end transparency today
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Middleboxes don't change
the Protocol field, but
some discard packets with a
Protocol field different than
TCP or UDP
23. Agenda
• Internet transport protocols
– TCP
– SCTP
• Multipath TCP
– Basic principles
– Use cases
• What's next ?
– QUIC
24. Multipath TCP
• How can we efficiently use the multiple
interfaces that are available on today's hosts?
25. Design objectives
• Multipath TCP is an evolution of TCP
• Design objectives
– Support unmodified applications
– Work over today’s networks (IPv4 and IPv6)
– Works in all networks where regular TCP works
26. The Multipath TCP bytestream model
29
Client Server
ABCDEF...111232
0988989 ... XYZZ
IP:1.2.3.4
IP:4.5.6.7
IP:2.3.4.5 IP:6.7.8.9
BCD A
27. The Multipath TCP protocol
• Control plane
– How to manage a Multipath TCP connection that
uses several paths ?
• Data plane
– How to transport data ?
• Congestion control
– How to control congestion over multiple paths ?
29. A naïve Multipath TCP
In today's Internet ?
SYN+Option
SYN+ACK+Option
ACK
seq=123, "abc"
seq=126, "def"
There is no
corresponding
TCP connection
30. Design decision
– A Multipath TCP connection is composed of one or
more regular TCP subflows that are combined
• Each host maintains state that glues the TCP subflows
that compose a Multipath TCP connection together
• Each TCP subflow is sent over a single path and appears
like a regular TCP connection along this path
31. Multipath TCP and the architecture
Physical
Datalink
Network
Transport
Application Multipath TCP
TCP1
socket
TCP2 TCPn...
Application
A. Ford, C. Raiciu, M. Handley, S. Barre, and J. Iyengar, “Architectural guidelines for multipath TCP
development", RFC6182 2011.
No modification
to ease deployment
Multiple subflows
to cope with
middleboxes
32. A regular TCP connection
• What is a regular TCP connection ?
– It starts with a three-way handshake
• SYN segments may contain special options
– All data segments are sent in sequence
• There is no gap in the sequence numbers
– It is terminated by using FIN or RST
34. How to combine two TCP subflows ?
SYN+Option
SYN+ACK+Option
ACK
SYN+OtherOption
SYN+ACK+OtherOption
ACK
How to link with
blue subflow ?
35. TCP 101
Identification of a TCP connection
Four tuple
– IPsource
– IPdest
– Portsource
– Portdest
All TCP segments
contain the four
tuple
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
IP
TCP
36. How to link TCP subflows ?
SYN, Portsrc=1234,Portdst=80+Option
SYN+ACK[...]
ACK
SYN, Portsrc=1235,Portdst=80
+Option[link Portsrc=1234,Portdst=80]
A NAT could change
addresses and
port numbers
37. How to link TCP subflows ?
SYN, Portsrc=1234,Portdst=80
+Option[Token=5678]
SYN+ACK+Option[Token=6543]
ACK
SYN, Portsrc=1235,Portdst=80
+Option[Token=6543]
MyToken=5678
YourToken=6543
MyToken=6543
YourToken=5678
38. TCP subflows in practice
• Multipath TCP supports subflow agility
– Client/server can add subflows at any time
– Client/server can remove subflows at any time
39. The Multipath TCP protocol
• Control plane
– How to manage a Multipath TCP connection that
uses several paths ?
• Data plane
– How to transport data ?
• Congestion control
– How to control congestion over multiple paths ?
40. How to transfer data ?
seq=123,"a"
seq=124,"b"
seq=125,"c"
seq=126,"d"
ack=124
ack=126
ack=125
ack=127
41. How to transfer data
in today's Internet ?
seq=123,"a"
seq=124,"b"
seq=125,"c"
ack=124
ack=126
ack=125
Gap in sequence numbering space
Some DPI will not allow this !
42. Multipath TCP Data transfer
• Two levels of sequence numbers
Multipath TCP
TCP1
socket
TCP2
Multipath TCP
TCP1
socket
TCP2
ABCDEF
Data sequence #
TCP1 sequence #
TCP2 sequence #
44. Multipath TCP
How to deal with losses ?
• Data losses over one TCP subflow
– Fast retransmit and timeout as in regular TCP
Dseq=0,seq=123,"a"
DAck=1,ack=12
4Dseq=0,seq=123,"a"
DAck=1,ack=124
45. Multipath TCP
• What happens when a TCP subflow fails ?
Dseq=0,seq=123,"a"
DSeq=1, seq=456,"b"
DAck=0,ack=457
Dseq=0,seq=457,"a"
DAck=2,ack=458
46. The Multipath TCP protocol
• Control plane
– How to manage a Multipath TCP connection that uses
several paths ?
• Data plane
– How to transport data ?
• Congestion control
– How to control congestion over multiple paths ?
– Congestion windows on subflows MUST be coupled to
ensure that TCP remains fair with regular TCP
52. Agenda
• Internet transport protocols
– TCP
– SCTP
• Multipath TCP
– Basic principles
– Use cases
• What's next ?
– QUIC
53. Issues with the current stack
Physical
Datalink
IPv4/IPv6
TCP
HTTP1.1
ASCII difficult to
parse, no priority
Unsecure
Wait for three way
handshake before
data transfer
Physical
Datalink
IPv4/IPv6
TCP
HTTP/2
TLS
Secure,
But adds more delay
Physical
Datalink
IPv4/IPv6
UDP
QUICFirst bytes
After 2 RTTs
First bytes
After 3-4 RTTs First bytes
After 0 RTT
54. QUIC in a nutshell
• First connection attempt
CHLO [SNI, VER]
CHLO[Token, Crypto info]
ServerName and Version
Rejected
REJ[Config, Token, Certificate]
DATA[Encrypted]
SHLO[Config, Token, Certificate]
DATA[Encrypted]
55. QUIC features
• Congestion control
– Leverages TCP's long history (CUBIC)
• Retransmissions
– Better than with regular TCP
– Each segment has a different seqnum
• Avoids retransmission ambiguities
• Selective acknowledgements
– Cleaner than in TCP
56. QUIC usage at google
QUIC handshakes fail when RTTs are greater than 2.5 seconds or
when UDP is blocked
Source : J. Iyengar, QUIC Overview, IETF93, July 2015, Prague
57. Why running QUIC over UDP ?
• Simplest transport protocol
– Supported correctly by all operating systems
– Supported correctly by all middleboxes
• Application can entirely control everything
– Same version of QUIC runs on all platforms
– QUIC can be upgraded as frequently as the application
– Application developer does not need to coordinate
with IETF or anyone
58. How to cope with middleboxes ?
• Very few middleboxes interfere with UDP
– Some middleboxes drop UDP segments
• Applications will detect and fallback to TCP
– Some middleboxes rate limit UDP
• Applications will detect and fallback to TCP
• What about middleboxes optimising QUIC/UDP
– Nightmare for google
– Everything in QUIC (payload and headers) is
encrypted
59. Internet transport layer
• Still lots of innovation for an old layer…
– TCP extensions
• Initial window, TCP Fast Open, …
– Multipath TCP is getting deployed
• RFC6824 was published in January 2013
– But Middleboxes have ossified the Internet
• Other protocols
– QUIC
• Pushed by google for web applications
– TCPINC
• Support encryption inside transport layer
– TLS 1.3
• Faster handshake and lower delays