.svg)
It’s a bit of a mouthful but Data Centre Quantized Congestion Notification (DCQCN) is the coming together of Explicit Congestion Notification (ECN) and Priority Flow Control (PFC). It was originally developed by Microsoft and Mellanox for Ethernet to provide a lossless transport for RoCE. ECN is used initially to provide end to end regulation of congestion, with PFC providing a hop-by-hop failsafe to prevent overflowing buffers and packet loss. Following on from last week and CBFC on InfiniBand, I had intended to cover both ECN and PFC this week, but I think they each deserve their own post, so today I am focusing on ECN.
ECN uses Weighted Random Early Detection (WRED) which will monitor the length of the queue buffer. If a queue buffer reaches a minimum threshold, the switch will mark the ECN field in the packet header to indicate congestion to the receiver. Upon receipt of a packet with the ECN field marked, the receiver will generate Congestion Notification Packets (CNP) back towards the original sender. When the CNP arrives at the sender, the sender will throttle transmission, this will help to back off traffic and clear down the queue buffers on the switches within the fabric.
If congestion continues to build, then the WRED maximum threshold will be breached, and the switch will mark all packets with an ECN. Subsequently, the receiver will send many CNP packets, and the original sender will drop off transmission accordingly. As the queue buffers clear, the WRED value will decrease again, the ECN field will no longer be marked, and the receiver will stop sending CNPs. This will allow the sender to increase the rate of transmission.
In my diagram below, hosts A and B are sending traffic to host C. Leaf-03 is experiencing congestion and setting the ECN field in packets sent to host C, which is in turn generating CNPs towards senders A and B. The final thing that I think is worth noting is that this whole process can be triggered by a microburst and only last for a very short time interval (millisecond).
As usual, if I have missed anything or gotten anything wrong, just let me know.
