At 4Gb, a full packet is about 1 km long, and at 2Gb a full packet is about 2km long!  Yes, at any given time 2k of your data is spread from the port to 1km down the cable (as the light travels).  Each packet burns 1 buffer, no matter the size of the packet.  The buffer credit isn’t released until the packet gets to the receiving switch and the sending switch receives an acknowledgment.  So, at 4Gb, you need 2 buffer credit for each km of distance.  1 to fill the 1km pipe to the receiving switch, and 1 waiting for the acknowledgment.

So, for a 10 km fibre path, you need 20 buffer credits, assuming that every packet is a full packet.  Even if every data packet is full, there is some inter-switch communication that doesn’t fill a packet.  That means that the packets are shorter and take less time to transmit, meaning there are more packets in the pipe.  So, the 2 packets per km number is the MINIMUM number you’ll need. And, we’re not counting the time it takes to process the frames on the switches, which is probably pretty minimal.

Also, be aware that on a 48k, there are 1024 buffers per ASIC, out of which 8 buffers per port are reserved. So, if you put a bunch of long distance ISLs all on one ASIC, you might run in into problems because there aren’t enough buffers to go around. If you have a GoldenEye based switch, you only get 700 buffer credits per ASIC, out of which 8 buffers per port are reserved. Use portbuffershow to see your buffer usage per ASIC.

At 4Gb, a full packet is about 1 km long, and at 2Gb a full packet is about 2km long!\u00a0 Yes, at any given time 2k of your data is spread from the port to 1km down the cable (as the light travels).\u00a0 Each packet burns 1 buffer, no matter the size of the packet.\u00a0 The buffer credit isn’t released until the packet gets to the receiving switch and the sending switch receives an acknowledgment.\u00a0 So, at 4Gb, you need 2 buffer credit for each km of distance.\u00a0 1 to fill the 1km pipe to the receiving switch, and 1 waiting for the acknowledgment.<\/p>\n

So, for a 10 km fibre path, you need 20 buffer credits, assuming that every packet is a full packet.\u00a0 Even if every data packet is full, there is some inter-switch communication that doesn’t fill a packet.\u00a0 That means that the packets are shorter and take less time to transmit, meaning there are more packets in the pipe.\u00a0 So, the 2 packets per km number is the MINIMUM number you’ll need. And, we’re not counting the time it takes to process the frames on the switches, which is probably pretty minimal.<\/p>\n

Also, be aware that on a 48k, there are 1024 buffers per ASIC, out of which 8 buffers per port are reserved. So, if you put a bunch of long distance ISLs all on one ASIC, you might run in into problems because there aren’t enough buffers to go around. If you have a GoldenEye based switch, you only get 700 buffer credits per ASIC, out of which 8 buffers per port are reserved. Use portbuffershow to see your buffer usage per ASIC.<\/p>\n","protected":false},"excerpt":{"rendered":null,"protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,1],"tags":[],"class_list":["post-230","post","type-post","status-publish","format-standard","hentry","category-san-notes","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/posts\/230","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/comments?post=230"}],"version-history":[{"count":0,"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/posts\/230\/revisions"}],"wp:attachment":[{"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/media?parent=230"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/categories?post=230"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rootuser.ninja\/index.php\/wp-json\/wp\/v2\/tags?post=230"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}