1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.6.1">
<meta name="author" content="Garrett D'Amore">
<title>ZeroTier Mapping for Scalability Protocols</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
<style>
/* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */
/* Remove comment around @import statement below when using as a custom stylesheet */
/*@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700";*/
article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block}
audio,canvas,video{display:inline-block}
audio:not([controls]){display:none;height:0}
[hidden],template{display:none}
script{display:none!important}
html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%}
a{background:transparent}
a:focus{outline:thin dotted}
a:active,a:hover{outline:0}
h1{font-size:2em;margin:.67em 0}
abbr[title]{border-bottom:1px dotted}
b,strong{font-weight:bold}
dfn{font-style:italic}
hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0}
mark{background:#ff0;color:#000}
code,kbd,pre,samp{font-family:monospace;font-size:1em}
pre{white-space:pre-wrap}
q{quotes:"\201C" "\201D" "\2018" "\2019"}
small{font-size:80%}
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
sup{top:-.5em}
sub{bottom:-.25em}
img{border:0}
svg:not(:root){overflow:hidden}
figure{margin:0}
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
legend{border:0;padding:0}
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
button,input{line-height:normal}
button,select{text-transform:none}
button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer}
button[disabled],html input[disabled]{cursor:default}
input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0}
input[type="search"]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box}
input[type="search"]::-webkit-search-cancel-button,input[type="search"]::-webkit-search-decoration{-webkit-appearance:none}
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
textarea{overflow:auto;vertical-align:top}
table{border-collapse:collapse;border-spacing:0}
*,*:before,*:after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box}
html,body{font-size:100%}
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto;tab-size:4;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
a:hover{cursor:pointer}
img,object,embed{max-width:100%;height:auto}
object,embed{height:100%}
img{-ms-interpolation-mode:bicubic}
.left{float:left!important}
.right{float:right!important}
.text-left{text-align:left!important}
.text-right{text-align:right!important}
.text-center{text-align:center!important}
.text-justify{text-align:justify!important}
.hide{display:none}
img,object,svg{display:inline-block;vertical-align:middle}
textarea{height:auto;min-height:50px}
select{width:100%}
.center{margin-left:auto;margin-right:auto}
.spread{width:100%}
p.lead,.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{font-size:1.21875em;line-height:1.6}
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr}
a{color:#2156a5;text-decoration:underline;line-height:inherit}
a:hover,a:focus{color:#1d4b8f}
a img{border:none}
p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
p aside{font-size:.875em;line-height:1.35;font-style:italic}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
h1{font-size:2.125em}
h2{font-size:1.6875em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
h4,h5{font-size:1.125em}
h6{font-size:1em}
hr{border:solid #ddddd8;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0}
em,i{font-style:italic;line-height:inherit}
strong,b{font-weight:bold;line-height:inherit}
small{font-size:60%;line-height:inherit}
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
ul,ol{margin-left:1.5em}
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em}
ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit}
ul.square{list-style-type:square}
ul.circle{list-style-type:circle}
ul.disc{list-style-type:disc}
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
dl dt{margin-bottom:.3125em;font-weight:bold}
dl dd{margin-bottom:1.25em}
abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help}
abbr{text-transform:none}
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)}
blockquote cite:before{content:"\2014 \0020"}
blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)}
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
@media only screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
h1{font-size:2.75em}
h2{font-size:2.3125em}
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
h4{font-size:1.4375em}}
table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede}
table thead,table tfoot{background:#f7f8f7;font-weight:bold}
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
table tr.even,table tr.alt,table tr:nth-of-type(even){background:#f8f8f7}
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6}
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
.clearfix:before,.clearfix:after,.float-group:before,.float-group:after{content:" ";display:table}
.clearfix:after,.float-group:after{clear:both}
*:not(pre)>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background-color:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed;word-wrap:break-word}
*:not(pre)>code.nobreak{word-wrap:normal}
*:not(pre)>code.nowrap{white-space:nowrap}
pre,pre>code{line-height:1.45;color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;text-rendering:optimizeSpeed}
em em{font-style:normal}
strong strong{font-weight:400}
.keyseq{color:rgba(51,51,51,.8)}
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background-color:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
.keyseq kbd:first-child{margin-left:0}
.keyseq kbd:last-child{margin-right:0}
.menuseq,.menuref{color:#000}
.menuseq b:not(.caret),.menuref{font-weight:inherit}
.menuseq{word-spacing:-.02em}
.menuseq b.caret{font-size:1.25em;line-height:.8}
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
b.button:before,b.button:after{position:relative;top:-1px;font-weight:400}
b.button:before{content:"[";padding:0 3px 0 2px}
b.button:after{content:"]";padding:0 2px 0 3px}
p a>code:hover{color:rgba(0,0,0,.9)}
#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
#header:before,#header:after,#content:before,#content:after,#footnotes:before,#footnotes:after,#footer:before,#footer:after{content:" ";display:table}
#header:after,#content:after,#footnotes:after,#footer:after{clear:both}
#content{margin-top:1.25em}
#content:before{content:none}
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #ddddd8}
#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #ddddd8;padding-bottom:8px}
#header .details{border-bottom:1px solid #ddddd8;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap}
#header .details span:first-child{margin-left:-.125em}
#header .details span.email a{color:rgba(0,0,0,.85)}
#header .details br{display:none}
#header .details br+span:before{content:"\00a0\2013\00a0"}
#header .details br+span.author:before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
#header .details br+span#revremark:before{content:"\00a0|\00a0"}
#header #revnumber{text-transform:capitalize}
#header #revnumber:after{content:"\00a0"}
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #ddddd8;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
#toc{border-bottom:1px solid #efefed;padding-bottom:.5em}
#toc>ul{margin-left:.125em}
#toc ul.sectlevel0>li>a{font-style:italic}
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
#toc li{line-height:1.3334;margin-top:.3334em}
#toc a{text-decoration:none}
#toc a:active{text-decoration:underline}
#toctitle{color:#7a2518;font-size:1.2em}
@media only screen and (min-width:768px){#toctitle{font-size:1.375em}
body.toc2{padding-left:15em;padding-right:0}
#toc.toc2{margin-top:0!important;background-color:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #efefed;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
#toc.toc2>ul{font-size:.9em;margin-bottom:0}
#toc.toc2 ul ul{margin-left:0;padding-left:1em}
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
body.toc2.toc-right{padding-left:0;padding-right:15em}
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #efefed;left:auto;right:0}}
@media only screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
#toc.toc2{width:20em}
#toc.toc2 #toctitle{font-size:1.375em}
#toc.toc2>ul{font-size:.95em}
#toc.toc2 ul ul{padding-left:1.25em}
body.toc2.toc-right{padding-left:0;padding-right:20em}}
#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px}
#content #toc>:first-child{margin-top:0}
#content #toc>:last-child{margin-bottom:0}
#footer{max-width:100%;background-color:rgba(0,0,0,.8);padding:1.25em}
#footer-text{color:rgba(255,255,255,.8);line-height:1.44}
.sect1{padding-bottom:.625em}
@media only screen and (min-width:768px){.sect1{padding-bottom:1.25em}}
.sect1+.sect1{border-top:1px solid #efefed}
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
#content h1>a.anchor:before,h2>a.anchor:before,h3>a.anchor:before,#toctitle>a.anchor:before,.sidebarblock>.content>.title>a.anchor:before,h4>a.anchor:before,h5>a.anchor:before,h6>a.anchor:before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
table.tableblock>caption.title{white-space:nowrap;overflow:visible;max-width:0}
.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{color:rgba(0,0,0,.85)}
table.tableblock #preamble>.sectionbody>.paragraph:first-of-type p{font-size:inherit}
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
.admonitionblock>table td.icon{text-align:center;width:80px}
.admonitionblock>table td.icon img{max-width:initial}
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #ddddd8;color:rgba(0,0,0,.6)}
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px}
.exampleblock>.content>:first-child{margin-top:0}
.exampleblock>.content>:last-child{margin-bottom:0}
.sidebarblock{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px}
.sidebarblock>:first-child{margin-top:0}
.sidebarblock>:last-child{margin-bottom:0}
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
.literalblock pre,.listingblock pre:not(.highlight),.listingblock pre[class="highlight"],.listingblock pre[class^="highlight "],.listingblock pre.CodeRay,.listingblock pre.prettyprint{background:#f7f7f8}
.sidebarblock .literalblock pre,.sidebarblock .listingblock pre:not(.highlight),.sidebarblock .listingblock pre[class="highlight"],.sidebarblock .listingblock pre[class^="highlight "],.sidebarblock .listingblock pre.CodeRay,.sidebarblock .listingblock pre.prettyprint{background:#f2f1f1}
.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;padding:1em;font-size:.8125em}
.literalblock pre.nowrap,.literalblock pre[class].nowrap,.listingblock pre.nowrap,.listingblock pre[class].nowrap{overflow-x:auto;white-space:pre;word-wrap:normal}
@media only screen and (min-width:768px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:.90625em}}
@media only screen and (min-width:1280px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:1em}}
.literalblock.output pre{color:#f7f7f8;background-color:rgba(0,0,0,.9)}
.listingblock pre.highlightjs{padding:0}
.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px}
.listingblock pre.prettyprint{border-width:0}
.listingblock>.content{position:relative}
.listingblock code[data-lang]:before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:#999}
.listingblock:hover code[data-lang]:before{display:block}
.listingblock.terminal pre .command:before{content:attr(data-prompt);padding-right:.5em;color:#999}
.listingblock.terminal pre .command:not([data-prompt]):before{content:"$"}
table.pyhltable{border-collapse:separate;border:0;margin-bottom:0;background:none}
table.pyhltable td{vertical-align:top;padding-top:0;padding-bottom:0;line-height:1.45}
table.pyhltable td.code{padding-left:.75em;padding-right:0}
pre.pygments .lineno,table.pyhltable td:not(.code){color:#999;padding-left:0;padding-right:.5em;border-right:1px solid #ddddd8}
pre.pygments .lineno{display:inline-block;margin-right:.25em}
table.pyhltable .linenodiv{background:none!important;padding-right:0!important}
.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
.quoteblock>.title{margin-left:-1.5em;margin-bottom:.75em}
.quoteblock blockquote,.quoteblock blockquote p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
.quoteblock blockquote{margin:0;padding:0;border:0}
.quoteblock blockquote:before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
.quoteblock .attribution{margin-top:.5em;margin-right:.5ex;text-align:right}
.quoteblock .quoteblock{margin-left:0;margin-right:0;padding:.5em 0;border-left:3px solid rgba(0,0,0,.6)}
.quoteblock .quoteblock blockquote{padding:0 0 0 .75em}
.quoteblock .quoteblock blockquote:before{display:none}
.verseblock{margin:0 1em 1.25em 1em}
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
.verseblock pre strong{font-weight:400}
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
.quoteblock .attribution br,.verseblock .attribution br{display:none}
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
.quoteblock.abstract{margin:0 0 1.25em 0;display:block}
.quoteblock.abstract blockquote,.quoteblock.abstract blockquote p{text-align:left;word-spacing:0}
.quoteblock.abstract blockquote:before,.quoteblock.abstract blockquote p:first-of-type:before{display:none}
table.tableblock{max-width:100%;border-collapse:separate}
table.tableblock td>.paragraph:last-child p>p:last-child,table.tableblock th>p:last-child,table.tableblock td>p:last-child{margin-bottom:0}
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
table.grid-all>thead>tr>.tableblock,table.grid-all>tbody>tr>.tableblock{border-width:0 1px 1px 0}
table.grid-all>tfoot>tr>.tableblock{border-width:1px 1px 0 0}
table.grid-cols>*>tr>.tableblock{border-width:0 1px 0 0}
table.grid-rows>thead>tr>.tableblock,table.grid-rows>tbody>tr>.tableblock{border-width:0 0 1px 0}
table.grid-rows>tfoot>tr>.tableblock{border-width:1px 0 0 0}
table.grid-all>*>tr>.tableblock:last-child,table.grid-cols>*>tr>.tableblock:last-child{border-right-width:0}
table.grid-all>tbody>tr:last-child>.tableblock,table.grid-all>thead:last-child>tr>.tableblock,table.grid-rows>tbody>tr:last-child>.tableblock,table.grid-rows>thead:last-child>tr>.tableblock{border-bottom-width:0}
table.frame-all{border-width:1px}
table.frame-sides{border-width:0 1px}
table.frame-topbot{border-width:1px 0}
th.halign-left,td.halign-left{text-align:left}
th.halign-right,td.halign-right{text-align:right}
th.halign-center,td.halign-center{text-align:center}
th.valign-top,td.valign-top{vertical-align:top}
th.valign-bottom,td.valign-bottom{vertical-align:bottom}
th.valign-middle,td.valign-middle{vertical-align:middle}
table thead th,table tfoot th{font-weight:bold}
tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7}
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
p.tableblock>code:only-child{background:none;padding:0}
p.tableblock{font-size:1em}
td>div.verse{white-space:pre}
ol{margin-left:1.75em}
ul li ol{margin-left:1.5em}
dl dd{margin-left:1.125em}
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
ul.unstyled,ol.unstyled{margin-left:0}
ul.checklist{margin-left:.625em}
ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
ul.checklist li>p:first-child>input[type="checkbox"]:first-child{margin-right:.25em}
ul.inline{margin:0 auto .625em auto;margin-left:-1.375em;margin-right:0;padding:0;list-style:none;overflow:hidden}
ul.inline>li{list-style:none;float:left;margin-left:1.375em;display:block}
ul.inline>li>*{display:block}
.unstyled dl dt{font-weight:400;font-style:normal}
ol.arabic{list-style-type:decimal}
ol.decimal{list-style-type:decimal-leading-zero}
ol.loweralpha{list-style-type:lower-alpha}
ol.upperalpha{list-style-type:upper-alpha}
ol.lowerroman{list-style-type:lower-roman}
ol.upperroman{list-style-type:upper-roman}
ol.lowergreek{list-style-type:lower-greek}
.hdlist>table,.colist>table{border:0;background:none}
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
td.hdlist1{font-weight:bold;padding-bottom:1.25em}
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
.colist>table tr>td:first-of-type{padding:.4em .75em 0 .75em;line-height:1;vertical-align:top}
.colist>table tr>td:first-of-type img{max-width:initial}
.colist>table tr>td:last-of-type{padding:.25em 0}
.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd}
.imageblock.left,.imageblock[style*="float: left"]{margin:.25em .625em 1.25em 0}
.imageblock.right,.imageblock[style*="float: right"]{margin:.25em 0 1.25em .625em}
.imageblock>.title{margin-bottom:0}
.imageblock.thumb,.imageblock.th{border-width:6px}
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
.image.left{margin-right:.625em}
.image.right{margin-left:.625em}
a.image{text-decoration:none;display:inline-block}
a.image object{pointer-events:none}
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
sup.footnote a,sup.footnoteref a{text-decoration:none}
sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline}
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em 0;border-width:1px 0 0 0}
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;text-indent:-1.05em;margin-bottom:.2em}
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none}
#footnotes .footnote:last-of-type{margin-bottom:0}
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0}
.gist .file-data>table td.line-data{width:99%}
div.unbreakable{page-break-inside:avoid}
.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background-color:#00fafa}
.black{color:#000}
.black-background{background-color:#000}
.blue{color:#0000bf}
.blue-background{background-color:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background-color:#fa00fa}
.gray{color:#606060}
.gray-background{background-color:#7d7d7d}
.green{color:#006000}
.green-background{background-color:#007d00}
.lime{color:#00bf00}
.lime-background{background-color:#00fa00}
.maroon{color:#600000}
.maroon-background{background-color:#7d0000}
.navy{color:#000060}
.navy-background{background-color:#00007d}
.olive{color:#606000}
.olive-background{background-color:#7d7d00}
.purple{color:#600060}
.purple-background{background-color:#7d007d}
.red{color:#bf0000}
.red-background{background-color:#fa0000}
.silver{color:#909090}
.silver-background{background-color:#bcbcbc}
.teal{color:#006060}
.teal-background{background-color:#007d7d}
.white{color:#bfbfbf}
.white-background{background-color:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background-color:#fafa00}
span.icon>.fa{cursor:default}
a span.icon>.fa{cursor:inherit}
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
.admonitionblock td.icon .icon-note:before{content:"\f05a";color:#19407c}
.admonitionblock td.icon .icon-tip:before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
.admonitionblock td.icon .icon-warning:before{content:"\f071";color:#bf6900}
.admonitionblock td.icon .icon-caution:before{content:"\f06d";color:#bf3400}
.admonitionblock td.icon .icon-important:before{content:"\f06a";color:#bf0000}
.conum[data-value]{display:inline-block;color:#fff!important;background-color:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
.conum[data-value] *{color:#fff!important}
.conum[data-value]+b{display:none}
.conum[data-value]:after{content:attr(data-value)}
pre .conum[data-value]{position:relative;top:-.125em}
b.conum *{color:inherit!important}
.conum:not([data-value]):empty{display:none}
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
h1,h2,p,td.content,span.alt{letter-spacing:-.01em}
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
p,blockquote,dt,td.content,span.alt{font-size:1.0625rem}
p{margin-bottom:1.25rem}
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
.exampleblock>.content{background-color:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc}
.print-only{display:none!important}
@media print{@page{margin:1.25cm .75cm}
*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important}
a{color:inherit!important;text-decoration:underline!important}
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
a[href^="http:"]:not(.bare):after,a[href^="https:"]:not(.bare):after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
abbr[title]:after{content:" (" attr(title) ")"}
pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
thead{display:table-header-group}
svg{max-width:100%}
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
#toc,.sidebarblock,.exampleblock>.content{background:none!important}
#toc{border-bottom:1px solid #ddddd8!important;padding-bottom:0!important}
.sect1{padding-bottom:0!important}
.sect1+.sect1{border:0!important}
#header>h1:first-child{margin-top:1.25rem}
body.book #header{text-align:center}
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em 0}
body.book #header .details{border:0!important;display:block;padding:0!important}
body.book #header .details span:first-child{margin-left:0!important}
body.book #header .details br{display:block}
body.book #header .details br+span:before{content:none!important}
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
.listingblock code[data-lang]:before{display:block}
#footer{background:none!important;padding:0 .9375em}
#footer-text{color:rgba(0,0,0,.6)!important;font-size:.9em}
.hide-on-print{display:none!important}
.print-only{display:block!important}
.hide-for-print{display:none!important}
.show-for-print{display:inherit!important}}
</style>
</head>
<body class="article">
<div id="header">
<h1>ZeroTier Mapping for Scalability Protocols</h1>
<div class="details">
<span id="author" class="author">Garrett D'Amore</span><br>
<span id="email" class="email"><a href="mailto:garrett@damore.org">garrett@damore.org</a></span><br>
<span id="revnumber">version 0.7,</span>
<span id="revdate">2017-11-07</span>
</div>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p><span class="image"><img src="https://img.shields.io/badge/status-draft-yellow.svg" alt="draft"></span></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_zeroter_mapping_for_scalability_protocols">ZeroTer Mapping for Scalability Protocols</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Status: draft</p>
</li>
<li>
<p>Authors: Garrett D’Amore <<a href="mailto:garrett@damore.org">garrett@damore.org</a>></p>
</li>
<li>
<p>Version 0.7</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_abstract">Abstract</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This document defines the
<a href="http://www.zerotier.com">ZeroTier</a> mapping for scalability protocols.
This enables SP protocols to run over a ZeroTier network. The transport
defined here sits on top of an unreliable virtual Layer 2 transport,
and does not require a TCP/IP stack.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_license">License</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Copyright 2017 Garrett D’Amore <<a href="mailto:garrett@damore.org">garrett@damore.org</a>><br>
Copyright 2017 Capitar IT Group BV <<a href="mailto:info@capitar.com">info@capitar.com</a>></p>
</div>
<div class="paragraph">
<p>This specification is licensed under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with the
License.
You may obtain a copy of the license
<a href="http://www.apache.org/licenses/LICENSE-2.0">online</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_language">Language</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in <a href="https://tools.ietf.org/html/rfc2119">RFC 2119</a>.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_underlying_protocol">Underlying protocol</h2>
<div class="sectionbody">
<div class="paragraph">
<p>ZeroTier expresses an 802.3 style layer 2, where frames maybe exchanged as if
they were Ethernet frames. Virtual broadcast domains are created within a
numbered "network", and frames may then be exchanged with any peers on that
network.</p>
</div>
<div class="paragraph">
<p>Frames may arrive in any order, or be lost, just a with Ethernet
(best effort delivery), but they are strongly protected by a
cryptographic checksum, so frames that do arrive will be uncorrupted.
Furthermore, ZeroTier guarantees that a given frame will be received
at most once.</p>
</div>
<div class="paragraph">
<p>Each application on a ZeroTier network has its own address, called a
ZeroTier ID (<code>ZTID</code>), which is globally unique — this is generated
from a hash of the public key associated with the application.</p>
</div>
<div class="paragraph">
<p>A given application may participate in multiple ZeroTier networks.</p>
</div>
<div class="paragraph">
<p>Sharing of ZeroTier IDs between applications, as well as use of multiple
ZTIDs within a single application, as well as management of the associated
ZeroTier-specific state is out of scope for this document.</p>
</div>
<div class="paragraph">
<p>ZeroTier networks have a standard MTU of 2800 bytes, but over typical
public networks an "optimum" MTU of 1400 bytes is used.
ZeroTier may be configured to have larger MTUs, but typically this involves
extensive reassembly at underlying layers, and implementations <strong>SHOULD</strong>
use the optimum MTU advertised by the ZeroTier implementation.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_packet_layout">Packet layout</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Each SP message sent over ZeroTier is comprised of one or
more fragments, where each fragment is mapped to a single underlying
ZeroTier L2 frame. We use the EtherType field of 0x0901 to indicate
SP over ZeroTier protocol (number to be registered with IEEE).</p>
</div>
<div class="paragraph">
<p>The ZeroTier L2 payload shall be encoded with a header as follows:</p>
</div>
<div class="literalblock">
<div class="content">
<pre> 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| op | flags | version |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| reserved | destination port |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| reserved | source port |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| op-specific payload...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+</pre>
</div>
</div>
<div class="paragraph">
<p>All numeric fields are in big-endian byte order. Note that ZeroTier
APIs present this as the L2 payload, but ZeroTier itself may prepend
additional data such as the Ethernet type, and source and destination
MAC addresses, as well as ZeroTier specific headers. The details of
such headers are out of scope for this document.</p>
</div>
<div class="paragraph">
<p>As above, the start of each frame is just as a normal Ethernet payload.
The Ethernet type (ethertype) we use for these frames is 0x901, with
a VLAN ID of 0.</p>
</div>
<div class="paragraph">
<p>The <code>op</code> is a field that indicates the type of message being sent. The
following values are defined: <code>DATA</code> (0x00), <code>CONN-REQ</code> (0x10),
<code>CONN-ACK</code> (0x12), <code>DISC</code> (0x20), <code>PING</code> (0x30), <code>PONG</code> (0x32),
and <code>ERR</code> (0x40). These are discussed further below. Implementations
<strong>MUST</strong> discard messages where the <code>op</code> is not one of these.</p>
</div>
<div class="paragraph">
<p>The <code>flags</code> field is reserved for future use, and <strong>MUST</strong> be zero.
Implementations <strong>MUST</strong> discard frames for which this is not true.</p>
</div>
<div class="paragraph">
<p>The <code>version</code> byte <strong>MUST</strong> be set to 0x1. Implementations <strong>MUST</strong> discard
any messages received for any other version.</p>
</div>
<div class="paragraph">
<p>The <code>source port</code> and <code>destination port</code> are used to construct a logical
conversation. These are 24-bits wide, and are discussed further below.
The <code>reserved</code> fields must be set to zero.</p>
</div>
<div class="paragraph">
<p>The remainder of frame varies depending on the <code>op</code> used.</p>
</div>
<div class="paragraph">
<p>The port fields are used to discriminate different uses, allowing one
application to have multiple connections or sockets open. The
purpose is analogous to TCP port numbers, except that instead of the
operating system performing the discrimination the application or
library code must do so. Note that port numbers are 24-bits. This
was chosen to allow a peer to allocate a unique port number for each
local conversation, allowing up to 16 million concurrent conversations.
This also allows a 40-bit node number to be combined with the 24-bit
port number to create a 64-bit unique address.</p>
</div>
<div class="paragraph">
<p>The <code>type</code> field is the numeric SP protocol ID, in big-endian form.
When receiving a message for a port, if the SP protocol ID does not
match the SP protocol expected on that port, the implementation <strong>MUST</strong>
discard this message.</p>
</div>
<div class="paragraph">
<p>Note that it is not by accident that the payload is 32-bit aligned in
this message format. The payload is actually 64-bit aligned.</p>
</div>
<div class="paragraph">
<p>Note that at this time, broadcast and multicast is not supported by
this mapping. (A future update may resolve this.)</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_data_messages">DATA messages</h2>
<div class="sectionbody">
<div class="paragraph">
<p><code>DATA</code> messages carry SP protocol payload data. They can only be sent
on an established session (see <code>CONN</code> messages below), and are never
acknowledged (in this version). The op-specific payload they carry
is formed like this:</p>
</div>
<div class="literalblock">
<div class="content">
<pre> 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| message ID | fragment size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fragment number | total fragments |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| user data...
+-+-+-+-+-+-+-+-</pre>
</div>
</div>
<div class="paragraph">
<p>All fragments, except for the last, <strong>MUST</strong> be the same size. The fragment
size field carries the size of every fragment, except that the last
fragment may be shorter; however even for the last fragment, the fragment
size <strong>MUST</strong> be the size of the rest of the fragments. This is necessary
to allow a receiver to know the fragment size of the other fragments even
if the final fragment is received before any others. (Typically this may
occur if a message consisting of two fragments arrives with fragments
out of order.)</p>
</div>
<div class="paragraph">
<p>The last fragment shall have the fragment number equal to
the total fragments minus one, and the first fragment shall have fragment
number 0. Under typical optimal conditions, with an optimal MTU of 1400
bytes, the largest message that can be transmitted is approximately 86 MB.
Specifically the limit is (65534 * (1400 - 20)) = 90,436,920 bytes.
(Larger MTUs may be used, if the implementation determines that it is
advantageous to do so. Doing so would necessarily give a larger maximum
message size.)</p>
</div>
<div class="paragraph">
<p>However, transmitting such a large message would require sending over
65 thousand fragments, and given the likelihood of fragment loss, and
the lack of acknowledgment, it is likely that the entire message would
be lost. As a result, implementations are encouraged to limit the
amount of data that they send to at most a few megabytes. Implementations
receiving the first fragment can easily calculate the worst case for
the message size (the size of the user payload multiplied by the total
number of fragments), and <strong>MAY</strong> reply to the sender with an <code>ERR</code> message
using the code 0x05, indicating that the message is larger than the
receiver is willing to accept.</p>
</div>
<div class="paragraph">
<p>Each fragment for a given message must carry the same <code>message ID</code>.
Implementations <strong>MUST</strong> initialize this to a random value when starting
a conversation, and <strong>MUST</strong> increment this each time a new message is sent.
Message IDs of zero are not permitted; implementations <strong>MUST</strong> skip past zero
when incrementing message IDs.</p>
</div>
<div class="paragraph">
<p>Implementations may detect the loss of a message by noticing skips in the
message IDs that are received, accounting for the expected skip past zero.</p>
</div>
<div class="paragraph">
<p>Note that no field conveys the length of the fragment itself, as
this can be determined from the L2 length — the user data within
the fragment extends to the end of the L2 payload supplied by ZeroTier.
(And, all fragments other than the final fragment for a message must
therefore have the same length.)</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_conn_req_and_conn_ack_messages">CONN-REQ and CONN-ACK messages</h2>
<div class="sectionbody">
<div class="paragraph">
<p><code>CONN-REQ</code> frames represent a request from an initiator to establish a
session, i.e. a new conversation or connection, and <code>CONN-ACK</code>
messages are the normal successful reply from the responder. They both
take the same form, which consists of the usual headers along with the
senders 16-bit (big-endian) SP protocol ID appended:</p>
</div>
<div class="literalblock">
<div class="content">
<pre> 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SP protocol ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+</pre>
</div>
</div>
<div class="paragraph">
<p>The connection is initiated by the initiator sending this message,
with its own SP protocol ID, with the <code>op</code> set to <code>CONN-REQ</code>.
The initiator must choose a <code>source port</code> number that is not currently
being used with the remote peer. (Most implementations will choose a
a source port that is not used at all. Source port numbers <strong>SHOULD</strong>
be chosen randomly.)</p>
</div>
<div class="paragraph">
<p>The responder will acknowledge this by replying with its SP protocol
ID in the 4-byte payload, using the <code>CONN-ACK</code> op. Additionally,
the source port number that the responder replies with <strong>MUST</strong> be the
one the intiator requested.</p>
</div>
<div class="paragraph">
<p>(Responders will identify the session using the initiators chosen
<code>source port</code>, which the initiator <strong>MUST NOT</strong> concurrently use for any
other sessions.)</p>
</div>
<div class="paragraph">
<p>Alternatively, a responder <strong>MAY</strong> reject the connection attempt by
sending a suitably formed ERR message (see below).</p>
</div>
<div class="paragraph">
<p>If a sender does not receive a reply, it <strong>SHOULD</strong> retry this message
before giving up and reporting an error to the user. It is recommended
that a configurable number of retries and time interval be used.</p>
</div>
<div class="paragraph">
<p>Given modern Internet latencies of generally less than 500 ms, resending
up to 12 <code>CONN-REQ</code> requests, once every 5 seconds, before giving up seems
reasonable. (These times are somewhat larger to allow for ZeroTier
path discovery to take place; this results in a timeout of approximately
a minute.)</p>
</div>
<div class="paragraph">
<p>The initiator <strong>MUST NOT</strong> send any <code>DATA</code> messages for a conversation until
it has received an ACK from the other party, and it <strong>MUST</strong> send all further
messages for the conversation to the port number supplied by the responder.</p>
</div>
<div class="paragraph">
<p>If a <code>CONN-REQ</code> frame is received by a responder for a conversation that already
exists, the responder <strong>MUST</strong> reply. Further, the source port it replies with,
and the SP protocol IDs <strong>MUST</strong> be identical to what it first sent. This
ensures that the <code>CONN-REQ</code> request is idempotent.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_disc_messages">DISC messages</h2>
<div class="sectionbody">
<div class="paragraph">
<p>DISC messages are used to request a session be terminated. This
notifies the remote sender that no more data will be sent or
accepted, and the session resources may be released. There is no
payload. There is no acknowledgment.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_ping_and_pong_messages">PING and PONG messages</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In order to keep session state, implementations will generally store
data for each session. In order to prevent a stale session from
consuming these resources forever, and in order to keep underlying
ZeroTier sessions alive, a <code>PING</code> message <strong>MAY</strong> be sent to a peer
with whom a session has been established. This message has no payload.</p>
</div>
<div class="paragraph">
<p>If the <code>PING</code> is is successful, then the responder <strong>MUST</strong> reply with a <code>PONG</code>
message. As with <code>PING</code>, the <code>PONG</code> message carries no payload.</p>
</div>
<div class="paragraph">
<p>There is no response to a <code>PONG</code> message.</p>
</div>
<div class="paragraph">
<p>In the event of an error, an implementation <strong>MAY</strong> reply with an <code>ERR</code>
message.</p>
</div>
<div class="paragraph">
<p>Implementations <strong>SHOULD NOT</strong> initiate <code>PING</code> messages if they have either
received other session messages recently.</p>
</div>
<div class="paragraph">
<p>Implementations <strong>SHOULD</strong> use a timeout <em>T1</em> seconds of be used before
initiating a message the first time, and that in the absence of a
reply, up to <em>N</em> further attempts be made, separated by <em>T2</em> seconds. If
no reply to the <em>N_th attempt is received after _T2</em> seconds have passed,
then the remote peer should be assumed offline or dead, and the
session closed.</p>
</div>
<div class="paragraph">
<p>The values for <em>T1</em>, <em>T2</em>, and <em>N</em> <strong>SHOULD</strong> be configurable, with
recommended default values of 60, 10, and 5. With these values,
sessions that appear dead after 2 minutes will be closed, and their
resources reclaimed.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_err_messages">ERR messages</h2>
<div class="sectionbody">
<div class="paragraph">
<p><code>ERR</code> messages indicate a failure in the session, and abruptly
terminate the session. The payload for these messages consists of a
single byte error code, followed by an ASCII message describing the
error (not terminated by zero). This message <strong>MUST NOT</strong> be more than
128 bytes in length.</p>
</div>
<div class="paragraph">
<p>The following error codes are defined:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>0x01 No party listening at that address or port.</p>
</li>
<li>
<p>0x02 No such session found.</p>
</li>
<li>
<p>0x03 SP protocol ID invalid.</p>
</li>
<li>
<p>0x04 Generic protocol error.</p>
</li>
<li>
<p>0x05 Message size too big.</p>
</li>
<li>
<p>0xff Other uncategorized error.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Implementations <strong>MUST</strong> discard any session state upon receiving an ERR
message. These messages are not acknowledged.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_message_reassembly">Message Reassembly</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Implementations <strong>MUST</strong> accept and reassemble fragmented <code>DATA</code> messages.
Implementations <strong>MUST</strong> discard fragmented messages of other types.</p>
</div>
<div class="paragraph">
<p>Messages larger than the ZeroTier MTU <strong>MUST</strong> be fragmented.</p>
</div>
<div class="paragraph">
<p>Implementations SHOULD limit the number of unassembled messages
retained for reassembly, to minimize the likelihood of intentional
abuse. It is suggested that at most 2 unassembled messages be
retained. It is further suggested that if 2 or more unfragmented
messages arrive before a message is reassembled, or more than 5
seconds pass before the reassembly is complete, that the unassembled
fragments be discarded.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_ports">Ports</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The port numbers are 24-bit fields, allowing a single ZT ID to
service multiple application layer protocols, which could be treated
as separate end points, or as separate sockets in the application.
The implementation is responsible for discriminating on these and
delivering to the appropriate consumer.</p>
</div>
<div class="paragraph">
<p>As with UDP or TCP, it is intended that each party have its own port
number, and that a pair of ports (combined with ZeroTier IDs) be used
to identify a single conversation.</p>
</div>
<div class="paragraph">
<p>An SP server <strong>SHOULD</strong> allocate a port for number advertisement. It is
expected clients will generate ephemeral port numbers.</p>
</div>
<div class="paragraph">
<p>Implementations are free to choose how to allocate port numbers, but
it is <strong>RECOMMENDED</strong> that manually configured port numbers are small, with
the high order bit clear, and that numbers larger than 2<sup>23</sup> (high order bit
set) be used for ephemeral allocations.</p>
</div>
<div class="paragraph">
<p>It is <strong>RECOMMENDED</strong> that separate short queues (perhaps just one or two
messages long) be kept per local port in implementations, to prevent
head-of-line blocking issues where backpressure on one consumer
(perhaps just a single thread or socket) blocks others.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_uri_format">URI Format</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The URI scheme used to represent ZeroTier addresses makes use of
ZeroTier IDs, ZeroTier network IDs, and our own 24-bit ports.</p>
</div>
<div class="paragraph">
<p>The format <strong>SHALL</strong> be <code>zt://<em>nwid</em>/<em>ztid:port</em></code>, where the <code><em>nwid</em></code>
component represents the 64-bit hexadecimal ZeroTier network ID,
the <code><em>ztid</em></code> represents the 40-bit hexadecimal ZeroTier Device ID,
and the <code><em>port</em></code> is the 24-bit port number (decimal) previously described.</p>
</div>
<div class="paragraph">
<p>A responder <strong>MAY</strong> elide the <code>/<em>ztid</em></code> portion, to just bind to itself,
in which case the format will be <code>zt://<em>nwid:port</em></code>.</p>
</div>
<div class="paragraph">
<p>A port number of 0 <strong>MAY</strong> be used when listening to indicate that a random
ephemeral port should be chosen.</p>
</div>
<div class="paragraph">
<p>An implementation <strong>MAY</strong> allow the <code><em>ztid</em></code> 0 be replaced with <code>*</code> to
indicate that the node’s local <code>ZTID</code> be used.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_security_considerations">Security Considerations</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The mapping isn’t intended to provide any additional security beyond that
provided by ZeroTier itself. Managing the key materials used by ZeroTier
is implementation-specific, and they must take the appropriate care when
dealing with them.</p>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Version 0.7<br>
Last updated 2017-11-07 17:29:02 UTC
</div>
</div>
</body>
</html>
|
