aboutsummaryrefslogtreecommitdiff
path: root/docs/ref/proto
diff options
context:
space:
mode:
authorGarrett D'Amore <garrett@damore.org>2024-10-14 18:08:13 -0700
committerGarrett D'Amore <garrett@damore.org>2024-10-14 18:08:13 -0700
commite8d6c2bf692dbcb4c503caa676314a6a626a8a68 (patch)
tree101c6492ad403a28b32e6a40a632924243360c90 /docs/ref/proto
parent356969c4e401fe3caa48d1c1c75f51fba24ebdab (diff)
downloadnng-e8d6c2bf692dbcb4c503caa676314a6a626a8a68.tar.gz
nng-e8d6c2bf692dbcb4c503caa676314a6a626a8a68.tar.bz2
nng-e8d6c2bf692dbcb4c503caa676314a6a626a8a68.zip
Converted protocols to mdbook.
Started an introductory section too.
Diffstat (limited to 'docs/ref/proto')
-rw-r--r--docs/ref/proto/bus.md62
-rw-r--r--docs/ref/proto/index.md42
-rw-r--r--docs/ref/proto/pair.md106
-rw-r--r--docs/ref/proto/pub.md42
-rw-r--r--docs/ref/proto/pull.md40
-rw-r--r--docs/ref/proto/push.md57
-rw-r--r--docs/ref/proto/rep.md62
-rw-r--r--docs/ref/proto/req.md144
-rw-r--r--docs/ref/proto/respondent.md64
-rw-r--r--docs/ref/proto/sub.md72
-rw-r--r--docs/ref/proto/surveyor.md129
11 files changed, 820 insertions, 0 deletions
diff --git a/docs/ref/proto/bus.md b/docs/ref/proto/bus.md
new file mode 100644
index 00000000..aa9d55d5
--- /dev/null
+++ b/docs/ref/proto/bus.md
@@ -0,0 +1,62 @@
+# BUS Protocol
+
+The {{i:*BUS* protocol}}{{hi:*BUS*}} provides for building mesh networks where
+every peer is connected to every other peer.
+In this protocol, each message sent by a node is sent to every one of
+its directly connected peers.
+
+> [!TIP]
+> Messages are only sent to directly connected peers.
+> This means that in the event that a peer is connected indirectly, it will not
+> receive messages.
+> When using this protocol to build mesh networks, it
+> is therefore important that a _fully-connected_ mesh network be constructed.
+
+All message delivery in this pattern is {{i:best-effort}}, which means that
+peers may not receive messages.
+Furthermore, delivery may occur to some,
+all, or none of the directly connected peers.
+(Messages are not delivered when peer nodes are unable to receive.)
+Hence, send operations will never block; instead if the
+message cannot be delivered for any reason it is discarded.
+
+> [!TIP]
+> In order to minimize the likelihood of message loss, this protocol
+> should not be used for high throughput communications.
+> Furthermore, the more traffic _in aggregate_ that occurs across the topology,
+> the more likely that message loss is to occur.
+
+## Socket Operations
+
+The [`nng_bus0_open`][nng_bus_open] functions create a bus socket.
+This socket may be used to send and receive messages.
+Sending messages will attempt to deliver to each directly connected peer.
+
+## Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+## Protocol Options
+
+The _BUS_ protocol has no protocol-specific options.
+
+## Protocol Headers
+
+When using a _BUS_ socket in [raw mode][raw], received messages will
+contain the incoming [pipe][pipe] ID as the sole element in the header.
+If a message containing such a header is sent using a raw _BUS_ socket, then,
+the message will be delivered to all connected pipes _except_ the one
+identified in the header.
+This behavior is intended for use with [device][device]
+configurations consisting of just a single socket.
+Such configurations are useful in the creation of rebroadcasters, and this
+capability prevents a message from being routed back to its source.
+If no header is present, then a message is sent to all connected pipes.
+
+When using normal (cooked mode) _BUS_ sockets, no message headers are present.
+
+[nng_bus_open]: TODO.md
+[device]: TODO.md
+[pipe]: TODO.md
+[raw]: TODO.md
diff --git a/docs/ref/proto/index.md b/docs/ref/proto/index.md
new file mode 100644
index 00000000..e7a4619f
--- /dev/null
+++ b/docs/ref/proto/index.md
@@ -0,0 +1,42 @@
+# Protocols
+
+{{hi:protocol}}
+The Scalability Protocols are a principally a collection of common networking
+patterns found in applications.
+
+The following patterns are included:
+
+## Request - Reply
+
+The {{i:request/reply pattern}} is made up of the [_REQ_][req] and [_REP_][rep] protocols.
+This most often used when implementing RPC-like services, where
+a given request is matched by a single reply.
+
+## Pipeline
+
+The {{i:pipeline pattern}} is made up of the [_PUSH_][push] and [_PULL_][pull]
+protocols.
+
+In this pattern communication is {{i:half-duplex}}, in that one side sends
+data and another side receives.
+
+This pattern is also characterized by its ability to solve distribution
+problems, and the fact that it has {{i:back-pressure}}, providing a measure
+of {{i:flow control}} to data production and consumption.
+
+## Publish - Subscribe
+
+## Bus
+
+## Pair
+
+[bus]: bus.md
+[pair]: pair.md
+[push]: push.md
+[pull]: pull.md
+[req]: req.md
+[rep]: rep.md
+[sub]: sub.md
+[pub]: pub.md
+[respondent]: respondent.md
+[surveyor]: surveyor.md
diff --git a/docs/ref/proto/pair.md b/docs/ref/proto/pair.md
new file mode 100644
index 00000000..13442f58
--- /dev/null
+++ b/docs/ref/proto/pair.md
@@ -0,0 +1,106 @@
+# PAIR protocol
+
+The {{i:*PAIR* protocol}}{{hi:*PAIR*}} implements a peer-to-peer pattern, where
+relationships between peers are one-to-one.
+
+### Socket Operations
+
+The [`nng_pair_open`][nng_pair_open] functions create a _PAIR_ socket.
+
+Normally, this pattern will block when attempting to send a message if
+no peer is able to receive the message.
+
+> [!NOTE]
+> Even though this mode may appear to be reliable, because back-pressure
+> prevents discarding messages most of the time, there are topologies involving
+> where messages may be discarded.
+> Applications that require reliable delivery semantics should consider using
+> [_REQ_][req] sockets, or implement their own acknowledgment layer on top of _PAIR_ sockets.
+
+### Protocol Versions
+
+Version 0 is the legacy version of this protocol.
+It lacks any header
+information, and is suitable when building simple one-to-one topologies.
+
+> [!TIP]
+> Use version 0 if you need to communicate with other implementations,
+> including the legacy [libnanomsg][nanomsg] library or
+> [mangos][mangos].
+
+Version 1 of the protocol offers improved protection against loops when
+used with [devices][device].
+
+### Polyamorous Mode
+
+> [!NOTE]
+> Polyamorous mode is deprecated, and support for it will likely
+> be removed in a future release, when a suitable mesh protocol is
+> available.
+> In the meantime, applications are encouraged to look to other patterns.
+
+Normally pair sockets are for one-to-one communication, and a given peer
+will reject new connections if it already has an active connection to another
+peer.
+
+_Polyamorous_{{hi:polyamorous mode}} changes this, to allow a socket to communicate with
+multiple directly-connected peers.
+This mode is enabled by opening a socket using [`nng_pair1_open_poly`][nng_pair_open].
+
+> [!TIP]
+> Polyamorous mode is only available when using pair version 1.
+
+In polyamorous mode a socket can support many one-to-one connections.
+In this mode, the application must
+choose the remote peer to receive an outgoing message by setting the
+[`nng_pipe`][pipe] to use for the outgoing message using
+[`nng_msg_set_pipe`][nng_msg_pipe].
+
+If no remote peer is specified by the sender, then the protocol will select
+any available connected peer.
+
+Most often the value of the outgoing pipe will be obtained from an incoming
+message using [`nng_msg_get_pipe`][nng_msg_pipe],
+such as when replying to an incoming message.
+
+> [!NOTE]
+> Directed send _only_ works with directly connected peers.
+> It will not function across [device][device] proxies.
+
+In order to prevent head-of-line blocking, if the peer on the given pipe
+is not able to receive (or the pipe is no longer available, such as if the
+peer has disconnected), then the message will be discarded with no notification
+to the sender.
+
+### Protocol Options
+
+The following protocol-specific options are available.
+
+- [`NNG_OPT_MAXTTL`][NNG_OPT_MAXTTL]:
+ (`int`, version 1 only). Maximum time-to-live.
+
+- `NNG_OPT_PAIR1_POLY`:
+ (`bool`, version 1 only) This option is no longer supported.
+ Formerly it was used to configure _polyamorous_ mode, but that mode
+ is now established by using the `nng_pair1_open_poly` function.
+
+### Protocol Headers
+
+Version 0 of the pair protocol has no protocol-specific headers.
+
+Version 1 of the pair protocol uses a single 32-bit unsigned value. The
+low-order (big-endian) byte of this value contains a "hop" count, and is
+used in conjunction with the
+[`NNG_OPT_MAXTTL`][NNG_OPT_MAXTTL] option to guard against
+device forwarding loops.
+This value is initialized to 1, and incremented each time the message is
+received by a new node.
+
+[nng_pair_open]: TODO.md
+[NNG_OPT_MAXTTL]: TODO.md
+[device]: TODO.md
+[nanomsg]: TODO.md
+[mangos]: TODO.md
+[pipe]: TODO.md
+[nng_msg_pipe]: ../api/msg/nng_msg_pipe.md
+[req]: ./req.md
diff --git a/docs/ref/proto/pub.md b/docs/ref/proto/pub.md
new file mode 100644
index 00000000..40a75706
--- /dev/null
+++ b/docs/ref/proto/pub.md
@@ -0,0 +1,42 @@
+# PUB Protocol
+
+The {{i:*PUB* protocol}}{{hi:*PUB*}} is one half of a {{i:publisher}}/subscriber pattern.
+In this pattern, a publisher sends data, which is broadcast to all
+subscribers.
+The subscribing applications only see the data to which
+they have subscribed.
+
+The _PUB_ protocol is the publisher side, and the
+[_SUB_](sub.md) protocol is the subscriber side.
+
+> [!NOTE]
+> In this implementation, the publisher delivers all messages to all
+> subscribers.
+> The subscribers maintain their own subscriptions, and filter them locally.
+> Thus, this pattern should not be used in an attempt to reduce bandwidth
+> consumption.
+
+The topics that subscribers subscribe to is just the first part of
+the message body.
+Applications should construct their messages accordingly.
+
+## Socket Operations
+
+The [`nng_pub0_open()`][nng_pub_open] functions create a publisher socket.
+This socket may be used to send messages, but is unable to receive them.
+Attempts to receive messages will result in `NNG_ENOTSUP`.
+
+## Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+## Protocol Options
+
+The _PUB_ protocol has no protocol-specific options.
+
+## Protocol Headers
+
+The _PUB_ protocol has no protocol-specific headers.
+
+[nng_pub_open]: TODO.md
diff --git a/docs/ref/proto/pull.md b/docs/ref/proto/pull.md
new file mode 100644
index 00000000..9946a887
--- /dev/null
+++ b/docs/ref/proto/pull.md
@@ -0,0 +1,40 @@
+# PULL protocol
+
+The {{i:*PULL* protocol}}{{hi:*PULL*}} is one half of a
+{{i:pipeline pattern}}.
+The other half is the [_PUSH_][push] protocol.
+
+In the pipeline pattern, pushers distribute messages to pullers.
+Each message sent
+by a pusher will be sent to one of its peer pullers,
+chosen in a round-robin fashion
+from the set of connected peers available for receiving.
+This property makes this pattern useful in {{i:load-balancing}} scenarios.
+
+### Socket Operations
+
+The [`nng_pull0_open`][nng_pull_open] functions create a
+_PULL_ socket.
+This socket may be used to receive messages, but is unable to send them.
+Attempts to send messages will result in `NNG_ENOTSUP`.
+
+When receiving messages, the _PULL_ protocol accepts messages as
+they arrive from peers.
+If two peers both have a message ready, the
+order in which messages are handled is undefined.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+### Protocol Options
+
+The _PULL_ protocol has no protocol-specific options.
+
+### Protocol Headers
+
+The _PULL_ protocol has no protocol-specific headers.
+
+[nng_pull_open]: TODO.md
+[push]: ./push.md
diff --git a/docs/ref/proto/push.md b/docs/ref/proto/push.md
new file mode 100644
index 00000000..e0218bbd
--- /dev/null
+++ b/docs/ref/proto/push.md
@@ -0,0 +1,57 @@
+# PUSH protocol
+
+## DESCRIPTION
+
+The {{i:*PUSH* protocol}}{{hi:*PUSH*}} is one half of a {{i:pipeline pattern}}.
+The other side is the [_PULL_][pull] protocol.
+
+In the pipeline pattern, pushers distribute messages to pullers.
+Each message sent by a pusher will be sent to one of its peer pullers,
+chosen in a round-robin fashion
+from the set of connected peers available for receiving.
+This property makes this pattern useful in {{i:load-balancing}} scenarios.
+
+### Socket Operations
+
+The [`nng_push0_open`][nng_push_open] call creates a _PUSH_ socket.
+This socket may be used to send messages, but is unable to receive them.
+Attempts to receive messages will result in `NNG_ENOTSUP`.
+
+Send operations will observe flow control (back-pressure), so that
+only peers capable of accepting a message will be considered.
+If no peer is available to receive a message, then the send operation will
+wait until one is available, or the operation times out.
+
+> [!NOTE]
+> Although the pipeline protocol honors flow control, and attempts
+> to avoid dropping messages, no guarantee of delivery is made.
+> Furthermore, as there is no capability for message acknowledgment,
+> applications that need reliable delivery are encouraged to consider the
+> [_REQ_][req] protocol instead.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+### Protocol Options
+
+- [`NNG_OPT_SENDBUF`][NNG_OPT_SENDBUF]:
+ (`int`, 0 - 8192)
+ Normally this is set to zero, indicating that send operations are unbuffered.
+ In unbuffered operation, send operations will wait until a suitable peer is available to receive the message.
+ If this is set to a positive value (up to 8192), then an intermediate buffer is
+ provided for the socket with the specified depth (in messages).
+
+> [!NOTE]
+> Transport layer buffering may occur in addition to any socket
+> buffer determined by this option.
+
+### Protocol Headers
+
+The _PUSH_ protocol has no protocol-specific headers.
+
+[nng_push_open]: TODO.md
+[NNG_OPT_SENDBUF]: TODO.md
+[pull]: ./pull.md
+[req]: ./req.md
diff --git a/docs/ref/proto/rep.md b/docs/ref/proto/rep.md
new file mode 100644
index 00000000..e7fe27ed
--- /dev/null
+++ b/docs/ref/proto/rep.md
@@ -0,0 +1,62 @@
+# REP Protocol
+
+The {{i:*REP* protocol}}{{hi:*REP*}} is one half of a {{i:request/reply pattern}}.
+In this pattern, a requester sends a message to one replier, who
+is expected to reply.
+The request is resent if no reply arrives,
+until a reply is received or the request times out.
+
+> [!TIP]
+> This protocol is useful in setting up RPC-like services.
+> It is also reliable, in that a requester will keep retrying until
+> a reply is received.
+
+The _REP_ protocol is the replier side, and the
+[_REP_][req] protocol is the requester side.
+
+## Socket Operations
+
+The [`nng_rep0_open`][nng_rep_open] functions create a replier socket.
+This socket may be used to receive messages (requests), and then to send
+replies.
+
+Generally a reply can only be sent after receiving a request.
+
+Send operations will result in `NNG_ESTATE` if no corresponding request
+was previously received.
+
+Likewise, only one receive operation may be pending at a time.
+Any additional concurrent receive operations will result in `NNG_ESTATE`.
+
+[Raw mode][raw] sockets ignore all these restrictions.
+
+## Context Operations
+
+This protocol supports the creation of [contexts][context] for concurrent
+use cases using [`nng_ctx_open`][nng_ctx_open].
+
+Each context may have at most one outstanding request, and operates
+independently of the others.
+The restrictions for order of operations with sockets apply equally
+well for contexts, except that each context will be treated as if it were
+a separate socket.
+
+## Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+## Protocol Options
+
+The _REP_ protocol has no protocol-specific options.
+
+## Protocol Headers
+
+The _REP_ protocol uses a {{ii:backtrace}} in the header.
+This is more fully documented in the [_REQ_][req] chapter.
+
+[nng_rep_open]: TODO.md
+[nng_ctx_open]: TODO.md
+[raw]: TODO.md
+[context]: TODO.md
+[req]: ./req.md
diff --git a/docs/ref/proto/req.md b/docs/ref/proto/req.md
new file mode 100644
index 00000000..e50662c3
--- /dev/null
+++ b/docs/ref/proto/req.md
@@ -0,0 +1,144 @@
+# REQ protocol
+
+The {{i:*REQ* protocol}}{{hi:*REQ*}} is one half of a {{i:request/reply pattern}}.
+In this pattern, a requester sends a message to one replier, who
+is expected to reply.
+The request is resent if no reply arrives,
+until a reply is received or the request times out.
+
+> [!TIP]
+> This protocol is useful in setting up RPC-like services.
+> It is also "reliable", in that a the requester will keep retrying until
+> a reply is received.
+
+> [!NOTE]
+> Because requests are resent, it is important that they be {{i:idempotent}}
+> to ensure predictable and repeatable behavior even in the face of duplicated
+> requests, which can occur (for example if a reply message is lost for
+> some reason.)
+
+{{hi: load-balancing}}
+The requester generally only has one outstanding request at a time unless
+in [raw mode][raw],
+and it will generally attempt to spread work requests to different peer repliers.
+
+> [!TIP]
+> This property, when combined with a [device][device]
+> can help provide a degree of load-balancing.
+
+The _REQ_ protocol is the requester side, and the [_REP_][rep] protocol
+is the replier side.
+
+### Socket Operations
+
+The [`nng_req0_open`][nng_req_open] functions create a _REQ_ socket.
+This socket may be used to send messages (requests), and then to receive replies.
+
+Generally a reply can only be received after sending a request.
+(Attempts to receive a message will result in `NNG_ESTATE` if there is no
+outstanding request.)
+
+Furthermore, only a single receive operation may be pending at a time.
+Attempts to post more receive operations concurrently will result in
+`NNG_ESTATE`.
+
+Requests may be canceled by sending a different request.
+This will cause the requester to discard any reply from the earlier request,
+but it will not stop a replier
+from processing a request it has already received or terminate a request
+that has already been placed on the wire.
+
+[Raw mode][raw] sockets ignore all these restrictions.
+
+### Context Operations
+
+This protocol supports the creation of [contexts][context] for concurrent
+use cases using [`nng_ctx_open`][nng_ctx_open].
+
+The `NNG_OPT_REQ_RESENDTIME` value may be configured differently
+on contexts created this way.
+
+Each context may have at most one outstanding request, and operates
+independently from the others.
+
+The restrictions for order of operations with sockets apply equally
+well for contexts, except that each context will be treated as if it were
+a separate socket.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+### Protocol Options
+
+The following protocol-specific option is available.
+
+- {{i:`NNG_OPT_REQ_RESENDTIME`}}: \
+ ([`nng_duration`][duration]) \
+ When a new request is started, a timer of this duration is also started.
+ If no reply is received before this timer expires, then the request will
+ be resent. \
+ \
+ Requests are also automatically resent if the peer to whom
+ the original request was sent disconnects. \
+ \
+ Resending may be deferred up to the value of the `NNG_OPT_RESENDTICK` parameter. \
+ \
+ If the value is set to [`NNG_DURATION_INFINITE`][duration], then resends are disabled
+ altogether. This should be used when the request is not idemptoent.
+
+- {{i:`NNG_OPT_REQ_RESENDTICK`}}: \
+ ([`nng_duration`][duration]) \
+ This is the granularity of the clock that is used to check for resending.
+ The default is a second. Setting this to a higher rate will allow for
+ more timely resending to occur, but may incur significant additional
+ overhead when the socket has many outstanding requests (contexts). \
+ \
+ When there are no requests outstanding that have a resend set, then
+ the clock does not tick at all. \
+ \
+ This option is shared for all contexts on a socket, and is only available for the socket itself.
+
+### Protocol Headers
+
+This protocol uses a {{ii:backtrace}} in the header.
+This form uses a stack of 32-bit big-endian identifiers.
+There _must_ be at least one identifier, the **request ID**, which will be the
+last element in the array, and _must_ have the most significant bit set.
+
+There may be additional **peer ID**s preceding the request ID.
+These will be distinguishable from the request ID by having their most
+significant bit clear.
+
+When a request message is received by a forwarding node (such as a [device][device]),
+the forwarding node prepends a
+32-bit peer ID (which _must_ have the most significant bit clear),
+which is the forwarder's way of identifying the directly connected
+peer from which it received the message.
+(This peer ID, except for the
+most significant bit, has meaning only to the forwarding node itself.)
+
+It may help to think of prepending a peer ID as pushing a peer ID onto the
+front of the stack of headers for the message.
+(It will use the peer ID
+it popped from the front to determine the next intermediate destination
+for the reply.)
+
+When a reply message is created, it is created using the same headers
+that the request contained.
+
+A forwarding node can pop the peer ID it originally pushed on the
+message, stripping it from the front of the message as it does so.
+
+When the reply finally arrives back at the initiating requester, it
+should have only a single element in the message, which will be the
+request ID it originally used for the request.
+
+[nng_req_open]: TODO.md
+[nng_ctx_open]: TODO.md
+[raw]: TODO.md
+[device]: TODO.md
+[context]: TODO.md
+[rep]: ./rep.md
+[duration]: ../api/util/nng_duration.md
diff --git a/docs/ref/proto/respondent.md b/docs/ref/proto/respondent.md
new file mode 100644
index 00000000..0f101550
--- /dev/null
+++ b/docs/ref/proto/respondent.md
@@ -0,0 +1,64 @@
+# RESPONDENT protocol
+
+The {{i:*RESPONDENT* protocol}}{{hi:*RESPONDENT*}} is one half of a {{i:survey pattern}}.
+In this pattern, a surveyor sends a survey, which is broadcast to all
+peer respondents.
+The respondents then have a chance to reply (but are not obliged to reply).
+The survey itself is a timed event, so that responses
+received after the survey has finished are discarded.
+
+> [!TIP]
+> This protocol is useful in solving voting problems, such as leader
+> election in cluster configurations, as well as certain kinds of service
+> discovery problems.
+
+The _RESPONDENT_ protocol is the respondent side, and the
+[_SURVEYOR_][surveyor] protocol is the surveyor side.
+
+### Socket Operations
+
+The [`nng_respondent0_open`][nng_respondent_open] functions create a
+respondent socket.
+This socket may be used to receive messages, and then to send replies.
+A reply can only be sent after receiving a survey, and generally the
+reply will be sent to surveyor from whom the last survey was received.
+
+Respondents may discard a survey by simply not replying to it.
+
+[Raw mode][raw] sockets ignore all these restrictions.
+
+### Context Operations
+
+This protocol supports the creation of [contexts][context] for concurrent
+use cases using [`nng_ctx_open`][nng_ctx_open].
+
+Incoming surveys will be routed to and received by only one context.
+Additional surveys may be received by other contexts in parallel.
+Replies made using a context will be returned to the the surveyor that
+issued the survey most recently received by that context.
+The restrictions for order of operations with sockets apply equally
+well for contexts, except that each context will be treated as if it were
+a separate socket.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+At the time of writing, no other versions of this protocol have been defined.
+{{footnote: An earlier and incompatible version of the protocol was used in older
+pre-releases of [nanomsg][nanomsg], but was not released in any production version.}}
+
+### Protocol Options
+
+The _respondent_ protocol has no protocol-specific options.
+
+### Protocol Headers
+
+The _RESPONDENT_ protocol uses a {{ii:backtrace}} in the header.
+This is more fully documented in the [_SURVEYOR_][surveyor] manual.
+
+[nng_respondent_open]: TODO.md
+[nng_ctx_open]: TODO.md
+[nanomsg]: TODO.md
+[context]: TODO.md
+[raw]: TODO.md
+[surveyor]: ./surveyor.md
diff --git a/docs/ref/proto/sub.md b/docs/ref/proto/sub.md
new file mode 100644
index 00000000..bd8df577
--- /dev/null
+++ b/docs/ref/proto/sub.md
@@ -0,0 +1,72 @@
+# SUB protocol
+
+The {{i:*SUB* protocol}}{{hi:*SUB*}} is one half of a publisher/{{i:subscriber}} pattern.
+In this pattern, a publisher sends data, which is broadcast to all subscribers.
+The subscribing applications only see the data to which they have subscribed.
+
+The _SUB_ protocol is the subscriber side, and the
+[_PUB_][pub] protocol is the publisher side.
+
+> [!NOTE]
+> The publisher delivers all messages to all subscribers.
+> The subscribers maintain their own subscriptions, and filter them locally.
+> Thus, this pattern should not be used in an attempt to
+> reduce bandwidth consumption.
+
+The topics that subscribers subscribe to is compared to the leading bytes of
+the message body.
+Applications should construct their messages accordingly.
+
+### Socket Operations
+
+The [`nng_sub0_open`][nng_sub_open] functions create a _SUB_ socket.
+This socket may be used to receive messages, but is unable to send them.
+Attempts to send messages will result in `NNG_ENOTSUP`.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+(At the time of writing, no other versions of this protocol have been defined.)
+
+### Protocol Options
+
+The following protocol-specific options are available.
+
+- {{i:`NNG_OPT_SUB_SUBSCRIBE`}}{{hi:subscribe}}: \
+ \
+ This option registers a topic that the subscriber is interested in.
+ The option is write-only, and takes an array of bytes, of arbitrary size.
+ Each incoming message is checked against the list of subscribed topics.
+ If the body begins with the entire set of bytes in the topic, then the
+ message is accepted. If no topic matches, then the message is
+ discarded. \
+ \
+ This option is a byte array. Thus if you use
+ [`nng_socket_set_string`][nng_socket_set] the `NUL` terminator byte will
+ be included in the topic.
+ If that isn't desired, consider using
+ [`nng_socket_set`][nng_socket_set] and using `strlen` of the topic
+ as the topic size. \
+ \
+ To receive all messages, an empty topic (zero length) can be used.
+
+- {{i:`NNG_OPT_SUB_UNSUBSCRIBE`}}: \
+ \
+ This option, also read-only, removes a topic from the subscription list.
+ Note that if the topic was not previously subscribed to with
+ `NNG_OPT_SUB_SUBSCRIBE` then an `NNG_ENOENT` error will result.
+
+- {{i:`NNG_OPT_SUB_PREFNEW`}}: \
+ (`bool`) \
+ \
+ This read/write option specifies the behavior of the subscriber when the queue is full.
+ When `true` (the default), the subscriber will make room in the queue by removing the oldest message.
+ When `false`, the subscriber will reject messages if the message queue does not have room.
+
+### Protocol Headers
+
+The _SUB_ protocol has no protocol-specific headers.
+
+[nng_sub_open]: TODO.md
+[nng_socket_set]: TODO.md
+[pub]: ./pub.md
diff --git a/docs/ref/proto/surveyor.md b/docs/ref/proto/surveyor.md
new file mode 100644
index 00000000..5222b629
--- /dev/null
+++ b/docs/ref/proto/surveyor.md
@@ -0,0 +1,129 @@
+# SURVEYOR protocol
+
+The {{i:*SURVEYOR* protocol}}{{hi:*SURVEYOR*}} is one half of a {{i:survey pattern}}.
+In this pattern, a surveyor sends a survey, which is broadcast to all
+peer respondents.
+The respondents then have a chance, but are not obliged, to reply.
+The survey itself is a timed event, so that responses
+received after the survey has finished are discarded.
+
+> [!TIP]
+> This protocol is useful in solving {{i:voting}} problems, such as
+> {{i:leader election}} in cluster configurations,
+> as well as certain kinds of {{i:service discovery}} problems.
+
+The _SURVEYOR_ protocol is the surveyor side, and the
+[_RESPONDENT_][respondent] protocol is the respondent side.
+
+### Socket Operations
+
+The [`nng_surveyor0_open`][nng_surveyor_open]
+functions create a surveyor socket.
+This socket may be used to send messages (surveys), and then to receive replies.
+A reply can only be received after sending a survey.
+A surveyor can normally expect to receive at most one reply from each responder.
+(Messages can be duplicated in some topologies,
+so there is no guarantee of this.)
+
+Attempts to receive on a socket with no outstanding survey will result
+in `NNG_ESTATE`.
+If the survey times out while the surveyor is waiting
+for replies, then the result will be `NNG_ETIMEDOUT`.
+
+Only one survey can be outstanding at a time; sending another survey will
+cancel the prior one, and any responses from respondents from the prior
+survey that arrive after this will be discarded.
+
+[Raw mode][raw] sockets ignore all these restrictions.
+
+### Context Operations
+
+This protocol supports the creation of [contexts][context] for concurrent
+use cases using [`nng_ctx_open`][nng_ctx_open].
+
+Each context can initiate its own surveys, and it will receive only
+responses to its own outstanding surveys.
+Other contexts on the same socket may have overlapping surveys
+operating at the same time.
+
+Each of these may have their own timeouts established with
+`NNG_OPT_SURVEYOR_SURVEYTIME`.
+
+Additionally, sending a survey on a context will only cancel an outstanding
+survey on the same context.
+
+> [!NOTE]
+> Due to the best-effort nature of this protocol, if too may contexts
+> are attempting to perform surveys simultaneously, it is possible for either
+> individual outgoing surveys or incoming responses to be lost.
+
+### Protocol Versions
+
+Only version 0 of this protocol is supported.
+At the time of writing, no other versions of this protocol have been defined.
+{{footnote: An earlier and incompatible version of the protocol was used in older
+pre-releases of [nanomsg][nanomsg], but was not released in any production version.}}
+
+### Protocol Options
+
+The following protocol-specific option is available.
+
+- {{i:`NNG_OPT_SURVEYOR_SURVEYTIME`}}: \
+ ([`nng_duration`][duration]) \
+ \
+ When a new survey is started, a timer of this duration is started.
+ Any responses arriving this time will be discarded.
+ Attempts to receive
+ after the timer expires with no other surveys started will result in
+ `NNG_ESTATE`.\
+ \
+ If a receive is pending when this timer expires, it will result in
+ `NNG_ETIMEDOUT`.
+
+### Protocol Headers
+
+{{hi:backtrace}}
+This form uses a stack of 32-bit big-endian identifiers.
+There _must_ be at least one identifier, the **survey ID**, which will be the
+last element in the array, and _must_ have the most significant bit set.
+
+There may be additional **peer ID**s preceding the survey ID.
+These will be distinguishable from the survey ID by having their most
+significant bit clear.
+
+When a survey message is received by a forwarding node (such as a
+[device][device]),
+the forwarding node prepends a
+32-bit peer ID (which _must_ have the most significant bit clear),
+which is the forwarder's way of identifying the directly connected
+peer from which it received the message.
+(This peer ID, except for the
+most significant bit, has meaning only to the forwarding node itself.)
+
+It may help to think of prepending a peer ID as pushing a peer ID onto the
+front of the stack of headers for the message.
+(It will use the peer ID
+it popped from the front to determine the next intermediate destination
+for the response.)
+
+When a response message is created, it is created using the same headers
+that the survey contained.
+
+A forwarding node can pop the peer ID it originally pushed on the
+message, stripping it from the front of the message as it does so.
+
+When the response finally arrives back at the initiating surveyor, it
+should have only a single element in the message, which will be the
+survey ID it originally used for the request.
+
+More detail can be found in the [sp-surveyor-01 RFC][survey_rfc] document.
+
+[nng_surveyor_open]: TODO.md
+[nng_ctx_open]: TODO.md
+[context]: TODO.md
+[nanomsg]: TODO.md
+[raw]: TODO.md
+[survey_rfc]: TODO.md
+[device]: TODO.md
+[duration]: ../api/util/nng_duration.md
+[respondent]: ./respondent.md