diff options
Diffstat (limited to 'RATIONALE.html')
| -rw-r--r-- | RATIONALE.html | 193 |
1 files changed, 159 insertions, 34 deletions
diff --git a/RATIONALE.html b/RATIONALE.html index 3ccbbb88..d6a3595b 100644 --- a/RATIONALE.html +++ b/RATIONALE.html @@ -8,8 +8,8 @@ title: Rationale: Or why am I bothering to rewrite nanomsg? <div class="details"> <span id="author" class="author">Garrett D’Amore</span><br> <span id="email" class="email"><a href="mailto:garrett@damore.org">garrett@damore.org</a></span><br> -<span id="revnumber">version 0.3,</span> -<span id="revdate">April 10, 2018</span> +<span id="revnumber">version 0.4,</span> +<span id="revdate">January 28, 2020</span> </div> </div> <div id="content"> @@ -73,7 +73,7 @@ and so forth.</p> <div class="paragraph"> <p>Perhaps it might be better to state that there were a number of opportunities to learn from the lessons of <em>nanomsg</em>, as well as lessons we learned while -building <em>nng</em> itself.</p> +building <em>NNG</em> itself.</p> </div> <div class="sect2"> <h3 id="_state_machine_madness">State Machine Madness</h3> @@ -85,7 +85,7 @@ machines is incredibly painful.</p> </div> <div class="paragraph"> <p>Worse, these state machines are designed to be run from a single worker -thread. This means that a given socket is entirely single theaded; you +thread. This means that a given socket is entirely single threaded; you could in theory have dozens, hundreds, or even thousands of connections open, but they would be serviced only by a single thread. (Admittedly non-blocking I/O is used to let the OS kernel calls run asynchronously @@ -151,7 +151,7 @@ This means that implementing new transports which might need something other than a file descriptor, is really non-trivial. This stymied my first attempt to add <a href="http://www.openssl.org">OpenSSL</a> support to get TLS added — <em>OpenSSL</em> has it’s own <code>struct BIO</code> for this stuff, and I could -not see an easy way to convert <em>nanomsg</em>'s <code>usock</code> stuff to accomodate the +not see an easy way to convert <em>nanomsg</em>'s <code>usock</code> stuff to accommodate the <code>struct BIO</code>.</p> </div> <div class="paragraph"> @@ -206,7 +206,7 @@ blocked in <code>poll()</code>.)</p> <div class="paragraph"> <p>But for many cases this is not necessary. A simple callback mechanism would be far better, with the FDs available only as an option for code -that needs them. This is the approach that we have taken with <em>nng</em>.</p> +that needs them. This is the approach that we have taken with <em>NNG</em>.</p> </div> <div class="paragraph"> <p>As another consequence of our approach, we do not require file descriptors @@ -245,7 +245,7 @@ constraints.</p> <div class="paragraph"> <p>Attempting to retain low numbered "socket descriptors" had its own problems — a huge source of use-after-close bugs, which made the -use of <code>nn_close()</code> incredibly dangerous for multithreaded sockets. +use of <code>nn_close()</code> incredibly dangerous for multi-threaded sockets. (If one thread closes and opens a new socket, other threads still using the old socket might wind up accessing the "new" socket without realizing it.)</p> @@ -258,12 +258,12 @@ coming to us from C++ (object oriented), <em>Java</em>, and <em>Python</ For them the BSD sockets API is frankly somewhat bizarre and alien.</p> </div> <div class="paragraph"> -<p>With <em>nng</em>, we realized that constraining ourselves to the mistakes of the -POSIX API was hurting rather than helping. So <em>nng</em> provides a much friendlier +<p>With <em>NNG</em>, we realized that constraining ourselves to the mistakes of the +POSIX API was hurting rather than helping. So <em>NNG</em> provides a much friendlier interface for getting properties associated with messages.</p> </div> <div class="paragraph"> -<p>In <em>nng</em> we also generally try hard to avoid reusing +<p>In <em>NNG</em> we also generally try hard to avoid reusing an identifier until no other option exists. This generally means most applications won’t see socket reuse until billions of other sockets have been opened. There is little chance for accidental reuse.</p> @@ -275,15 +275,15 @@ have been opened. There is little chance for accidental reuse.</p> <h2 id="_compatibility">Compatibility</h2> <div class="sectionbody"> <div class="paragraph"> -<p>Of course, there are a number of existing <em>nanomsg</em> consumers "in the wild" +<p>Of course, there are a number of existing <em>nanomsg</em> consumers “in the wild” already. It is important to continue to support them. So I decided from -the get go to implement a "compatibility" layer, that provides the same +the get go to implement a “compatibility” layer, that provides the same API, and as much as possible the same ABI, as legacy <em>nanomsg</em>. However, new features and capabilities would not necessarily be exposed to the the legacy API.</p> </div> <div class="paragraph"> -<p>Today <em>nng</em> offers this. You can relink an existing <em>nanomsg</em> binary against +<p>Today <em>NNG</em> offers this. You can relink an existing <em>nanomsg</em> binary against <em>libnng</em> instead of <em>libnn</em>, and it usually Just Works™. Source compatibility is almost as easy, although the application code needs to be modified to use different header files.</p> @@ -308,11 +308,11 @@ flag change would be needed. <h2 id="_asynchronous_io">Asynchronous IO</h2> <div class="sectionbody"> <div class="paragraph"> -<p>As a consequence of our experience with threads being so unscalable, +<p>As a consequence of our experience with threads being so un-scalable, we decided to create a new underlying abstraction modeled largely on Windows IO completion ports. (As bad as so many of the Windows APIs are, the IO completion port stuff is actually pretty nice.) Under the -hood in <em>nng</em> all I/O is asynchronous, and we have <code>nni_aio</code> objects +hood in <em>NNG</em> all I/O is asynchronous, and we have <code>nni_aio</code> objects for each pending I/O. These have an associated completion routine.</p> </div> <div class="paragraph"> @@ -321,7 +321,7 @@ for each pending I/O. These have an associated completion routine.</p> available number of CPU cores to ensure that we never wait while a CPU core is available for work), but they can be run "synchronously" if the I/O provider knows it is safe to do so (for example the completion -is occuring in a context where no locks are held.)</p> +is occurring in a context where no locks are held.)</p> </div> <div class="paragraph"> <p>The <code>nni_aio</code> structures are accessible to user applications as well, which can @@ -340,7 +340,7 @@ but that work is already in progress.</p> <h2 id="_portability_embeddability">Portability & Embeddability</h2> <div class="sectionbody"> <div class="paragraph"> -<p>A significant goal of <em>nng</em> is to be portable to many kinds of different +<p>A significant goal of <em>NNG</em> is to be portable to many kinds of different kinds of systems, and embedded in systems that do not support POSIX or Win32 APIs. To that end we have a clear platform portability layer. We do require that platforms supply entry points for certain networking, synchronization, @@ -372,7 +372,7 @@ If you want to write a coroutine-based platform, let me know! <h2 id="_new_transports">New Transports</h2> <div class="sectionbody"> <div class="paragraph"> -<p>The other, most critical, motivation behind <em>nng</em> was to enable an easier +<p>The other, most critical, motivation behind <em>NNG</em> was to enable an easier creation of new transports. In particular, one client ( <a href="http://www.capitar.com">Capitar IT Group BV</a>) contracted the creation of a <a href="http://www.zerotier.com">ZeroTier</a> transport for @@ -385,7 +385,7 @@ I had created for <em>mangos</em>.</p> </div> <div class="paragraph"> <p>In retrospect, I’m not sure that the answer was a clear and definite yes -in favor of <em>nng</em>, but for the other things I want to do, it has enabled a +in favor of <em>NNG</em>, but for the other things I want to do, it has enabled a lot of new work. The ZeroTier transport was created with a relatively modest amount of effort, in spite of being based upon a connectionless transport. I do not believe I could have done this easily in the existing @@ -394,12 +394,12 @@ transport. I do not believe I could have done this easily in the existing <div class="paragraph"> <p>I’ve since added a rich TLS transport, and have implemented a WebSocket transport that is far more capable than that in <em>nanomsg</em>, as it can -support TLS and sharing the TCP port across multiple <em>nng</em> sockets (using +support TLS and sharing the TCP port across multiple <em>NNG</em> sockets (using the path to discriminate) or even other HTTP services.</p> </div> <div class="paragraph"> <p>There are already plans afoot for other kinds of transports using QUIC -or KCP or SSH, as well as a pure UDP transport. The new <em>nng</em> transport +or KCP or SSH, as well as a pure UDP transport. The new <em>NNG</em> transport layer makes implementation of these all fairly straight-forward.</p> </div> </div> @@ -410,9 +410,9 @@ layer makes implementation of these all fairly straight-forward.</p> <div class="paragraph"> <p>As part of implementing a real WebSocket transport, it was necessary to implement at least some HTTP capabilities. Rather than just settle for a toy -implementation, <em>nng</em> has a very capable HTTP server and client framework. +implementation, <em>NNG</em> has a very capable HTTP server and client framework. The server can be used to build real web services, so it becomes possible -for example to serve static content, REST API, and <em>nng</em> based services +for example to serve static content, REST API, and <em>NNG</em> based services all from the same TCP port using the same program.</p> </div> <div class="paragraph"> @@ -421,13 +421,35 @@ a plethora of other kinds of transports and services.</p> </div> <div class="paragraph"> <p>There is also a portability layer — so some common services (threading, -timing, etc.) are provided in the <em>nng</em> library to help make writing -portable <em>nng</em> applications easier.</p> +timing, etc.) are provided in the <em>NNG</em> library to help make writing +portable <em>NNG</em> applications easier.</p> </div> <div class="paragraph"> -<p>It will not surprise me if developers start finding uses for <em>nng</em> that +<p>It will not surprise me if developers start finding uses for <em>NNG</em> that have nothing to do with Scalability Protocols.</p> </div> +<div class="sidebarblock"> +<div class="content"> +<div class="paragraph"> +<p>Actually, now in 2020, this has come to pass. There exists, for +example, a project that provides a full REST API framework built on top +NNG.</p> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_streaming_api">Streaming API</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Along with the ability to use Scalability Protocols, and a generic +HTTP framework, we’ve provided a nice abstraction for byte-stream +oriented protocols. This makes it possible to build portable applications +that can use WebSockets, TCP, IPC, and TLS all in much the same way. +In 2019, we refactored much of the core Scalability Protocols code to +be based on top of this work.</p> +</div> </div> </div> <div class="sect1"> @@ -437,7 +459,7 @@ have nothing to do with Scalability Protocols.</p> <p>As part of working on a demo suite of applications, I realized that the requirement to use raw mode sockets for concurrent applications was rather onerous, forcing application developers to re-implement much of the -same logic that is already in <em>nng</em>.</p> +same logic that is already in <em>NNG</em>.</p> </div> <div class="paragraph"> <p>Thus was the born the idea of separating the context for protocols from @@ -457,7 +479,7 @@ always, if it has no need for extra concurrency.</p> </div> <div class="paragraph"> <p>One side benefit of this work was that we identified several places -to make <em>nng</em> perform more efficiently, reducing the number of context +to make <em>NNG</em> perform more efficiently, reducing the number of context switches and extra raw vs. cooked logic.</p> </div> </div> @@ -466,14 +488,117 @@ switches and extra raw vs. cooked logic.</p> <h2 id="_towards_nanomsg_2_0">Towards <em>nanomsg</em> 2.0</h2> <div class="sectionbody"> <div class="paragraph"> -<p>It is my intention that <em>nng</em> ultimately replace <em>nanomsg</em>. I do think of it -as "nanomsg 2.0". In fact "nng" stands for "nanomsg next generation" in -my mind. Some day soon I’m hoping that the various website -references to nanomsg my simply be updated to point at <em>nng</em>. It is not -clear to me whether at that time I will simply rename the existing -code to <em>nanomsg</em>, nanomsg2, or leave it as <em>nng</em>.</p> +<p>It is my intention that <em>NNG</em> ultimately replace <em>nanomsg</em>. I do think of it +as "nanomsg 2.0". In fact “NNG” stands for "nanomsg next generation" in mind. +Some day soon I’m hoping that the various website +references to nanomsg my simply be updated to point at <em>NNG</em>. +It is not clear to me whether at that time I will simply rename the existing +code to <em>nanomsg</em>, nanomsg2, or leave it as <em>NNG</em>.</p> +</div> +<div class="sidebarblock"> +<div class="content"> +<div class="paragraph"> +<p>My thinking in 2020, is that the <em>nanomsg</em> has significant value, but now +<em>NNG</em> has carved an identity out of it’s own, as has mangos. +The legacy <em>nanomsg</em> project is likely to thus be re-branded as <em>libnanomsg</em>, +and we will re-purpose the <em>nanomsg</em> brand to represent the entire family +of projects around these protocols. In fact <em>NNG</em> has already been +relocated to it’s own <a href="https://nng.nanomsg.org">site</a> under +<a href="https://nanomsg.org">nanomsg.org</a>.</p> +</div> </div> </div> </div> </div> +<div class="sect1"> +<h2 id="_future_directions">Future Directions</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>In the few years since I started the <em>NNG</em> project, it’s become apparent +that a few enhancements are sorely needed. A few things we have planned +as of January 2020 are:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>support for pluggable TLS libraries (including FIPS and liberal licensed options)</p> +</li> +<li> +<p>better support for UDP (including multicast when appropriate)</p> +</li> +<li> +<p>new wire protocols to negotiate capabilities between peers</p> +</li> +<li> +<p>publisher side filtered PUB/SUB protocol</p> +</li> +<li> +<p>a true mesh protocol, including directed unicast and peer discovery</p> +</li> +<li> +<p>better support for QoS tuning in certain protocols</p> +</li> +<li> +<p>richer statistics and observability</p> +</li> +<li> +<p>support for HTTP/2</p> +</li> +<li> +<p>more platform ports to platforms like FreeRTOS</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Of course, we will also continue to make strides in improving the performance +and scalability of the library. In the past several months alone there have +been significant moves in this direction, and we’ve identified several other +major opportunities to improve performance and reduce latency.</p> +</div> +<div class="paragraph"> +<p>Also, we’ve been working to improve our test coverage, which has uncovered +a raft of bugs (and we expect will continue to help us find and squash them), +as well as identifying some areas to tighten the code leading to both +simpler, and more efficient implementation.</p> +</div> +<div class="paragraph"> +<p>The future of NNG is looking bright, indeed.</p> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="_history">History</h2> +<div class="sectionbody"> +<table class="tableblock frame-all grid-all stretch"> +<colgroup> +<col style="width: 33.3333%;"> +<col style="width: 33.3333%;"> +<col style="width: 33.3334%;"> +</colgroup> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">v0.4</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">January 28, 2020</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Branding, updates for the new decade.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">v0.3</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">April 12, 2018</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Minor fixes, added contexts.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">v0.2</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">February 23, 2018</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Updates with new information, markup fixes.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">v0.1</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">October 18, 2017</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">First version.</p></td> +</tr> +</tbody> +</table> +</div> +</div> +</div> </main> |