Move some docs to docs directory, add CONTRIBUTING and templates.

3 files changed, 516 insertions, 0 deletions

diff --git a/docs/CODE_OF_CONDUCT.adoc b/docs/CODE_OF_CONDUCT.adoc
new file mode 100644
index 00000000..80176a21
--- /dev/null
+++ b/docs/CODE_OF_CONDUCT.adoc
@@ -0,0 +1,74 @@
+= NNG Code of Conduct
+
+== Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of
+experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+== Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+= Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+== Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+== Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at garrett@damore.org. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an
+incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+== Attribution
+
+This Code of Conduct is adapted from the
+https://www.contributor-convent.org[Contributor Covenant], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
diff --git a/docs/CONTRIBUTING.adoc b/docs/CONTRIBUTING.adoc
new file mode 100644
index 00000000..1ec5d7eb
--- /dev/null
+++ b/docs/CONTRIBUTING.adoc
@@ -0,0 +1,126 @@
+= NNG Contributing Guidelines
+
+Legal stuff:
+
+For any submission you make to us, either by filing an issue, or submitting a
+defect, or sending us a message, you agree and certify that:
+
+        ** You are the author or legal owner of the content submitted, or
+           you have authorization to submit the content by the owner
+           (such as your employer).
+
+        ** Unless the submission is otherwise labeled, you or the owner
+           are granting the NNG Project, its contributors, and licensees,
+           a non-exclusive license to your use submission under the terms of
+           MIT License (as documented in the <<LICENSE#,LICENSE>> file in this
+           repository.)  This grant shall include a right to use for any
+           patents covering content in the submission.
+
+        ** You are not aware of any legal encumberances to the use of the
+           submission (such as patents held by others) unless explicitly
+           stated otherwise.
+
+        ** You have read, and agree to abide by, our
+           <<CODE_OF_CONDUCT#,Code of Conduct>>.
+
+        ** You understand that participation in this project is voluntary,
+           and at no point shall any of the project members or leadership
+           be obligated to review, merge, or handle any submission.
+
+== Filing Issues
+
+We use github issues for NNG.  We ask that you keep in mind the following
+when filing issues:
+
+* First look to see if your issue is already covered by an existing issue.
+
+* Synopsis should be brief, but descriptive. (We may alter the synopsis
+  to correct its accuracy or improve its brevity.)
+
+* Include details about the platform (operating system, language).
+
+* Include CMake configuration used to build NNG.
+
+* Inclue expected results or behavior, and observed results or behavior.
+
+* If at all possible, a reproducible test case is helpful. We prefer test
+  cases as the minimal amount of C code to demonstrate the defect.
+
+* Please file issues only when filing things that should be fixed by
+  changes to the source or documentation in the repository (such as bugs
+  or feature requests).  For support questions and other discussion, please
+  instead use the mailing list or Gitter chat.
+
+== Pull Requests
+
+* We use Github PRs
+
+* Please follow the existing coding style.
+
+        ** See the `.clang-format` for whitespace rules.
+        ** Return values should be enclosed in parenthesis.
+        ** Conditional clauses are always enclosed in braces, and thus
+           span multiple lines.
+
+TIP: The `.clang-format` file and `clang-format` may be helpful, but
+is not necessarily complete.  Also you may need to use a recent version
+of `clang-format` as older versions can differ in their output.
+
+* If you submit a PR, we would appreciate it if you would respond to review
+  feedback, rather than just leaving your baby on our doorstep.  Abandoned
+  PRs or issues may be closed without further action.
+
+* Commits should have comments formatted like this:
+
+----
+    fixes #<issue#1> <issue#1-synopsis>
+    fixes #<issue#2> <issue#2-synopsis>
+
+    <Comments describing the issues, or fix, that you want to have
+    in the commit history.  Not every issue needs such comments.>
+----
+
+* We have an "always release ready" policy for our tree, meaning we should
+  be able to build a functional release at any commit boundary.  This ensures
+  that others taking the work can use git bisect and similar approaches when
+  hunting down regressions, or when cherry-picking changes for their own
+  forked copies.  This has a few consequences:
+
+	** PRs should be functionally complete (that is, don't submit a
+           PR with partial work in progress, unless you have reached a 
+           milestone that can stand on its own merit.)
+
+	** Each commit in a PR should also be "complete", so that the
+           a release could be cut in between your commits without
+           leaving the tree in a broken or dysfunctional state.
+
+        ** Usually this means its best to address only one issue in a PR,
+           with just one commit (squashed).
+
+        ** Please have already merged and tested your code against the
+           latest changes in master.  (If your code has merge conflicts,
+           your PR will be rejected until you have fixed the conflicts.)
+
+* As a special exception, you may submit a PR for handling by our test
+  infrastructure; in that case please clearly identify that the PR is
+  submitted for that purpose, and should not considered for integration
+  yet.  When that testing is done, please close the PR, and submit a new
+  one when you are ready for final review and integration.
+
+* Code submitted should contain updates to any copyright files to add
+  your copyright to the license.  We prefer you follow the existing
+  style of copyright, `// Copyright <year> <owner> <email or url>`
+
+NOTE: We only use the most recent year, and we do not use `\(C)` which has
+no legal value when the word "`Copyright`" is already present. We also
+do not use the old "`All rights reserved.`" statement, which is equally
+meaningless under current international copyright law.
+
+* When updating an existing file, any new Copyright notice should be
+  added immediately below the pre-existing one, just before the license
+  boilerplate.
+
+* All new files should use the MIT license and the same boilerplate
+  from one of the other files.  We do not normally approve submissions 
+  with different license or boilerplates -- please contact us if you
+  have a compelling reason for wanting to do that.
diff --git a/docs/RATIONALE.adoc b/docs/RATIONALE.adoc
new file mode 100644
index 00000000..b4ef5468
--- /dev/null
+++ b/docs/RATIONALE.adoc
@@ -0,0 +1,316 @@
+= Rational: Or why am I bothering to rewrite nanomsg?
+Garrett D'Amore <garrett@damore.org>
+v0.2, February 22, 2018
+
+
+NOTE: You might want to review
+      http://nanomsg.org/documentation-zeromq.html[Martin Sustrik's rationale]
+      for nanomsg vs. ZeroMQ.
+
+
+== Background
+
+I became involved in the
+http://www.nanomsg.org[nanomsg] community back in 2014, when
+I wrote https://github.com/go-mangos/mangos[mangos] as a pure
+http://www.golang.org[Go] implementation of the wire protocols behind
+_nanomsg_.  I did that work because I was dissatisfied with the
+http://zeromq.org[_ZeroMQ_] licensing model
+and the {cpp} baggage that came with it. I also needed something that would
+work with _Go_ on http://www.illumos.org[illumos], which at the time
+lacked support for `cgo` (so I could not just use an FFI binding.)
+
+
+At the time, it was the only alternate implementation those protocols.
+Writing _mangos_ gave me a lot of detail about the internals of _nanomsg_ and
+the SP protocols.
+
+It would not be wrong to say that one of the goals of _mangos_ was to teach
+me about _Go_.  It was my first non-trivial _Go_ project.
+
+While working with _mangos_, I wound up implementing a number of additional
+features, such as a TLS transport, the ability to bind to wild card ports,
+and the ability to determine more information about the sender of a message.
+This was incredibly useful in a number of projects.
+
+I initially looked at _nanomsg_ itself, as I wanted to add a TLS transport
+to it, and I needed to make some bug fixes (for protocol bugs for example),
+and so forth.
+
+== Lessons Learned
+
+Perhaps it might be better to state that there were a number of opportunities
+to learn from the lessons of _nanomsg_, as well as lessons we learned while
+building _nng_ itself.
+
+=== State Machine Madness
+
+What I ran into in _nanomsg_, when attempting to improve it, was a
+challenging mess of state machines. _nanomsg_ has dozens of state machines,
+many of which feed into others, such that tracking flow through the state
+machines is incredibly painful.
+
+Worse, these state machines are designed to be run from a single worker
+thread.  This means that a given socket is entirely single theaded; you
+could in theory have dozens, hundreds, or even thousands of connections
+open, but they would be serviced only by a single thread.  (Admittedly
+non-blocking I/O is used to let the OS kernel calls run asynchronously
+perhaps on multiple cores, but nanomsg itself runs all socket code on
+a single worker thread.)
+
+There is another problem too -- the `inproc` code that moves messages
+between one socket and another was incredibly racy.  This is because the
+two sockets have different locks, and so dealing with the different
+contexts was tricky (and consequently buggy).  (I've since, I think, fixed
+the worst of the bugs here, but only after many hours of pulling out hair.)
+
+The state machines also make fairly linear flow really difficult to follow.
+For example, there is a state machine to read the header information.  This
+may come a byte a time, and the state machine has to add the bytes, check
+for completion, and possibly change state, even if it is just reading a
+single 32-bit word.  This is a lot more complex than most programmers are
+used to, such as `read(fd, &val, 4)`.
+
+Now to be fair, Martin Sustrik had the best intentions when he created the
+state machine model around which _nanomsg_ is built.  I do think that from
+experience this is one of the most dense and unapproachable parts of _nanomsg_,
+in spite of the fact that Martin's goal was precisely the opposite.  I
+consider this a "failed experiment" -- but hey failed experiments are the
+basis of all great science.
+
+=== Thread Challenges
+
+While _nanomsg_ is mostly internally single threaded, I decided to try to
+emulate the simple architecture of _mangos_ using system threads.  (_mangos_
+benefits greatly from _Go_'s excellent coroutine facility.)  Having been well
+and truly spoiled by _illumos_ threading (and especially _illumos_ kernel
+threads), I thought this would be a reasonable architecture.
+
+Sadly, this initial effort, while it worked, scaled incredibly poorly --
+even so-called "modern" operating systems like _macOS_ 10.12 and _Windows_ 8.1
+simply melted or failed entirely when creating any non-trivial number of
+threads.  (To me, creating 100 threads should be a no-brainer, especially if
+one limits the stack size appropriately.  I'm used to be able to create
+thousands of threads without concern.  As I said, I've been spoiled.
+If your system falls over at a mere 200 threads I consider it a toy
+implementation of threading. Unfortunately most of the mainstream operating
+systems are therefore toy implementations.)
+
+Chalk up another failed experiment.
+
+I did find another approach which is discussed further.
+
+=== File Descriptor Driven
+
+Most of the underlying I/O in _nanomsg_ is built around file descriptors,
+and it's internal usock structure, which is also state machine driven.
+This means that implementing new transports which might need something
+other than a file descriptor, is really non-trivial.  This stymied my
+first attempt to add http://www.openssl.org[OpenSSL] support to get TLS
+added -- _OpenSSL_ has it's own `struct BIO` for this stuff, and I could
+not see an easy way to convert _nanomsg_'s `usock` stuff to accomodate the
+`struct BIO`.
+
+In retrospect, _OpenSSL_ wasn't the ideal choice for an SSL/TLS library,
+and we have since chosen another (https://tls.mbed.org[mbed TLS]).
+Still, we needed an abstraction model that was better than just file
+descriptors for I/O.
+
+=== Poll
+
+In order to support use in event driven programming, asynchronous
+situations, etc. _nanomsg_ offers non-blocking I/O.  In order to make
+this work for end-users, a notification mechanism is required, and
+nanomsg, in the spirit of following POSIX, offers a notification method
+based on `poll(2)` or `select(2)`.
+
+In order for this to work, it offers up a selectable file descriptor
+for send and another one for receive.  When events occur, these are
+written to, and the user application "clears" these by reading from
+them.  (This is done on behalf of the application by _nanomsg_'s API calls.)
+
+This means that in addition to the context switch code, there are not
+fewer than 2 extra system calls executed per message sent or received, and
+on a mostly idle system as many as 3.  This means that to send a message
+from one process to another you may have to execute up to 6 extra system
+calls, beyond the 2 required to actually send and receive the message.
+
+NOTE: Its even more hideous to support this on Windows, where there is no
+      `pipe(2)` system call, so we have to cobble up a loopback TCP connection
+      just for this event notification, in addition to the system call
+      explosion.
+
+There are cases where this file descriptor logic is easier for existing
+applications to integrate into event loops (e.g. they already have a thread
+blocked in `poll()`.)
+
+But for many cases this is not necessary.  A simple callback mechanism
+would be far better, with the FDs available only as an option for code
+that needs them.  This is the approach that we have taken with _nng_.
+
+As another consequence of our approach, we do not require file descriptors
+for sockets at all, so it is possible to create applications containing
+_many_ thousands of `inproc` sockets with no files open at all.  (Obviously
+if you're going to perform real I/O to other processes or other systems,
+you're going to need to have the underlying transport file descriptors
+open, but then the only real limit should be the number of files that you
+can open on your system.  And the number of active connections you can maintain
+should ideally approach that system limit closely.)
+
+=== POSIX APIs
+
+Another of Martin's goals, which seems worthwhile at first, was the
+attempt to provide a familiar POSIX API (based upon the BSD socket API).
+As a C programmer coming from UNIX systems, this really attracted me.
+
+The problem is that the POSIX APIs are actually really horrible.  In
+particular the semantics around `cmsg` are about as arcane and painful as
+one can imagine.  Largely, this has meant that extensions to the `cmsg
+API simply have not occurred in _nanomsg_.
+
+The `cmsg` API specified by POSIX is as bad as it is because POSIX had
+requirements not to break APIs that already existed, and they needed to
+shim something that would work with existing implementations, including
+getting across a system call boundary. _nanomsg_ has never had such
+constraints.
+
+Oh, and there was that whole "design by committee" aspect.
+
+Attempting to retain low numbered "socket descriptors" had its own
+problems -- a huge source of use-after-close bugs, which made the
+use of `nn_close()` incredibly dangerous for multithreaded sockets.
+(If one thread closes and opens a new socket, other threads still using
+the old socket might wind up accessing the "new" socket without realizing
+it.)
+
+The other thing is that BSD socket APIs are super familiar to UNIX C
+programmers -- but experience with _nanomsg_ has taught us already that these
+are actually in the minority of _nanomsg_'s users.  Most of our users are
+coming to us from {cpp} (object oriented), _Java_, and _Python_ backgrounds.
+For them the BSD sockets API is frankly somewhat bizarre and alien.
+
+With _nng_, we realized that constraining ourselves to the mistakes of the
+POSIX API was hurting rather than helping. So _nng_ provides a much friendlier
+interface for getting properties associated with messages.
+
+In _nng_ we also generally try hard to avoid reusing
+an identifier until no other option exists.  This generally means most
+applications won't see socket reuse until billions of other sockets
+have been opened.  There is little chance for accidental reuse.
+
+
+== Compatibility
+
+Of course, there are a number of existing _nanomsg_ consumers "in the wild"
+already.  It is important to continue to support them.  So I decided from
+the get go to implement a "compatibility" layer, that provides the same
+API, and as much as possible the same ABI, as legacy _nanomsg_.  However,
+new features and capabilities would not necessarily be exposed to the
+the legacy API.
+
+Today _nng_ offers this.  You can relink an existing _nanomsg_ binary against
+_libnng_ instead of _libnn_, and it usually Just Works(TM).  Source
+compatibility is almost as easy, although the application code needs to be
+modified to use different header files.
+
+NOTE: I am considering changing the include file in the future so that
+it matches exactly the _nanomsg_ include path, so that only a compiler
+flag change would be needed.
+
+== Asynchronous IO
+
+As a consequence of our experience with threads being so unscalable,
+we decided to create a new underlying abstraction modeled largely on
+Windows IO completion ports.  (As bad as so many of the Windows APIs
+are, the IO completion port stuff is actually pretty nice.)  Under the
+hood in _nng_ all I/O is asynchronous, and we have `nni_aio` objects
+for each pending I/O.  These have an associated completion routine.
+
+The completion routines are _usually_ run on a separate worker thread
+(we have many such workers; in theory the number should be tuned to the
+available number of CPU cores to ensure that we never wait while a CPU
+core is available for work), but they can be run "synchronously" if
+the I/O provider knows it is safe to do so (for example the completion
+is occuring in a context where no locks are held.)
+
+The `nni_aio` structures are accessible to user applications as well, which can
+lead to much more efficient and easier to write asynchronous applications,
+and can aid integration into event-driven systems and runtimes, without
+requiring extra system calls required by the legacy _nanomsg_ approach.
+
+There is still performance tuning work to do, especially optimization for
+specific pollers like `epoll()` and `kqueue()` to address the C10K problem,
+but that work is already in progress.
+
+== Portability & Embeddability
+
+A significant goal of _nng_ is to be portable to many kinds of different
+kinds of systems, and embedded in systems that do not support POSIX or Win32
+APIs.  To that end we have a clear platform portability layer.  We do require
+that platforms supply entry points for certain networking, synchronization,
+threading, and timekeeping functions, but these are fairly straight-forward
+to implement on any reasonable 32-bit or 64-bit system, including most
+embedded operating systems.
+
+Additionally, this portability layer may be used to build other kinds of
+experiments -- for example it should be relatively straight-forward to provide
+a "platform" based on one of the various coroutine libraries such as Martin's
+http://libdill.org[libdill] or https://swtch.com/libtask/[libtask].
+
+TIP: If you want to write a coroutine-based platform, let me know!
+
+== New Transports
+
+The other, most critical, motivation behind _nng_ was to enable an easier
+creation of new transports.  In particular, one client (
+http://www.capitar.com[Capitar IT Group BV])
+contracted the creation of a http://www.zerotier.com[ZeroTier] transport for
+_nanomsg_.
+
+After beating my head against the state machines some more, I finally asked
+myself if it would not be easier just to rewrite _nanomsg_ using the model
+I had created for _mangos_.
+
+In retrospect, I'm not sure that the answer was a clear and definite yes
+in favor of _nng_, but for the other things I want to do, it has enabled a
+lot of new work.  The ZeroTier transport was created with a relatively
+modest amount of effort, in spite of being based upon a connectionless
+transport.  I do not believe I could have done this easily in the existing
+_nanomsg_.
+
+I've since added a rich TLS transport, and have implemented a WebSocket
+transport that is far more capable than that in _nanomsg_, as it can
+support TLS and sharing the TCP port across multiple _nng_ sockets (using
+the path to discriminate) or even other HTTP services.
+
+There are already plans afoot for other kinds of transports using QUIC
+or KCP or SSH, as well as a pure UDP transport.  The new _nng_ transport
+layer makes implementation of these all fairly straight-forward.
+
+== HTTP and Other services
+
+As part of implementing a real WebSocket transport, it was necessary to
+implement at least some HTTP capabilities.  Rather than just settle for a toy
+implementation, _nng_ has a very capable HTTP server and client framework. 
+The server can be used to build real web services, so it becomes possible
+for example to serve static content, REST API, and _nng_ based services
+all from the same TCP port using the same program.
+
+We've also made the WebSocket services fairly generic, which may support
+a plethora of other kinds of transports and services.
+
+There is also a portability layer -- so some common services (threading,
+timing, etc.) are provided in the _nng_ library to help make writing
+portable _nng_ applications easier.
+
+It will not surprise me if developers start finding uses for _nng_ that
+have nothing to do with Scalability Protocols.
+
+== Towards _nanomsg_ 2.0
+
+It is my intention that _nng_ ultimately replace _nanomsg_.  I do think of it
+as "nanomsg 2.0".  In fact "nng" stands for "nanomsg next generation" in
+my mind.  Some day before too long I'm hoping that the various website
+references to nanomsg my simply be updated to point at _nng_.  It is not
+clear to me whether at that time I will simply rename the existing
+code to _nanomsg_, nanomsg2, or leave it as _nng_.