adds nng_dialer_start_aio (#2163)

* adds nng_dialer_start_aio

This change adds `nng_dialer_start_aio` (if you have a better name, I'm
happy to change it), whose docs read:

> `nng_dialer_start_aio` starts the endpoint dialing asynchronously.
> This is only possible if the dialer is not already dialing. Unlike
> `nng_dialer_start`, this accepts an AIO such that the caller can learn
> when the dialing eventually succeeds or fails. The supplied AIO must
> have been initialized, and is only triggered with the result of the
> first dial attempt.

This new function makes it possible for applications to perform a
non-blocking dial, but still later be notified of the result of that
dial. Arguably, this obviates the need for `NNG_FLAG_NONBLOCK` in
`dialer_start` altogether, but no need to break backwards compatibility.

There is technically a functional change here, which is that the
"Starting dialer for socket" message now gets printed _before_ the dial
is complete in the blocking case, rather than after. It's possible to
change this if we're willing to make the code slightly more complicated,
but given it says "Starting", not "Started", this change felt fine.

6 files changed, 140 insertions, 10 deletions

diff --git a/docs/man/nng_dialer_start.3.adoc b/docs/man/nng_dialer_start.3.adoc
index b6cb6e89..fa25e7a6 100644
--- a/docs/man/nng_dialer_start.3.adoc
+++ b/docs/man/nng_dialer_start.3.adoc
@@ -44,7 +44,8 @@ Furthermore, if the connection was closed for a synchronously dialed
 connection, the dialer will still attempt to redial asynchronously.
 
 TIP: While `NNG_FLAG_NONBLOCK` can help an application be more resilient,
-it also generally makes diagnosing failures somewhat more difficult.
+it also generally makes diagnosing failures somewhat more difficult. You may
+want to use xref:nng_dialer_start_aio.3.adoc[`nng_dialer_start_aio()`].
 
 Once a dialer has started, it is generally not possible to change
 its configuration.
@@ -61,6 +62,7 @@ This function returns 0 on success, and non-zero otherwise.
 `NNG_ECONNREFUSED`:: The remote peer refused the connection.
 `NNG_ECONNRESET`:: The remote peer reset the connection.
 `NNG_EINVAL`:: An invalid set of _flags_ was specified.
+`NNG_ENOENT`:: The dialer handle is invalid.
 `NNG_ENOMEM`:: Insufficient memory is available.
 `NNG_EPEERAUTH`:: Authentication or authorization failure.
 `NNG_EPROTO`:: A protocol error occurred.
@@ -72,6 +74,7 @@ This function returns 0 on success, and non-zero otherwise.
 [.text-left]
 xref:nng_dial.3.adoc[nng_dial(3)],
 xref:nng_dialer_create.3.adoc[nng_dialer_create(3)]
+xref:nng_dialer_start_aio.3.adoc[nng_dialer_start_aio(3)]
 xref:nng_strerror.3.adoc[nng_strerror(3)],
 xref:nng_dialer.5.adoc[nng_dialer(5)],
 xref:nng.7.adoc[nng(7)]
diff --git a/docs/man/nng_dialer_start_aio.3.adoc b/docs/man/nng_dialer_start_aio.3.adoc
new file mode 100644
index 00000000..0414bd28
--- /dev/null
+++ b/docs/man/nng_dialer_start_aio.3.adoc
@@ -0,0 +1,75 @@
+= nng_dialer_start_aio(3)
+//
+// Copyright 2018 Staysail Systems, Inc. <info@staysail.tech>
+// Copyright 2018 Capitar IT Group BV <info@capitar.com>
+//
+// This document is supplied under the terms of the MIT License, a
+// copy of which should be located in the distribution where this
+// file was obtained (LICENSE.txt).  A copy of the license may also be
+// found online at https://opensource.org/licenses/MIT.
+//
+
+== NAME
+
+nng_dialer_start_aio - start dialer with an asynchronous I/O handle
+
+== SYNOPSIS
+
+[source, c]
+----
+#include <nng/nng.h>
+
+int nng_dialer_start_aio(nng_dialer d, int flags, nng_aio *aiop);
+----
+
+== DESCRIPTION
+
+The `nng_dialer_start_aio()` function starts the dialer _d_ using the
+explicit asynchronous I/O operation handle _aiop_.
+
+This causes the dialer to asynchronously start connecting to the address with
+which it was created.
+
+When a connection is established, it results in a pipe being created,
+which will be attached to the dialer's socket.
+
+This function must be called with `NNG_FLAG_NONBLOCK` set in _flags_ to
+indicate that the connection attempt should be made asynchronously. Failing to
+do so will return `NNG_EINVAL`.
+
+When the dialer finishes dialing, it invokes
+xref:nng_aio_finish.3.adoc[`nng_aio_finish()`] with the dialing result. Only
+the first dialing result is communicated this way.
+
+Once a dialer has started, it is generally not possible to change
+its configuration.
+
+== RETURN VALUES
+
+None. (The operation completes asynchronously.)
+
+== ERRORS
+
+[horizontal]
+`NNG_EADDRINVAL`:: An invalid _url_ was specified.
+`NNG_ECLOSED`:: The socket _s_ is not open.
+`NNG_ECONNREFUSED`:: The remote peer refused the connection.
+`NNG_ECONNRESET`:: The remote peer reset the connection.
+`NNG_EINVAL`:: An invalid set of _flags_ was specified.
+`NNG_ENOENT`:: The dialer handle is invalid.
+`NNG_ENOMEM`:: Insufficient memory is available.
+`NNG_EPEERAUTH`:: Authentication or authorization failure.
+`NNG_EPROTO`:: A protocol error occurred.
+`NNG_ESTATE`:: The dialer _d_ is already started.
+`NNG_EUNREACHABLE`:: The remote address is not reachable.
+
+== SEE ALSO
+
+[.text-left]
+xref:nng_aio_result.3.adoc[nng_aio_result(3)]
+xref:nng_dial.3.adoc[nng_dial(3)],
+xref:nng_dialer_create.3.adoc[nng_dialer_create(3)]
+xref:nng_dialer_start.3.adoc[nng_dialer_start(3)]
+xref:nng_strerror.3.adoc[nng_strerror(3)],
+xref:nng_dialer.5.adoc[nng_dialer(5)],
+xref:nng.7.adoc[nng(7)]
diff --git a/include/nng/nng.h b/include/nng/nng.h
index a71a3f53..a6c797ab 100644
--- a/include/nng/nng.h
+++ b/include/nng/nng.h
@@ -385,6 +385,14 @@ NNG_DECL int nng_listener_create_url(
 // the dialer is not already dialing.
 NNG_DECL int nng_dialer_start(nng_dialer, int);
 
+// nng_dialer_start_aio starts the endpoint dialing asynchronously.  This is
+// only possible if the dialer is not already dialing.  Unlike
+// nng_dialer_start, this accepts an AIO such that the caller can learn when
+// the dialing eventually succeeds or fails.  The supplied AIO must have been
+// initialized, and is only triggered with the result of the first dial
+// attempt.
+NNG_DECL void nng_dialer_start_aio(nng_dialer, int, nng_aio *);
+
 // nng_listener_start starts the endpoint listening.  This is only possible if
 // the listener is not already listening.
 NNG_DECL int nng_listener_start(nng_listener, int);
diff --git a/src/core/dialer.c b/src/core/dialer.c
index bef110d7..9b868224 100644
--- a/src/core/dialer.c
+++ b/src/core/dialer.c
@@ -458,22 +458,16 @@ nni_dialer_start(nni_dialer *d, unsigned flags)
 	nni_aio  aio;
 	nni_aio *aiop = NULL;
 
-	if (nni_atomic_flag_test_and_set(&d->d_started)) {
-		return (NNG_ESTATE);
-	}
-
 	if ((flags & NNG_FLAG_NONBLOCK) != 0) {
 		aiop = NULL;
 	} else {
 		nni_aio_init(&aio, NULL, NULL);
 		aiop = &aio;
-		nni_aio_start(aiop, NULL, NULL);
 	}
 
-	nni_mtx_lock(&d->d_mtx);
-	d->d_user_aio = aiop;
-	dialer_connect_start(d);
-	nni_mtx_unlock(&d->d_mtx);
+	if ((rv = nni_dialer_start_aio(d, flags, aiop)) != 0) {
+		return (rv);
+	}
 
 	if (aiop != NULL) {
 		nni_aio_wait(aiop);
@@ -481,6 +475,31 @@ nni_dialer_start(nni_dialer *d, unsigned flags)
 		nni_aio_fini(aiop);
 	}
 
+	return (rv);
+}
+
+int
+nni_dialer_start_aio(nni_dialer *d, unsigned flags, nni_aio *aiop)
+{
+	int      rv = 0;
+
+	if (nni_atomic_flag_test_and_set(&d->d_started)) {
+		return (NNG_ESTATE);
+	}
+
+	if (aiop != NULL) {
+		nni_aio_start(aiop, NULL, NULL);
+	}
+
+	// Note that flags is currently unused, since the only flag is
+	// NONBLOCK, which is handled in callers.
+	NNI_ARG_UNUSED(flags);
+
+	nni_mtx_lock(&d->d_mtx);
+	d->d_user_aio = aiop;
+	dialer_connect_start(d);
+	nni_mtx_unlock(&d->d_mtx);
+
 	nng_log_info("NNG-DIAL", "Starting dialer for socket<%u>",
 	    nni_sock_id(d->d_sock));
 
diff --git a/src/core/dialer.h b/src/core/dialer.h
index bd987866..26cd5c0d 100644
--- a/src/core/dialer.h
+++ b/src/core/dialer.h
@@ -23,6 +23,7 @@ extern int      nni_dialer_create(nni_dialer **, nni_sock *, const char *);
 extern int  nni_dialer_create_url(nni_dialer **, nni_sock *, const nng_url *);
 extern void nni_dialer_close(nni_dialer *);
 extern int  nni_dialer_start(nni_dialer *, unsigned);
+extern int  nni_dialer_start_aio(nni_dialer *, unsigned, nni_aio *);
 extern nni_sock *nni_dialer_sock(nni_dialer *);
 
 extern int nni_dialer_setopt(
diff --git a/src/nng.c b/src/nng.c
index a425d599..9b155599 100644
--- a/src/nng.c
+++ b/src/nng.c
@@ -666,6 +666,30 @@ nng_dialer_start(nng_dialer did, int flags)
 	return (rv);
 }
 
+void
+nng_dialer_start_aio(nng_dialer did, int flags, nng_aio *aio)
+{
+	nni_dialer *d;
+	int       rv;
+
+	if (aio != NULL) {
+		nni_aio_reset(aio);
+	}
+	if ((flags & NNG_FLAG_NONBLOCK) == 0) {
+		nni_aio_finish_error(aio, NNG_EINVAL);
+		return;
+	}
+	if ((rv = nni_dialer_find(&d, did.id)) != 0) {
+		nni_aio_finish_error(aio, rv);
+		return;
+	}
+	if ((rv = nni_dialer_start_aio(d, flags, aio)) != 0) {
+		nni_aio_finish_error(aio, rv);
+		// fall-through
+	}
+	nni_dialer_rele(d);
+}
+
 int
 nng_dialer_id(nng_dialer d)
 {