| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| | |
|
| |
|
|
|
|
|
|
| |
fixes #325 synchronous aio completion crash
fixes #327 move nni_clock() operations to outside the nni_aio_lk.
This work was done for the context tree, and is necessary to properly
enable that branch.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #22 Consider using synchronous completions sometimes
Transport improvements for IPC, TCP, and TLS.
This change does three things.
First it permits multiple outstanding receives or sends on the transport.
This change is being made to accomodate some other changes in the protocols
where it might be advantageous to post send or receives directly against
the transport pipe without going through another level of indirection.
Second, it changes the normal completions to be performed synchronously.
This translates into a rather major performance improvement, reducing
latency by some 27%, and thereby improving performance altogether. (This
elminates two extra context switches per transaction!)
FInally, we can save some extra checks and conditions because we know
that completions cannot happen if we don't have a pending operation
(we no longer complete out of sequence), and we only call the dosend
operation when we have something to send. This can eliminate some
pipeline stalls.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #302 nng_dialer/listener/pipe_getopt_sockaddr desired
This adds plumbing to pass and check the type of options
all the way through.
NNG_ZT_OPT_ORBIT is type UINT64, but you can use the untyped form to
pass two of them if needed.
No typed access for retrieving strings yet. I think this should allocate
a pointer and copy that out, but that's for later.
|
| |
|
|
| |
fixes #290 sockaddr improvements
|
| | |
|
| |
|
|
|
|
|
|
|
| |
These are incremental updates... we avoid using install() in the
subdirectories, so that we can adapt properly to them in the
single parent directory.
We have started some of the work to improve support for CPack. This
is still not yet done, but work in progress.
|
| |
|
|
|
|
|
|
| |
This addresses the use of the pipe special field, and eliminates it.
The message APIs (recvmsg, sendmsg) need to be updated as well still,
but I want to handle that as part of a separate issue.
While here we fixed various compiler warnings, etc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This eliminates a bunch of redundant URL parsing, using the common
URL logic we already have in place.
While here I fixed a problem with the TLS and WSS test suites that
was failing on older Ubuntu -- apparently older versions of mbedTLS
were unhappy if selecting OPTIONAL verification without a validate
certificate chain.
|
| |
|
|
|
|
|
| |
I'm pretty sure I need to go back and review the handling of
send messages for websocket too. We still have a receive leak
in websocket and leaks caused by the new URL parsing code which
needs to be refactored.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a new transport (compatible with the TLS
transport from mangos), using TLS v1.2.
To use the new transport, you must have the mbed TLS library
available on your system (Xenial libmbedtls-dev). You can use
version 2.x or newer -- 1.3.x and PolarSSL versions are not
supported.
You enable the TLS transport with -DNNG_TRANSPORT_TLS=ON in the CMake
configuration.
You must configure the server certificate by default, and this can only
be done using nng options. See the nng_tls man page for details.
This work is experimental, and was made possible by Capitar IT Group BV,
and Staysail Systems, Inc.
|
| |
|
|
| |
fixes #155 POSIX TCP & IPC could avoid a lot of context switches
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This makes all the protocols and transports optional. All
of them except ZeroTier are enabled by default, but you can
now disable them (remove from the build) with cmake options.
The test suite is modified so that tests still run as much
as they can, but skip over things caused by missing functionality
from the library (due to configuration).
Further, the constant definitions and prototypes for functions
that are specific to transports or protocols are moved into
appropriate headers, which should be included directly by
applications wishing to use these.
We have also added and improved documentation -- all of the
transports are documented, and several more man pages for
protocols have been added. (Req/Rep and Surveyor are still
missing.)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This makes the APIs use string keys, and largely eliminates the use of
integer option IDs altogether. The underlying registration for options
is also now a bit richer, letting protcols and transports declare the
actual options they use, rather than calling down into each entry point
carte blanche and relying on ENOTSUP.
This code may not be as fast as the integers was, but it is more intuitive,
easier to extend, and is not on any hot code paths. (If you're diddling
options on a hot code path you're doing something wrong.)
|
| |
|
|
|
|
|
|
|
|
| |
This fleshes most of the pipe API out, making it available to end user
code. It also adds a URL option that is independent of the address
options (which would be sockaddrs.)
Also, we are now setting the pipe for req/rep. The other protocols need
to have the same logic added to set the receive pipe on the message. (Pair
is already done.)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
We allocate AIO structures dynamically, so that we can use them
abstractly in more places without inlining them. This will be used
for the ZeroTier transport to allow us to create operations consisting
of just the AIO. Furthermore, we provide accessors for some of the
aio members, in the hopes that we will be able to wrap these for
"safe" version of the AIO capability to export to applications, and
to protocol and transport implementors.
While here we cleaned up the protocol details to use consistently
shorter names (no nni_ prefix for static symbols needed), and we
also fixed a bug in the surveyor code.
|
| |
|
|
|
|
|
|
| |
We have our versions of strdup, strlcat, and strlcpy.
This means we can avoid using snprintf() in many cases
(saving cycles), and we can get safer checks. We use
the platform supplied versions of these if they exist
(wrapping with nni_xxx versions.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This eliminates all the old #define's or enum values, making all
option IDs now totally dynamic, and providing well-known string
values for well-behaved applications.
We have added tests of some of these options, including lookups, and
so forth. We have also fixed a few problems; including at least
one crasher bug when the timeouts on reconnect were zero.
Protocol specific options are now handled in the protocol. We will
be moving the initialization for a few of those well known entities
to the protocol startup code, following the PAIRv1 pattern, later.
Applications must therefore not depend on the value of the integer IDs,
at least until the application has opened a socket of the appropriate
type.
|
| |
|
|
|
|
|
| |
This eliminates tests for code that we cannot reach, because the
upper layer endpoint code already ensures that we don't get called
if we are closing, that the mode is correct, and that only one
outstanding endpoint operation is in progress on any given endpoint.
|
| |
|
|
|
|
|
|
|
| |
This moves the DNS related functionality into common code, and also
removes all the URL parsing stuff out of the platform specific code
and into the transports. Now the transports just take sockaddr's on
initialization. (We may want to move this until later.)
We also add UDP resolution as another separate API.
|
| |
|
|
|
|
| |
This supports creating listeners and dialers, managing options
on them (though only a few options are supported at present),
starting them and closing them, all independently.
|
| |
|
|
|
|
| |
This also includes tests for some of the edge cases surrounding
pluggable transports, such as version mismatch, duplication registration,
and failure to initialize.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the underlying platform fails (FreeBSD is the only one I'm aware
of that does this!), we use a global lock or condition variable instead.
This means that our lock initializers never ever fail.
Probably we could eliminate most of this for Linux and Darwin, since
on those platforms, mutex and condvar initialization reasonably never
fails. Initial benchmarks show little difference either way -- so we
can revisit (optimize) later.
This removes a lot of otherwise untested code in error cases and so forth,
improving coverage and resilience in the face of allocation failures.
Platforms other than POSIX should follow a similar pattern if they need
this. (VxWorks, I'm thinking of you.) Most sane platforms won't have
an issue here, since normally these initializations do not need to allocate
memory. (Reportedly, even FreeBSD has plans to "fix" this in libthr2.)
While here, some bugs were fixed in initialization & teardown.
The fallback code is properly tested with dedicated test cases.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This passes valgrind 100% clean for both helgrind and deep leak
checks. This represents a complete rethink of how the AIOs work,
and much simpler synchronization; the provider API is a bit simpler
to boot, as a number of failure modes have been simply eliminated.
While here a few other minor bugs were squashed.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
We need to remember that protocol stops can run synchronously, and
therefore we need to wait for the aio to complete. Further, we need
to break apart shutting down aio activity from deallocation, as we need
to shut down *all* async activity before deallocating *anything*.
Noticed that we had a pipe race in the surveyor pattern too.
|
| |
|
|
|
|
| |
This resolves the orphaned pipedesc, which actually could have affected
Windows too. I think maybe we are race free. Lots more testing is
still required, but stress runs seem to be passing now.
|
| |
|
|
|
|
|
|
|
|
|
| |
We have seen leaks of pipes causing test failures (e.g. the Windows
IPC test) due to EADDRINUSE. This was caused by a case where we
failed to pass the pipe up because the AIO had already been canceled,
and we didn't realize that we had oprhaned the pipe. The fix is to
add a return value to nni_aio_finish, and verify that we did finish
properly, or if we did not then we must free the pipe ourself. (The
zero return from nni_aio_finish indicates that it accepts ownership
of resources passed via the aio.)
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It turns out that I had to fix a number of subtle asynchronous
handling bugs, but now TCP is fully asynchronous. We need to
change the high-level dial and listen interfaces to be async
as well.
Some of the transport APIs have changed here, and I've elected
to change what we expose to consumers as endpoints into seperate
dialers and listeners. Under the hood they are the same, but
it turns out that its helpful to know the intended use of the
endpoint at initialization time.
Scalability still occasionally hangs on Linux. Investigation
pending.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The connect & accept logic for IPC is now fully asynchronous.
This will serve as a straight-forward template for TCP. Note that
the upper logic still uses a thread to run this "synchronously", but
that will be able to be removed once the last transport (TCP) is made
fully async.
The unified ipcsock is also now separated, and we anticipate being
able to remove the posix_sock.c logic shortly. Separating out the
endpoint logic from the pipe logic helps makes things clearer, and
may faciliate a day where endpoints have multiple addresses (for
example with a connect() endpoint that uses a round-robin DNS list
and tries to run the entire list in parallel, stopping with the first
connection made.)
The platform header got a little cleanup while we were here.
|
| |
|
|
|
|
| |
This prevents a slow partner from blocking new connections from being
established on the server. Before this a single partner could cause
the server to block waiting to complete the negotiation.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
As with TCP, we're still using threads under the hood. But this
completes the send/recv logic conversion for POSIX to our AIO framework,
and hence represents a substantial milestone towards full asyncronous
operation.
We still need to do accept/connect operations asynchronously, then making.
Windows overlapped IO work properly. After that, poll/epoll/kqueue, etc.
|
