| Commit message (Collapse) | Author | Age |
|---|
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #179 DNS resolution should be done at connect time
fixes #586 Windows IO completion port work could be better
fixes #339 Windows iocp could use synchronous completions
fixes #280 TCP abstraction improvements
This is a rather monstrous set of changes, which refactors TCP, and
the underlying Windows I/O completion path logic, in order to obtain
a cleaner, simpler API, with support for asynchronous DNS lookups performed
on connect rather than initialization time, the ability to have multiple
connects or accepts pending, as well as fewer extraneous function calls.
The Windows code also benefits from greatly reduced context switching,
fewer lock operations performed, and a reduced number of system calls
on the hot code path. (We use automatic event resetting instead of manual.)
Some dead code was removed as well, and a few potential edge case leaks
on failure paths (in the websocket code) were plugged.
Note that all TCP based transports benefit from this work. The IPC code
on Windows still uses the legacy IOCP for now, as does the UDP code (used
for ZeroTier.) We will be converting those soon too.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #170 Make more use of reaper
This is a complete restructure/rethink of how child objects interact
with the socket. (This also backs out #576 as it turns out not to be
needed.) While 568 says reader/writer lock, for now we have settled
for a single writer lock. Its likely that this is sufficient.
Essentially we use the single socket lock to guard lists of the socket
children. We also use deferred deletion in the idhash to facilitate
teardown, which means endpoint closes are no longer synchronous.
We use the reaper to clean up objects when the reference count drops
to zero. We make a special exception for pipes, since they really
are not reference counted by their parents, and they are leaf objects
anyway.
We believe this addresses the main outstanding race conditions in
a much more correct and holistic way.
Note that endpoint shutdown is a little tricky, as it makes use of
atomic flags to guard against double entry, and against recursive
lock entry. This is something that would be nice to make a bit more
obvious, but what we have is safe, and the complexity is at least
confined to one place.
|
| | |
|
| |
|
|
|
|
| |
This uses id lookups for pipe listener and dialer during pipe getopt,
while still retaining the pointer fields for use during tear down.
More changes coming.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #573 atomic flags could help
This introduces a new atomic flag, and reduces some of the global
locking. The lock refactoring work is not yet complete, but this is
a positive step forward, and should help with certain things.
While here we also fixed a compile warning due to incorrect types.
|
| |
|
|
|
|
| |
fixes #565 Option getting should validate sizes more aggressively
fixes #563 Reconnect timeouts should be settable on dialers
fixes #562 pipe test is fragile
|
| |
|
| |
a missing return causes ipc and inproc URLs to be parsed as usual. Results in a memory leak.
|
| |
|
|
|
|
|
|
|
|
| |
This separates the plumbing for endpoints into distinct
dialer and listeners. Some of the transports could benefit
from further separation, but we've done some rather larger
separation e.g. for the websocket transport.
IPC would be a good one to update later, when we start looking
at exposing a more natural underlying API.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #538 setopt should have an explicit chkopt routine
fixes #537 Internal TCP API needs better name separation
fixes #524 Option types should be "typed"
This is a rework of the option management code, to make it both clearer
and to prepare for further work to break up endpoints. This reduces
a certain amount of dead or redundant code, and actually saves cycles
when setting options, as some loops were not terminated that should have
been.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This changes nni_aio_begin so that it immediately terminates when
it encounters aio->a_closed, much like it does for aio->a_stop.
The semantic for nni_aio_close() is supposed to be like nni_aio_stop(),
but without blocking.
I suspect that this might be responsible for use-after-free bugs that
seem to have been rearing their head lately.
|
| |
|
|
|
|
| |
This should work on both Windows and the most common POSIX
variants. We will create at least two threads for running
completions, but there are numerous other threads in the code.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Essentially, if we're destroying an aio, and we are doing so from the
thread that is running the callback, then we should defer the destruction
of the task until it returns.
Note that calling nni_aio_wait() or anything else that calls it from
the callback is still verboten and will result in a single party
deadlock.
|
| |
|
|
|
|
| |
This special cases the URL parser for inproc and IPC urls,
changing so that they no longer parse the thing after the ://
as anything special. This allows IPC URLs to be relative.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #464 Support NN_WS_MSG_TYPE option (compat)
fixes #415 websocket does not honor recv maxsize
This fixes a significant (and security) issue in websocket, where the
code does not honor a maximum receive size. We've exposed new API
(internal) to set the limit on the frame size, and we've changed the
default to *unlimited* for that internal API. (But the default for SP
sockets, which are the only consumers at present, is still 1MB just like
all other SP transports.)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #468 TCP nodelay and keepalive should start usable
fixes #467 NN_RCVMAXSZ option does not work (compat)
fixes #465 Support NN_OPT_TCPNODELAY (compat)
This is a rather larger change set than I'd like, but when adding
support for legacy TCP keepalive, I found a number if issues using
the legacy TCP test (which we are introducing with this commit.)
This fixes the concerns that are relevant and addressible.
We have elected not to try to support to local address binding at this
time, and the IPv6 test case in the old code was wrong, so changes
relevant to that are commented out.
I've also updated the nng_compat manual page to reflect additional
caveats that folks should be aware of, including the previously
undocumented caveat around the NN_SNDBUF and NN_RCVBUF options.
|
| |
|
|
|
|
|
|
|
| |
As part of this code fix, we needed to add filtering support to the
msgq_tryput code path -- it turns out that code path was bypassing
the filterfn altogether.
Eventually we'll remove all this filtering stuff from the msgq code
and replace it with inline filtering directly in sub.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This changes nni_task_fini to always run synchronously, waiting
for the task to finish before cleaning up. Much simpler code.
Additionally, we've refactored the resolver code to avoid the
use of taskqs, which added complexity and inefficiency. The
approach of just allocating its own threads and a work queue
to process them turns out to be vastly simpler, and actually
reduces extra allocations and context switches.
wip
POSIX resolv threads.
(Taskqs are just overhead and complexity here.)
Windows resolver changes.
Task cleanup.
fix up windows mutex.
|
| |
|
|
|
|
|
|
|
| |
This changes the signature of nng_pipe_notify(), and the associated
events. The documentation is updated to reflect this.
We have also broken the lock up so that we don't hold the master
socket lock for some of these things, which may have beneficial
impact on performance.
|
| |
|
|
|
|
|
|
| |
This changes the array of flags, which was confusing, brittle, and
racy, into a much simpler reference (busy) count on the task structures.
This allows us to support certain kinds of "reentrant" dispatching,
where either a synchronous or asynchronous task can reschedule / dispatch
itself. The new code also helps reduce certain lock pressure, as a bonus.
|
| |
|
|
| |
We also fixed some warnings found by gcc8.
|
| |
|
|
|
|
|
| |
While here, there is a little more endpoint termination hardening.
Running this code base seems to not incur any faults beyond the very
rare TCP port conflict that can occur from our random port selection
in the test suite.
|
| |
|
|
|
|
| |
While here I also improved the taskq.h comments (and removed a
stale prototype for nni_task_cancel), and addressed leaks in
the reqstress and multistress test programs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #429 async websocket reap leads to crash
This tightens up the code for shutdown, ensuring that transport
callbacks are completely stopped before advancing to the next step
of teardown of transport pipes or endpoints.
It also fixes a problem where task_wait would sometimes get "stuck"
as tasks transitioned between asynch and synchronous completions.
Finally, it saves a few cycles by only calling a cancellation callback
once during cancellation of an aio.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #397 Need to cast zoneid
fixes #395 sun is predefined on illumos/Solaris
fixes #394 alloca needs to #include <alloca.h>
fixes #399 Cannot use SVR4.2 specific msghdr
fixes #402 getpeerucred needs a NULL initialized ucred
fixes #403 syntax error in posix_tcp - attempt to return void
fixes #407 illumos getegid wrong
fixes #406 nni_idhash_count is dead code
fixes #404 idhash typedef redeclared
fixes #405 warning: newline not last character in file
This is basically a slew of related bug fixes required to make this
work on illumos. Note that the fixes are not "complete", because
more work is required to support port events given that epoll is busted
on illumos.
We also fixed a bunch of things that aren't actually "bugs" per se, but
really just warnings. Silencing them makes things better for everyone.
Apparently not all compilers are equally happy with redundant (but
otherwise identical) typedefs; we use structs in some places instead of
shorter type names to silence these complaints.
Note that IPC permissions (the mode bits on the socket vnode) are not
validated on SunOS systems. This change includes documentation to reflect
that.
|
| |
|
|
|
| |
We offer uid, gid, process id, and even zone id where we have them.
Docs and tests are provided.
|
| |
|
|
|
|
|
|
| |
This adds a new pipe event notification API (callbacks called
on either pipe add or remove), including both tests and docs.
Also supporting APIs to get the socket or endpoint associated
with a pipe are included (tested and documented as well.)
|
| |
|
|
|
| |
We already closed the pipe before putting it on the reaplist,
so don't do it again.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
fixes #382 Permissions support for IPC on POSIX
This adds support for permission management on Windows and
POSIX systems. There are two different properties, and they
are very different.
Tests and documentation are included.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change converts the various integer types like nng_socket
in the public API to opaque structures that are passed by value.
Basically we just wrap the integer ID. This "hack" give us strong
type checks by the compiler (yay!), at the expense of not being able
to directly use these as numbers (so comparisions for example don't
work, and neither does initialization to zero using the normal
method.
Comparison of disassembly output shows that at least with the optimizer
enabled there is no difference in the compiler output between using
a structure or an integral value.
|
| |
|
|
| |
fixes #106 TCP keepalive tuning
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #360 core should nng_aio_begin before nng_aio_finish_error
fixes #361 nng_send_aio should check for NULL message
fixes #362 nni_msgq does not signal pollable on certain events
This adds support for contexts for both sides of the surveyor pattern.
Prior to this commit, the raw mode was completely broken, and there
were numerous other bugs found and fixed. This integration includes
*much* deeper validation of this pattern.
Some changes to the core and other patterns have been made, where it
was obvioius that we could make such improvements. (The obviousness
stemming from the fact that RESPONDENT in particular is very closely
derived from REP.)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This includes the test from legacy libnanomsg and a man page.
We have refactored the message queue notification system so
that it uses nni_pollable, leading we hope to a more consistent
system, and reducing the code size and complexity.
We also fixed the size of the NN_RCVFD and NN_SNDFD so that they
are a SOCKET on Windows systems, rather than an integer. This
addresses 64-bit compilation problems.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ultimately, this just removes the support for lingering altogether.
Based on prior experience, lingering has always been unreliable, and
was removed in legacy libnanomsg ages ago.
The problem is that operating system support for lingering is very
inconsistent at best, and for some transports the very concept is somewhat
meaningless.
Making things worse, we were never able to adequately capture an exit()
event from another thread -- so lingering was always a false promise.
Applications that need to be sure that messages are delivered should
either include an ack in their protocol, use req/rep (which has an ack),
or inject a suitable delay of their own.
For things going over local networks, an extra delay of 100 msec should
be sufficient *most of the time*.
|
| |
|
|
|
|
|
| |
This technically doesn't need the lock, as we're only trying to catch
a possible problem during development rather than in the field (and
this should never occur), but the false positive in valgrind obscures
other possible errors, so leave it in the lock.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This provides context support for REQ and REP sockets.
More discussion around this is in the issue itself.
Optionally we would like to extend this to the surveyor pattern.
Note that we specifically do not support pollable descriptors
for non-default contexts, and the results of using file descriptors
for polling (NNG_OPT_SENDFD and NNG_OPT_RECVFD) is undefined.
In the future, it might be nice to figure out how to factor in
optional use of a message queue for users who want more buffering,
but we think there is little need for this with cooked mode.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This makes the raw mode something that is immutable, determined
at socket construction. This is an enabling change for the
separate context support coming soon.
As a result, this is an API breaking change for users of the raw
mode option (NNG_OPT_RAW). There aren't many of them out there.
Cooked mode is entirely unaffected.
There are changes to tests and documentation included.
|
