| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
This changeset needs work. We are seeing errors described by
This reverts commit d7f7c896c0ede24249ef63b1e45b1878bf4bd473.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #208 pipe start should occur before connect / accept
fixes #616 Race condition closing between header & body
This refactors the transports to handle their own connection
handshaking before passing the pipe to the socket. This
changes and simplifies the setup. This also fixes a rather
challenging race condition described by #616.
|
| | |
|
| |
|
|
| |
On QNX, specifying a numeric servname while leaving ai_socktype unspecified would result in EAI_SERVICE.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #596 POSIX IPC should move away from pipedesc/epdesc
fixes #598 TLS and TCP listeners could support NNG_OPT_LOCADDR
fixes #594 Windows IPC should use "new style" win_io code.
fixes #597 macOS could support PEER PID
This large change set cleans up the IPC support on Windows and
POSIX. This has the beneficial impact of significantly reducing
the complexity of the code, reducing locking, increasing
concurrency (multiple dial and accepts can be outstanding now),
reducing context switches (we complete thins synchronously now).
While here we have added some missing option support, and fixed a
few more bugs that we found in the TCP code changes from last week.
|
| |
|
|
|
| |
This also arranges for server shutdown to be handled using
the reaper, leading to more elegant cleanup.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #179 DNS resolution should be done at connect time
fixes #586 Windows IO completion port work could be better
fixes #339 Windows iocp could use synchronous completions
fixes #280 TCP abstraction improvements
This is a rather monstrous set of changes, which refactors TCP, and
the underlying Windows I/O completion path logic, in order to obtain
a cleaner, simpler API, with support for asynchronous DNS lookups performed
on connect rather than initialization time, the ability to have multiple
connects or accepts pending, as well as fewer extraneous function calls.
The Windows code also benefits from greatly reduced context switching,
fewer lock operations performed, and a reduced number of system calls
on the hot code path. (We use automatic event resetting instead of manual.)
Some dead code was removed as well, and a few potential edge case leaks
on failure paths (in the websocket code) were plugged.
Note that all TCP based transports benefit from this work. The IPC code
on Windows still uses the legacy IOCP for now, as does the UDP code (used
for ZeroTier.) We will be converting those soon too.
|
| | |
|
| | |
|
| |
|
|
|
| |
This sets the kqueue events to autoclear, reducing CPU usage to normal
sane levels, and eliminating the hard spin.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #573 atomic flags could help
This introduces a new atomic flag, and reduces some of the global
locking. The lock refactoring work is not yet complete, but this is
a positive step forward, and should help with certain things.
While here we also fixed a compile warning due to incorrect types.
|
| |
|
|
|
|
| |
This should work on both Windows and the most common POSIX
variants. We will create at least two threads for running
completions, but there are numerous other threads in the code.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This enables the software to be built for Android, going back to
at least Android SDK r15 (IceCreamSandwich) and at least up to
SDK r27 (Oreo). Older versions of Android may work, but we have
no way to build them to test.
While here we have changed our CMake configuration to disable
building tools or tests when we detect a cross-compile situation.
Documentation for cross-compilation is updated as well.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #490 posix_epdesc use-after-free bug
fixes #489 Sanitizer based testing would help
fixes #492 Numerous memory leaks found with sanitizer
This introduces support for compiler-based sanitizers when using
clang or gcc (and not on Windows). See NNG_SANITIZER for possible
settings such as "thread" or "address".
Furthermore, we have fixed the issues we found with both the
thread and address sanitizers. We believe that the thread issues
pointed to a low frequency use-after-free responsible for rare
crashes in some of the tests.
The tests generally have their timeouts doubled when running under
a sanitizer, to account for the extra long times that the sanitizer
can cause these to take.
While here, we also changed the compat_ws test to avoid a particularly
painful and time consuming DNS lookup, and we made the nngcat_unlimited
test a bit more robust by waiting before sending traffic.
|
| |
|
|
|
|
|
| |
The fallback logic was unnecessarily complicated, and found to be
somewhat data-racy; on modern systems initializing these things
never fails, and on BSD systems that only occurs under extreme
memory shortage.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #468 TCP nodelay and keepalive should start usable
fixes #467 NN_RCVMAXSZ option does not work (compat)
fixes #465 Support NN_OPT_TCPNODELAY (compat)
This is a rather larger change set than I'd like, but when adding
support for legacy TCP keepalive, I found a number if issues using
the legacy TCP test (which we are introducing with this commit.)
This fixes the concerns that are relevant and addressible.
We have elected not to try to support to local address binding at this
time, and the IPv6 test case in the old code was wrong, so changes
relevant to that are commented out.
I've also updated the nng_compat manual page to reflect additional
caveats that folks should be aware of, including the previously
undocumented caveat around the NN_SNDBUF and NN_RCVBUF options.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This changes nni_task_fini to always run synchronously, waiting
for the task to finish before cleaning up. Much simpler code.
Additionally, we've refactored the resolver code to avoid the
use of taskqs, which added complexity and inefficiency. The
approach of just allocating its own threads and a work queue
to process them turns out to be vastly simpler, and actually
reduces extra allocations and context switches.
wip
POSIX resolv threads.
(Taskqs are just overhead and complexity here.)
Windows resolver changes.
Task cleanup.
fix up windows mutex.
|
| |
|
|
| |
fixes #438 Consider dropping AI_V4MAPPED
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| |
|
|
|
|
|
|
| |
This replaces the epoll support with proper illumos/SunOS port
events. The port event support is structured so that it actually
is superior to epoll and kqueue, because it avoids a single master
lock on the poller. In the future we will explore this for macOS
and Linux pollers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #397 Need to cast zoneid
fixes #395 sun is predefined on illumos/Solaris
fixes #394 alloca needs to #include <alloca.h>
fixes #399 Cannot use SVR4.2 specific msghdr
fixes #402 getpeerucred needs a NULL initialized ucred
fixes #403 syntax error in posix_tcp - attempt to return void
fixes #407 illumos getegid wrong
fixes #406 nni_idhash_count is dead code
fixes #404 idhash typedef redeclared
fixes #405 warning: newline not last character in file
This is basically a slew of related bug fixes required to make this
work on illumos. Note that the fixes are not "complete", because
more work is required to support port events given that epoll is busted
on illumos.
We also fixed a bunch of things that aren't actually "bugs" per se, but
really just warnings. Silencing them makes things better for everyone.
Apparently not all compilers are equally happy with redundant (but
otherwise identical) typedefs; we use structs in some places instead of
shorter type names to silence these complaints.
Note that IPC permissions (the mode bits on the socket vnode) are not
validated on SunOS systems. This change includes documentation to reflect
that.
|
| |
|
|
|
| |
We offer uid, gid, process id, and even zone id where we have them.
Docs and tests are provided.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #382 Permissions support for IPC on POSIX
This adds support for permission management on Windows and
POSIX systems. There are two different properties, and they
are very different.
Tests and documentation are included.
|
| |
|
|
| |
fixes #106 TCP keepalive tuning
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| |
|
|
|
|
|
| |
* 127.0.0.1.32 is treated as a hostname, returns EAI_NODATA on my fedora 27 box
* since this is not in POSIX, and deprecated from some libc resolvers
protect with an ifdef
|
| | |
|
| |
|
|
| |
Turns out that shutdown is sufficient for most needs.
|
| | |
|
| |
|
|
|
| |
This uses numeric identifiers and an idhash table to make sure that
the values we get back are still use.
|
| | |
|
| |
|
|
| |
fixes #33
|
| |
|
|
| |
fixes #290 sockaddr improvements
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This does a few things. First it closes a case where a dropped
message could prevent subsequent connection attempts from getting through.
Second, it changes the rate at which we retry, and the timeout, to be
a lot more aggressive when attempting to establish a connection. We
retry every 500 ms, for up to 2 minutes, before giving up. This gives
a lot more resilience in the face of message loss that is typical of
ZeroTier in some environments when first establishing communication.
Third, makes the values for the connection attempts *tunable*, so
that applications can adjust for different deployment scenarios.
Fourth, it includes the ability to get the UDP socket name. This was
needed during some debug, and may be useful for a real UDP transport
later, so we're keeping it.
Finally, we added documentation for the above items.
|
| | |
|
| |
|
|
| |
fixes #267 zerotier transport should lock ZT_HOME
|
| |
|
|
|
|
|
|
| |
This causes TCP, TLS, and ZT endpoints to resolve any
wildcards, and even IP addresses, when reporting the listen
URL. The dialer URL is reported unresolved. Test cases
for this are added as well, and nngcat actually reports this
if --verbose is supplied.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We changed the timers to use msec granularity, but we missed this
one. The result is that in certain code flows the IPC connection
times can look quite long -- with weird 10 sec stalls.
|
| |
|
|
|
|
| |
ConnectNamedPipe can return ERROR_PIPE_CONNECTED, and does not
enqueue a completion packet if it does. So we need to handle
that specially.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was possible for pollq arm to be called on a node that was removed
in some circumstances -- particularly and ep that was closed in the
callback.
While here, lets use normal booleans for closed state, and only call
the arm function (which is not free -- typicall it involves a mutex
and may even involve a system call) if we are going to arm some events.
We also initialize these things properly, and clean up a stale comment.
This work is done to faciliate the kqueue work by @liamstask.
|
