| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
| |
This special cases the URL parser for inproc and IPC urls,
changing so that they no longer parse the thing after the ://
as anything special. This allows IPC URLs to be relative.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #490 posix_epdesc use-after-free bug
fixes #489 Sanitizer based testing would help
fixes #492 Numerous memory leaks found with sanitizer
This introduces support for compiler-based sanitizers when using
clang or gcc (and not on Windows). See NNG_SANITIZER for possible
settings such as "thread" or "address".
Furthermore, we have fixed the issues we found with both the
thread and address sanitizers. We believe that the thread issues
pointed to a low frequency use-after-free responsible for rare
crashes in some of the tests.
The tests generally have their timeouts doubled when running under
a sanitizer, to account for the extra long times that the sanitizer
can cause these to take.
While here, we also changed the compat_ws test to avoid a particularly
painful and time consuming DNS lookup, and we made the nngcat_unlimited
test a bit more robust by waiting before sending traffic.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #464 Support NN_WS_MSG_TYPE option (compat)
fixes #415 websocket does not honor recv maxsize
This fixes a significant (and security) issue in websocket, where the
code does not honor a maximum receive size. We've exposed new API
(internal) to set the limit on the frame size, and we've changed the
default to *unlimited* for that internal API. (But the default for SP
sockets, which are the only consumers at present, is still 1MB just like
all other SP transports.)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #429 async websocket reap leads to crash
This tightens up the code for shutdown, ensuring that transport
callbacks are completely stopped before advancing to the next step
of teardown of transport pipes or endpoints.
It also fixes a problem where task_wait would sometimes get "stuck"
as tasks transitioned between asynch and synchronous completions.
Finally, it saves a few cycles by only calling a cancellation callback
once during cancellation of an aio.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| |
|
|
|
|
|
| |
fixes #249 nngcat needs test cases
fixes #416 transports do not permit unlimited message size with 0
fixes #417 nngcat truncates input files to 4k
fixes #348 nngcat should have switch to adjust maximum receive size
|
| |
|
|
|
| |
We offer uid, gid, process id, and even zone id where we have them.
Docs and tests are provided.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #382 Permissions support for IPC on POSIX
This adds support for permission management on Windows and
POSIX systems. There are two different properties, and they
are very different.
Tests and documentation are included.
|
| |
|
|
| |
fixes #106 TCP keepalive tuning
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ultimately, this just removes the support for lingering altogether.
Based on prior experience, lingering has always been unreliable, and
was removed in legacy libnanomsg ages ago.
The problem is that operating system support for lingering is very
inconsistent at best, and for some transports the very concept is somewhat
meaningless.
Making things worse, we were never able to adequately capture an exit()
event from another thread -- so lingering was always a false promise.
Applications that need to be sure that messages are delivered should
either include an ack in their protocol, use req/rep (which has an ack),
or inject a suitable delay of their own.
For things going over local networks, an extra delay of 100 msec should
be sufficient *most of the time*.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
fixes #325 synchronous aio completion crash
fixes #327 move nni_clock() operations to outside the nni_aio_lk.
This work was done for the context tree, and is necessary to properly
enable that branch.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #22 Consider using synchronous completions sometimes
Transport improvements for IPC, TCP, and TLS.
This change does three things.
First it permits multiple outstanding receives or sends on the transport.
This change is being made to accomodate some other changes in the protocols
where it might be advantageous to post send or receives directly against
the transport pipe without going through another level of indirection.
Second, it changes the normal completions to be performed synchronously.
This translates into a rather major performance improvement, reducing
latency by some 27%, and thereby improving performance altogether. (This
elminates two extra context switches per transaction!)
FInally, we can save some extra checks and conditions because we know
that completions cannot happen if we don't have a pending operation
(we no longer complete out of sequence), and we only call the dosend
operation when we have something to send. This can eliminate some
pipeline stalls.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #302 nng_dialer/listener/pipe_getopt_sockaddr desired
This adds plumbing to pass and check the type of options
all the way through.
NNG_ZT_OPT_ORBIT is type UINT64, but you can use the untyped form to
pass two of them if needed.
No typed access for retrieving strings yet. I think this should allocate
a pointer and copy that out, but that's for later.
|
| |
|
|
|
|
|
| |
on a python wrapper (cffi).
Mostly this is fixing inconsistencies in our public API and the actual
implementation.
|
| |
|
|
|
|
|
|
|
|
|
| |
fixes #275 nng_pipe_getopt_ptr() missing?
fixes #285 nng_setopt_ptr MIS
fixes #297 nng_listener/dialer_close does not validate mode
This change adds some missing APIs, and changes others.
In particular, certain options are now of type bool, with size
of just one. This is a *breaking* change for code that uses those
options -- NNG_OPT_RAW, NNG_OPT_PAIR1_POLY, NNG_OPT_TLS_VERIFIED.
|
| |
|
|
| |
fixes #290 sockaddr improvements
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This does a few things. First it closes a case where a dropped
message could prevent subsequent connection attempts from getting through.
Second, it changes the rate at which we retry, and the timeout, to be
a lot more aggressive when attempting to establish a connection. We
retry every 500 ms, for up to 2 minutes, before giving up. This gives
a lot more resilience in the face of message loss that is typical of
ZeroTier in some environments when first establishing communication.
Third, makes the values for the connection attempts *tunable*, so
that applications can adjust for different deployment scenarios.
Fourth, it includes the ability to get the UDP socket name. This was
needed during some debug, and may be useful for a real UDP transport
later, so we're keeping it.
Finally, we added documentation for the above items.
|
| |
|
|
| |
fixes #267 zerotier transport should lock ZT_HOME
|
| |
|
|
|
|
|
| |
The dev branch changed, and the timestamps are signed. (They
are also passed around using volatiles -- probably incorrectly
so, but that is not important for us, the compiler understands
that volatile is meaningless when passing values by value.)
|
| |
|
|
|
|
|
|
| |
The primary ZeroTier RFC is in the "rfcs" repository for nanomsg,
and we should remove this one because it is stale and misleading.
Also, the URI format is incorrect in the header so fix that, and
point to the real ZeroTier RFC.
|
| |
|
|
|
|
|
|
| |
This causes TCP, TLS, and ZT endpoints to resolve any
wildcards, and even IP addresses, when reporting the listen
URL. The dialer URL is reported unresolved. Test cases
for this are added as well, and nngcat actually reports this
if --verbose is supplied.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We move the HTTP definitions out of the core nng.h and into
a supplemental header. Most of this change was trivial updates
to all of the HTTP related manual pages.
|
| |
|
|
|
|
|
|
|
| |
These are incremental updates... we avoid using install() in the
subdirectories, so that we can adapt properly to them in the
single parent directory.
We have started some of the work to improve support for CPack. This
is still not yet done, but work in progress.
|
| |
|
|
|
|
|
| |
We enabled verbose compiler warnings, and found a lot of issues.
Some of these were even real bugs. As a bonus, we actually save
some initialization steps in the compat layer, and avoid passing
some variables we don't need.
|
| |
|
|
|
|
|
|
| |
This addresses the use of the pipe special field, and eliminates it.
The message APIs (recvmsg, sendmsg) need to be updated as well still,
but I want to handle that as part of a separate issue.
While here we fixed various compiler warnings, etc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
| | |
|
| |
|
|
|
|
| |
We are adopting a more standard URL format for zerotier, and making
more use of the URL parsing common layer. While here we updated
the docs to reflect correctly the URI syntax we are using everywhere.
|
| |
|
|
|
|
|
|
|
|
| |
This eliminates a bunch of redundant URL parsing, using the common
URL logic we already have in place.
While here I fixed a problem with the TLS and WSS test suites that
was failing on older Ubuntu -- apparently older versions of mbedTLS
were unhappy if selecting OPTIONAL verification without a validate
certificate chain.
|
| | |
|
| | |
|
| |
|
|
|
| |
fixes #210 Want NNG_OPT_TLS_* options for TLS transport
fixes #212 Eliminate a_endpt member of aio
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is useful to have support for validating that a peer *was*
verified, especially in the presence of optional validation.
We have added a property that does this, NNG_OPT_TLS_VERIFIED.
Further, all the old NNG_OPT_WSS_TLS_* property names have also been
renamed to generic NNG_OPT_TLS property names, which have been
moved to nng.h to facilitate reuse and sharing, with the comments
moved and corrected as well.
Finally, the man pages have been updated, with substantial
improvements to the nng_ws man page in particular.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for configuration of TLS websockets using the files
for keys, certificates, and CRLs. Significant changes to the websocket,
TLS, and HTTP layers were made here. We now expect TLS configuration to
be tied to the HTTP layer, and the HTTP code creates default configuration
objects based on the URL supplied. (HTTP dialers and listeners are now
created with a URL rather than a sockaddr, giving them access to the scheme
as well.)
We fixed several bugs affecting TLS validation, and added a test suite
that confirms that validation works as it should. We also fixed an orphaned
socket during HTTP negotiation, responsible for an occasional assertion
error if the http handshake does not complete successfully. Finally several
use-after-free races were closed.
TLS layer changes include reporting of handshake failures using newly
created "standard" error codes for peer authentication and cryptographic
failures.
The use of the '*' wild card in URLs at bind time is no longer supported
for websocket at least.
Documentation updates for all this are in place as well.
|
| |
|
|
|
|
|
|
|
| |
This refactor of the file API provides a simpler and easier to use
interface for our needs (and simpler to implement) in both the
ZeroTier transport and the HTTP/TLS file accesses. It also removes
some restrictions present on the old one, although it is still not
suitable for working with large files. (It will work, just be
very inefficient as the entire file must be loaded into memory.)
|
| | |
|
| |
|
|
|
|
| |
This also fixes a use-after-free bug in the HTTP framework, where the
handler could be deleted why callbacks were still using it. (We now
reference count the handlers.)
|
| | |
|
| |
|
|
|
|
|
| |
I'm pretty sure I need to go back and review the handling of
send messages for websocket too. We still have a receive leak
in websocket and leaks caused by the new URL parsing code which
needs to be refactored.
|
