| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #490 posix_epdesc use-after-free bug
fixes #489 Sanitizer based testing would help
fixes #492 Numerous memory leaks found with sanitizer
This introduces support for compiler-based sanitizers when using
clang or gcc (and not on Windows). See NNG_SANITIZER for possible
settings such as "thread" or "address".
Furthermore, we have fixed the issues we found with both the
thread and address sanitizers. We believe that the thread issues
pointed to a low frequency use-after-free responsible for rare
crashes in some of the tests.
The tests generally have their timeouts doubled when running under
a sanitizer, to account for the extra long times that the sanitizer
can cause these to take.
While here, we also changed the compat_ws test to avoid a particularly
painful and time consuming DNS lookup, and we made the nngcat_unlimited
test a bit more robust by waiting before sending traffic.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #464 Support NN_WS_MSG_TYPE option (compat)
fixes #415 websocket does not honor recv maxsize
This fixes a significant (and security) issue in websocket, where the
code does not honor a maximum receive size. We've exposed new API
(internal) to set the limit on the frame size, and we've changed the
default to *unlimited* for that internal API. (But the default for SP
sockets, which are the only consumers at present, is still 1MB just like
all other SP transports.)
|
| |
|
|
|
|
|
| |
While here, there is a little more endpoint termination hardening.
Running this code base seems to not incur any faults beyond the very
rare TCP port conflict that can occur from our random port selection
in the test suite.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #397 Need to cast zoneid
fixes #395 sun is predefined on illumos/Solaris
fixes #394 alloca needs to #include <alloca.h>
fixes #399 Cannot use SVR4.2 specific msghdr
fixes #402 getpeerucred needs a NULL initialized ucred
fixes #403 syntax error in posix_tcp - attempt to return void
fixes #407 illumos getegid wrong
fixes #406 nni_idhash_count is dead code
fixes #404 idhash typedef redeclared
fixes #405 warning: newline not last character in file
This is basically a slew of related bug fixes required to make this
work on illumos. Note that the fixes are not "complete", because
more work is required to support port events given that epoll is busted
on illumos.
We also fixed a bunch of things that aren't actually "bugs" per se, but
really just warnings. Silencing them makes things better for everyone.
Apparently not all compilers are equally happy with redundant (but
otherwise identical) typedefs; we use structs in some places instead of
shorter type names to silence these complaints.
Note that IPC permissions (the mode bits on the socket vnode) are not
validated on SunOS systems. This change includes documentation to reflect
that.
|
| | |
|
| |
|
|
| |
fixes #106 TCP keepalive tuning
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| |
|
|
|
| |
Applications which may not even be using TLS should not have to
know about the TLS configuration structure.
|
| |
|
|
|
|
|
|
| |
fixes #325 synchronous aio completion crash
fixes #327 move nni_clock() operations to outside the nni_aio_lk.
This work was done for the context tree, and is necessary to properly
enable that branch.
|
| |
|
|
|
| |
This also gives a performance benefit to WebSocket, by making
the completion logic run synchronously.
|
| |
|
|
|
|
| |
This includes platform portability stuff like mutexes and threads,
etc. While here we fixed a problem with nng_opts_parse and we
addressed the type of the platform API.
|
| |
|
|
|
|
|
| |
on a python wrapper (cffi).
Mostly this is fixing inconsistencies in our public API and the actual
implementation.
|
| | |
|
| |
|
|
|
|
|
|
| |
This causes TCP, TLS, and ZT endpoints to resolve any
wildcards, and even IP addresses, when reporting the listen
URL. The dialer URL is reported unresolved. Test cases
for this are added as well, and nngcat actually reports this
if --verbose is supplied.
|
| | |
|
| |
|
|
|
|
|
| |
While here we also fixed a bug in the --file handling that we noticed
while writing the TLS handling.
We also fixed a warning in the core (msgqueue) for set but unused variables.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is intended to provide compatibility with, and has been tested
against, legacy nanocat. There are a few differences though.
At this time support for the alias names (where argv[0] is set to
something like nngreq or somesuch) is missing.
By default this library operations without NNG_FLAG_NONBLOCK on
dial and listen, so that failures here are immediately diagnosable.
(This behavior can be changed with the --async flag.)
By default --pair means PAIRv1, but you can specify --pair0
or --pair1 explicitly. (There is also a --compat mode, and in
that mode --pair means PAIRv0. The --compat mode also turns on
NNG_FLAG_NONBLOCK by default.)
The "quoted" mode also quotes tabs. (Legacy nanocat did not.)
It is possible to connect to *multiple* peers by using the --dial
or --listen (or similar) options multiple times.
Shorthands can be used for long options that are not ambiguous. For
example, --surv can be used to mean surveyor, but --re is invalid because
it can mean req, rep, or respondent.
We assume you have a reasonable standard C environment. This won't work
in embedded environments without support for FILE *.
TLS options are missing but to be added soon.
A man page is still to be written.
|
| |
|
|
|
|
|
|
|
|
|
| |
We have implemented this alternative to getopt() so that we can
create nngcat. The reason we did not just use getopt() is that
getopt() does not understand long options (which nanocat uses, and
we want to preserve for compatibility) and getopt() is not available
on Windows (and possibly other non-POSIX platforms.)
This function handles long and short options, but does not have support
for option clustering. It also is threadsafe & reentrant, unlike getopt.
|
| |
|
|
|
|
| |
We move the HTTP definitions out of the core nng.h and into
a supplemental header. Most of this change was trivial updates
to all of the HTTP related manual pages.
|
| |
|
|
|
|
|
|
|
| |
These are incremental updates... we avoid using install() in the
subdirectories, so that we can adapt properly to them in the
single parent directory.
We have started some of the work to improve support for CPack. This
is still not yet done, but work in progress.
|
| |
|
|
|
|
|
| |
This introduces portable primitives for time, random numbers,
synchronization primitives, and threading. These are somewhat
primitive (least common denominiators), but they can help with writing
portable applications, especially our own demo apps.
|
| |
|
|
|
|
|
| |
We enabled verbose compiler warnings, and found a lot of issues.
Some of these were even real bugs. As a bonus, we actually save
some initialization steps in the compat layer, and avoid passing
some variables we don't need.
|
| |
|
|
|
|
|
|
| |
This addresses the use of the pipe special field, and eliminates it.
The message APIs (recvmsg, sendmsg) need to be updated as well still,
but I want to handle that as part of a separate issue.
While here we fixed various compiler warnings, etc.
|
| |
|
|
| |
While here, we cleaned up a few other unused variables in the HTTP code.
|
| | |
|
| | |
|
| |
|
|
|
| |
This member is the value passed in actual HTTP protocol, so it
is useful with the function nng_http_req_set_uri().
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This eliminates a bunch of redundant URL parsing, using the common
URL logic we already have in place.
While here I fixed a problem with the TLS and WSS test suites that
was failing on older Ubuntu -- apparently older versions of mbedTLS
were unhappy if selecting OPTIONAL verification without a validate
certificate chain.
|
| |
|
|
|
|
|
|
|
|
|
| |
This changes the backend (internal) HTTP API to provide a much more
sensible handler scheme, where the handlers are opaque objects and we
can allocate a handler for different types of tasks.
We've also added support serving up directories of static content, and
added code to validate that the directory serving is working as intended.
This is a key enabling step towards the public API.
|
| | |
|
| |
|
|
|
| |
fixes #210 Want NNG_OPT_TLS_* options for TLS transport
fixes #212 Eliminate a_endpt member of aio
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is useful to have support for validating that a peer *was*
verified, especially in the presence of optional validation.
We have added a property that does this, NNG_OPT_TLS_VERIFIED.
Further, all the old NNG_OPT_WSS_TLS_* property names have also been
renamed to generic NNG_OPT_TLS property names, which have been
moved to nng.h to facilitate reuse and sharing, with the comments
moved and corrected as well.
Finally, the man pages have been updated, with substantial
improvements to the nng_ws man page in particular.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for configuration of TLS websockets using the files
for keys, certificates, and CRLs. Significant changes to the websocket,
TLS, and HTTP layers were made here. We now expect TLS configuration to
be tied to the HTTP layer, and the HTTP code creates default configuration
objects based on the URL supplied. (HTTP dialers and listeners are now
created with a URL rather than a sockaddr, giving them access to the scheme
as well.)
We fixed several bugs affecting TLS validation, and added a test suite
that confirms that validation works as it should. We also fixed an orphaned
socket during HTTP negotiation, responsible for an occasional assertion
error if the http handshake does not complete successfully. Finally several
use-after-free races were closed.
TLS layer changes include reporting of handshake failures using newly
created "standard" error codes for peer authentication and cryptographic
failures.
The use of the '*' wild card in URLs at bind time is no longer supported
for websocket at least.
Documentation updates for all this are in place as well.
|
| |
|
|
|
|
|
|
|
| |
This refactor of the file API provides a simpler and easier to use
interface for our needs (and simpler to implement) in both the
ZeroTier transport and the HTTP/TLS file accesses. It also removes
some restrictions present on the old one, although it is still not
suitable for working with large files. (It will work, just be
very inefficient as the entire file must be loaded into memory.)
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This also fixes a use-after-free bug in the HTTP framework, where the
handler could be deleted why callbacks were still using it. (We now
reference count the handlers.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a problem where the websocket would only send one message,
then no others, due to not clearing the "frame" busy flag on completion
of the frame transmit.
We have also added a test that tries to send 10 messages back and
forth to make sure that we catch this kind of problem in the future.
Finally we've fixed some problems that were found when testing edge
cases around the protocol, which were responsible for invalid memory
accesses.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
First, httpbin.org was having some high latency (load) earlier today,
so we needed to bump the timeout up.
Next, this also uncovered a bug where our cancellation of http channels
was a bit dodgy. This is changed to be a bit more robust, separating the
"current" active http streams (for read or write) into separate tracking
variables variables. Also, now cancellation immediately calls the aio
finish for those -- there were assumptions elsewhere (expire timeouts)
that cancellation caused nni_aio_finish() to be called.
Finally there was a use after free bug in the websocket listener code
where the listener could be freed while still having outstanding streams
waiting to send the websocket reply.
|
| |
|
|
|
| |
Big thanks to @bertrand- for figuring this out, and a prototype of
the fix.
|
| | |
|
| | |
|
