| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
This is not quite complete, but it sets the stage for other
protocols (such as zmq or mqtt) to be added to the project.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This is based on testutil/acutest, but is cleaner and fixes some
short-comings. We will be adding more support for additional
common paradigms to better facilitate transport tests.
While here we added some more test cases, and fixed a possible
symbol collision in the the stats framework (due to Linux use
of a macro definition of "si_value" in a standard OS header).
Test coverage may regress slightly as we are no longer using
some of the legacy APIs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are a few major areas in this change.
* CMake options are now located in a common cmake/NNGOptions.cmake
file. This should make it easier for folks to figure out what
the options are, and how they are used.
* Tests are now scoped with their directory name, which should
avoid possible name collisions with test names.
* A number of tests have been either moved or incorporated into
the newer testutil/acutest framework. We are moving away from
my old c-convey framework to something easier to debug.
* We use CMake directories a bit more extensively leading to a much
cleaner CMake structure. It's not complete, but a big step in the
right direction, and a preview of future work.
* Tests are now run with verbose flags, so we get more test results
in the CI/CD logs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a sweeping cleanup of the transport logic around options,
and also harmonizes the names used when setting or getting options.
Additionally, legacy methods are now moved into a separate file and
can be elided via CMake or a preprocessor define.
Fundamentally, the ability to set to transport options via the socket
is deprecated; there are numerous problems with this and my earlier
approaches to deal with this have been somewhat misguided. Further
these approaches will not work with future protocol work that is
planned (were some options need to be negotiated with peers at the
time of connection establishment.)
Documentation has been updated to reflect this. The test suites still
make rather broad use of the older APIs, and will be converted later.
|
| |
|
|
|
|
| |
This moves some of the fragile tests to a new test suite that
is a bit more careful with IPv4 vs. IPv6. Hopefully it will be
a bit more resilient as a result.
|
| | |
|
| |
|
|
| |
fixes #1277 FreeBSD errors due to bad v4 vs. v6 assumptions
|
| |
|
|
|
|
|
|
| |
This introduces support for an external wolfSSL plugin, and generally
creates the framework for pluggable TLS implementations.
The wolfSSL engine is provided via an external module (git submodule),
available either under a GPLv3 license or a commercial license.
|
| |
|
|
|
| |
This only does it for rep, but it also has changes that should increase
the overall test coverage for the REP protocol
|
| |
|
|
|
|
|
|
|
| |
This is a major change, and includes changes to use a polymorphic
stream API for all transports. There have been related bugs fixed
along the way. Additionally the man pages have changed.
The old non-polymorphic APIs are removed now. This is a breaking
change, but the old APIs were never part of any released public API.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This changes much of the internal API for TCP option handling, and
includes hooks for some of this in various consumers. Note that the
consumers still need to have additional work done to complete them,
which will be part of providing public "raw" TLS and WebSocket APIs.
We would also like to finish addressing the call sites of
nni_tcp_listener_start() that assume the sockaddr is modified --
it would be superior to use the NNG_OPT_LOCADDR option. Thaat will be
addressed in a follow up PR.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This change makes embedding nng + nggpp (or other projects depending on
nng) in cmake easier. The header files are moved to a separate include
directory. This also makes installation of the headers easier, and
allows clearer identification of private vs public heade files.
Some additional cleanups were performed by @gedamore, but the main
credit for this change belongs with @gregorburger.
|
| |
|
|
| |
fixes #776 Configuration of mbedTLS should warn about license
|
| |
|
|
|
|
|
|
|
|
|
| |
This is a significant refactor of the library configuration.
We use the modern package configuration helper, with a template
script that also does the find_package dance for any of our
dependencies.
We also have restructured the code so that most protocols and
transports have their configuration isolated to their own CMakeLists
file, reducing the size of the global CMakeLists file.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This changes the signature of the aio cancellation routines
to take the argument for cancellation directly, so we do not
need to lookup the argument using the nni_aio_get_prov_data.
We should probably consider eliminating nni_aio_get_prov_data,
and co, and changing the prov_extra to reflect prov_data. Later.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
fixes #573 atomic flags could help
This introduces a new atomic flag, and reduces some of the global
locking. The lock refactoring work is not yet complete, but this is
a positive step forward, and should help with certain things.
While here we also fixed a compile warning due to incorrect types.
|
| |
|
|
|
|
|
|
|
|
| |
This separates the plumbing for endpoints into distinct
dialer and listeners. Some of the transports could benefit
from further separation, but we've done some rather larger
separation e.g. for the websocket transport.
IPC would be a good one to update later, when we start looking
at exposing a more natural underlying API.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #538 setopt should have an explicit chkopt routine
fixes #537 Internal TCP API needs better name separation
fixes #524 Option types should be "typed"
This is a rework of the option management code, to make it both clearer
and to prepare for further work to break up endpoints. This reduces
a certain amount of dead or redundant code, and actually saves cycles
when setting options, as some loops were not terminated that should have
been.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #490 posix_epdesc use-after-free bug
fixes #489 Sanitizer based testing would help
fixes #492 Numerous memory leaks found with sanitizer
This introduces support for compiler-based sanitizers when using
clang or gcc (and not on Windows). See NNG_SANITIZER for possible
settings such as "thread" or "address".
Furthermore, we have fixed the issues we found with both the
thread and address sanitizers. We believe that the thread issues
pointed to a low frequency use-after-free responsible for rare
crashes in some of the tests.
The tests generally have their timeouts doubled when running under
a sanitizer, to account for the extra long times that the sanitizer
can cause these to take.
While here, we also changed the compat_ws test to avoid a particularly
painful and time consuming DNS lookup, and we made the nngcat_unlimited
test a bit more robust by waiting before sending traffic.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #464 Support NN_WS_MSG_TYPE option (compat)
fixes #415 websocket does not honor recv maxsize
This fixes a significant (and security) issue in websocket, where the
code does not honor a maximum receive size. We've exposed new API
(internal) to set the limit on the frame size, and we've changed the
default to *unlimited* for that internal API. (But the default for SP
sockets, which are the only consumers at present, is still 1MB just like
all other SP transports.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #429 async websocket reap leads to crash
This tightens up the code for shutdown, ensuring that transport
callbacks are completely stopped before advancing to the next step
of teardown of transport pipes or endpoints.
It also fixes a problem where task_wait would sometimes get "stuck"
as tasks transitioned between asynch and synchronous completions.
Finally, it saves a few cycles by only calling a cancellation callback
once during cancellation of an aio.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #302 nng_dialer/listener/pipe_getopt_sockaddr desired
This adds plumbing to pass and check the type of options
all the way through.
NNG_ZT_OPT_ORBIT is type UINT64, but you can use the untyped form to
pass two of them if needed.
No typed access for retrieving strings yet. I think this should allocate
a pointer and copy that out, but that's for later.
|
| |
|
|
|
|
|
| |
on a python wrapper (cffi).
Mostly this is fixing inconsistencies in our public API and the actual
implementation.
|
| |
|
|
|
|
|
|
|
|
|
| |
fixes #275 nng_pipe_getopt_ptr() missing?
fixes #285 nng_setopt_ptr MIS
fixes #297 nng_listener/dialer_close does not validate mode
This change adds some missing APIs, and changes others.
In particular, certain options are now of type bool, with size
of just one. This is a *breaking* change for code that uses those
options -- NNG_OPT_RAW, NNG_OPT_PAIR1_POLY, NNG_OPT_TLS_VERIFIED.
|
| | |
|
| |
|
|
|
|
| |
We move the HTTP definitions out of the core nng.h and into
a supplemental header. Most of this change was trivial updates
to all of the HTTP related manual pages.
|
| |
|
|
|
|
|
|
|
| |
These are incremental updates... we avoid using install() in the
subdirectories, so that we can adapt properly to them in the
single parent directory.
We have started some of the work to improve support for CPack. This
is still not yet done, but work in progress.
|
| |
|
|
|
|
|
| |
We enabled verbose compiler warnings, and found a lot of issues.
Some of these were even real bugs. As a bonus, we actually save
some initialization steps in the compat layer, and avoid passing
some variables we don't need.
|
| |
|
|
|
|
|
|
| |
This addresses the use of the pipe special field, and eliminates it.
The message APIs (recvmsg, sendmsg) need to be updated as well still,
but I want to handle that as part of a separate issue.
While here we fixed various compiler warnings, etc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
| |
|
|
|
|
|
|
|
|
| |
This eliminates a bunch of redundant URL parsing, using the common
URL logic we already have in place.
While here I fixed a problem with the TLS and WSS test suites that
was failing on older Ubuntu -- apparently older versions of mbedTLS
were unhappy if selecting OPTIONAL verification without a validate
certificate chain.
|
| | |
|
| |
|
|
|
| |
fixes #210 Want NNG_OPT_TLS_* options for TLS transport
fixes #212 Eliminate a_endpt member of aio
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is useful to have support for validating that a peer *was*
verified, especially in the presence of optional validation.
We have added a property that does this, NNG_OPT_TLS_VERIFIED.
Further, all the old NNG_OPT_WSS_TLS_* property names have also been
renamed to generic NNG_OPT_TLS property names, which have been
moved to nng.h to facilitate reuse and sharing, with the comments
moved and corrected as well.
Finally, the man pages have been updated, with substantial
improvements to the nng_ws man page in particular.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for configuration of TLS websockets using the files
for keys, certificates, and CRLs. Significant changes to the websocket,
TLS, and HTTP layers were made here. We now expect TLS configuration to
be tied to the HTTP layer, and the HTTP code creates default configuration
objects based on the URL supplied. (HTTP dialers and listeners are now
created with a URL rather than a sockaddr, giving them access to the scheme
as well.)
We fixed several bugs affecting TLS validation, and added a test suite
that confirms that validation works as it should. We also fixed an orphaned
socket during HTTP negotiation, responsible for an occasional assertion
error if the http handshake does not complete successfully. Finally several
use-after-free races were closed.
TLS layer changes include reporting of handshake failures using newly
created "standard" error codes for peer authentication and cryptographic
failures.
The use of the '*' wild card in URLs at bind time is no longer supported
for websocket at least.
Documentation updates for all this are in place as well.
|
| |
|
|
|
|
|
| |
I'm pretty sure I need to go back and review the handling of
send messages for websocket too. We still have a receive leak
in websocket and leaks caused by the new URL parsing code which
needs to be refactored.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces the wss:// scheme, which is available and works like
the ws:// scheme if TLS is enabled in the library.
The library modularization is refactored somewhat, to make it easier
to use. There is now a single NNG_ENABLE_TLS that enables TLS support
under the hood.
This also adds a new option for the TLS transport, NNG_OPT_TLS_CONFIG
(and a similar one for WSS, NNG_OPT_TLS_WSS_CONFIG) that offer access
to the underlying TLS configuration object, which now has a public API
to go with it as well.
Note that it is also possible to use pure HTTPS using the *private*
API, which will be exposed in a public form soon.
|
| | |
|
