| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
| |
We have a lot of "private" wrappers around public functions, which
doesn't really help at all, and just add needless extra stack frames
and extra cruft in the linker tables. We should eliminate the
trivially thin wrappers where possible, and this is a start.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This represents a major change in the HTTP code base, consisting
of a complete revamp of the HTTP API. The changes here are too
numerous to mention, but the end result should be a vastly
simpler API for both server and client applications.
Many needless allocations were removed by providing fixed buffers
for various parameters and headers when possible.
A few bugs were fixed. Most especially we have fixed some bugs
around very large URIs and headers, and we have also addressed
conformance bugs to more closely conform to RFCs 9110 and 9112.
As part of this work, the APIs for WebSockets changed slightly
as well. In particular the properties available for accessing
headers have changed.
There is still documentation conversion work to do, and additional
functionality (such as proper support for chunked transfers), but
this is a big step in the right direction.
|
| |
|
|
|
|
|
|
|
|
| |
This simplified API lets callbacks obtain the response from the
connection objection directly, and does not require the aio to carry
it as a parameter. Further, the request and response are both
stored inline in the connection, reducing allocations.
This is at present only for the server; the client will get a similar
set of changes.
|
| |
|
|
|
|
| |
There are no other consumers for this, and reasonably unlikely to
be others for now. (Other use cases are JWTs, but that would be
another whole set of functionality that we're not ready to take on.)
|
| |
|
|
|
|
| |
Nothing else uses it, and nothing else *should* use it because SHA1 is insecure.
WebSockets have to use it by definition, unfortunately. The implementation is
not very fast, but doesn't have to be for the use case of websocket keying.
|
| | |
|
| | |
|
| |
|
|
| |
This also makes `nng_http_handler_set_host` never fail (API break).
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This allows us to explicitly stop streams, dialers, and listeners,
before we start tearing down things. This hopefully will be useful
in resolving use-after-free bugs in http, tls, and websockets.
The new functions are not yet documented, but they are
nng_stream_stop, nng_stream_dialer_stop, and nng_stream_listener_stop.
They should be called after close, and before free. The close
functions now close without blocking, but the stop function is
allowed to block.
|
| |
|
|
| |
This covers both the ttransport and the supplemental layers.
|
| | |
|
| |
|
|
| |
This is simpler, and more reliable than using socket options.
|
| |
|
|
| |
This was a crasher bug (found by test suite).
|
| |
|
|
|
|
|
|
| |
This also fixes a couple of minor bugs -- changing the socket name
could be incorrect as the termination was not applied properly,
and the ZeroTier code got a change -- note that it is only possible
in this revised version to pass a single moon ID for ZT orbit.
The ZT code is a bit stale, and untested anyway.
|
| |
|
|
| |
This was not really used or useful.
|
| |
|
|
|
|
| |
This also allows to remove most of the transport headers.
Only zerotier.h sticks around, and only for now. (We expect to
eject it into a separate module.)
|
| |
|
|
|
|
| |
It's not absolutely clear that this fixes all of the possible
cases, but we're hopeful that this does -- and in any event the
bug does not seem to reproduce easily anymore.
|
| |
|
|
| |
fixes #1733 deadlock in websocket listener close
|
| |
|
|
|
|
| |
This takes one less parameter, and is simpler. It will let us
reclaim the aio_prov_extra data space as well, so that we can
use it for other purposes.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a sweeping cleanup of the transport logic around options,
and also harmonizes the names used when setting or getting options.
Additionally, legacy methods are now moved into a separate file and
can be elided via CMake or a preprocessor define.
Fundamentally, the ability to set to transport options via the socket
is deprecated; there are numerous problems with this and my earlier
approaches to deal with this have been somewhat misguided. Further
these approaches will not work with future protocol work that is
planned (were some options need to be negotiated with peers at the
time of connection establishment.)
Documentation has been updated to reflect this. The test suites still
make rather broad use of the older APIs, and will be converted later.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
This adds new options, NNG_OPT_WS_SEND_TEXT and NNG_OPT_WS_RECV_TEXT
that permit communication with WebSocket peers that insist on using
TEXT frames (stream mode only). The support is limited, as NNG does
no validation of the frame contents to check for UTF-8 compliance.
|
| |
|
|
|
| |
This only does it for rep, but it also has changes that should increase
the overall test coverage for the REP protocol
|
| | |
|
| |
|
|
|
|
| |
Also, this has refactored the websocket stream test to the new
acutest.h, and includes a much deeper test of fragmentation and
reassembly of websocket streams.
|
| |
|
|
|
| |
It's possible for an empty chunk to have a NULL data pointer.
Even when copying zero bytes, this makes clang somewhat unhappy.
|
| | |
|
| | |
|
| |
|
|
|
| |
This permits the stats dump to avoid some extra buffering,
and resolves a complaint about possible format buffer overruns.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This is a major change, and includes changes to use a polymorphic
stream API for all transports. There have been related bugs fixed
along the way. Additionally the man pages have changed.
The old non-polymorphic APIs are removed now. This is a breaking
change, but the old APIs were never part of any released public API.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This changes much of the internal API for TCP option handling, and
includes hooks for some of this in various consumers. Note that the
consumers still need to have additional work done to complete them,
which will be part of providing public "raw" TLS and WebSocket APIs.
We would also like to finish addressing the call sites of
nni_tcp_listener_start() that assume the sockaddr is modified --
it would be superior to use the NNG_OPT_LOCADDR option. Thaat will be
addressed in a follow up PR.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This changes the signature of the aio cancellation routines
to take the argument for cancellation directly, so we do not
need to lookup the argument using the nni_aio_get_prov_data.
We should probably consider eliminating nni_aio_get_prov_data,
and co, and changing the prov_extra to reflect prov_data. Later.
|
| |
|
|
| |
fixes #651 Use after free in websocket
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #490 posix_epdesc use-after-free bug
fixes #489 Sanitizer based testing would help
fixes #492 Numerous memory leaks found with sanitizer
This introduces support for compiler-based sanitizers when using
clang or gcc (and not on Windows). See NNG_SANITIZER for possible
settings such as "thread" or "address".
Furthermore, we have fixed the issues we found with both the
thread and address sanitizers. We believe that the thread issues
pointed to a low frequency use-after-free responsible for rare
crashes in some of the tests.
The tests generally have their timeouts doubled when running under
a sanitizer, to account for the extra long times that the sanitizer
can cause these to take.
While here, we also changed the compat_ws test to avoid a particularly
painful and time consuming DNS lookup, and we made the nngcat_unlimited
test a bit more robust by waiting before sending traffic.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #464 Support NN_WS_MSG_TYPE option (compat)
fixes #415 websocket does not honor recv maxsize
This fixes a significant (and security) issue in websocket, where the
code does not honor a maximum receive size. We've exposed new API
(internal) to set the limit on the frame size, and we've changed the
default to *unlimited* for that internal API. (But the default for SP
sockets, which are the only consumers at present, is still 1MB just like
all other SP transports.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
