| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
| |
This is part of our work to improve type safety/awareness, and also
improve debugger support, for NNG error codes. There are still quite
a few more but this should help.
|
| |
|
| |
basically when aio timeout before the connaio of dialer, and users try to free the http_client obj will end in infinite blocking at nni_http_client_fini. Possibly at nni_aio_free(c->aio); or nng_stream_dialer_free(c->dialer); Both racing case is due to the ingnored aio aborting here. Because the aio_begin is called before it is put into the nni_list. I assume you shall abort it no matter if it is in the dialing list.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The body content not being consumed was leading to misparses, where
we consumed body data as if it were a request. When mixed with proxies
this could lead to a security problem where the following request
content submitted from a different client winds up as stolen request
body content.
This also ensures we actually deliver errors to clients without
prematurely closing the connection. (There are still problems
where the connection may be closed prematurely for an overlarge
header.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This represents a major change in the HTTP code base, consisting
of a complete revamp of the HTTP API. The changes here are too
numerous to mention, but the end result should be a vastly
simpler API for both server and client applications.
Many needless allocations were removed by providing fixed buffers
for various parameters and headers when possible.
A few bugs were fixed. Most especially we have fixed some bugs
around very large URIs and headers, and we have also addressed
conformance bugs to more closely conform to RFCs 9110 and 9112.
As part of this work, the APIs for WebSockets changed slightly
as well. In particular the properties available for accessing
headers have changed.
There is still documentation conversion work to do, and additional
functionality (such as proper support for chunked transfers), but
this is a big step in the right direction.
|
| |
|
|
|
| |
This is a step towards simplifying this API and ultimately simplifying
the HTTP callback API used for the server side.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This allows us to explicitly stop streams, dialers, and listeners,
before we start tearing down things. This hopefully will be useful
in resolving use-after-free bugs in http, tls, and websockets.
The new functions are not yet documented, but they are
nng_stream_stop, nng_stream_dialer_stop, and nng_stream_listener_stop.
They should be called after close, and before free. The close
functions now close without blocking, but the stop function is
allowed to block.
|
| |
|
|
| |
the separation of nni_url and nng_url.
|
| |
|
|
| |
This is simpler, and more reliable than using socket options.
|
| |
|
|
|
|
| |
This eliminates some run-time initialization, moving it to compile time.
Additional follow up work will expand on this to simplify initialization
and reduce the need for certain locks.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This also properly reenables the wssfile_test.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a sweeping cleanup of the transport logic around options,
and also harmonizes the names used when setting or getting options.
Additionally, legacy methods are now moved into a separate file and
can be elided via CMake or a preprocessor define.
Fundamentally, the ability to set to transport options via the socket
is deprecated; there are numerous problems with this and my earlier
approaches to deal with this have been somewhat misguided. Further
these approaches will not work with future protocol work that is
planned (were some options need to be negotiated with peers at the
time of connection establishment.)
Documentation has been updated to reflect this. The test suites still
make rather broad use of the older APIs, and will be converted later.
|
| |
|
|
|
|
|
|
| |
This introduces support for an external wolfSSL plugin, and generally
creates the framework for pluggable TLS implementations.
The wolfSSL engine is provided via an external module (git submodule),
available either under a GPLv3 license or a commercial license.
|
| |
|
|
|
| |
This only does it for rep, but it also has changes that should increase
the overall test coverage for the REP protocol
|
| | |
|
| |
|
|
| |
Otherwise connections are not closed in reaper.
|
| |
|
|
|
|
|
|
|
| |
This is a major change, and includes changes to use a polymorphic
stream API for all transports. There have been related bugs fixed
along the way. Additionally the man pages have changed.
The old non-polymorphic APIs are removed now. This is a breaking
change, but the old APIs were never part of any released public API.
|
| |
|
|
|
|
|
|
|
|
| |
This change makes embedding nng + nggpp (or other projects depending on
nng) in cmake easier. The header files are moved to a separate include
directory. This also makes installation of the headers easier, and
allows clearer identification of private vs public heade files.
Some additional cleanups were performed by @gedamore, but the main
credit for this change belongs with @gregorburger.
|
| |
|
|
| |
fixes #776 Configuration of mbedTLS should warn about license
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the client side only, although the work is structured to
support server applications. The chunked API is for now private,
although the intent to is to make it public for applications who
really want to use it.
Note that chunked transfer encoding puts data through extra copies.
First it copies through the buffering area (because I have to be able
to extract variable length strings from inside the data stream), and then
again to reassemble the chunks into a single unified object.
We do assume that the user wants the entire thing as a single object.
This means that using this to pull unbounded data will just silently
consume all memory. Use caution!
|
| |
|
|
|
| |
fixes #718 http_transact is still not right
fixes #719 calculation of buffer size is incorrect in http
|
| |
|
|
|
| |
This adds a couple of new methods, and related documentation
and test cases.
|
| |
|
|
|
|
|
|
|
| |
This changes the signature of the aio cancellation routines
to take the argument for cancellation directly, so we do not
need to lookup the argument using the nni_aio_get_prov_data.
We should probably consider eliminating nni_aio_get_prov_data,
and co, and changing the prov_extra to reflect prov_data. Later.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #179 DNS resolution should be done at connect time
fixes #586 Windows IO completion port work could be better
fixes #339 Windows iocp could use synchronous completions
fixes #280 TCP abstraction improvements
This is a rather monstrous set of changes, which refactors TCP, and
the underlying Windows I/O completion path logic, in order to obtain
a cleaner, simpler API, with support for asynchronous DNS lookups performed
on connect rather than initialization time, the ability to have multiple
connects or accepts pending, as well as fewer extraneous function calls.
The Windows code also benefits from greatly reduced context switching,
fewer lock operations performed, and a reduced number of system calls
on the hot code path. (We use automatic event resetting instead of manual.)
Some dead code was removed as well, and a few potential edge case leaks
on failure paths (in the websocket code) were plugged.
Note that all TCP based transports benefit from this work. The IPC code
on Windows still uses the legacy IOCP for now, as does the UDP code (used
for ZeroTier.) We will be converting those soon too.
|
| |
|
|
|
|
|
| |
While here, there is a little more endpoint termination hardening.
Running this code base seems to not incur any faults beyond the very
rare TCP port conflict that can occur from our random port selection
in the test suite.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #397 Need to cast zoneid
fixes #395 sun is predefined on illumos/Solaris
fixes #394 alloca needs to #include <alloca.h>
fixes #399 Cannot use SVR4.2 specific msghdr
fixes #402 getpeerucred needs a NULL initialized ucred
fixes #403 syntax error in posix_tcp - attempt to return void
fixes #407 illumos getegid wrong
fixes #406 nni_idhash_count is dead code
fixes #404 idhash typedef redeclared
fixes #405 warning: newline not last character in file
This is basically a slew of related bug fixes required to make this
work on illumos. Note that the fixes are not "complete", because
more work is required to support port events given that epoll is busted
on illumos.
We also fixed a bunch of things that aren't actually "bugs" per se, but
really just warnings. Silencing them makes things better for everyone.
Apparently not all compilers are equally happy with redundant (but
otherwise identical) typedefs; we use structs in some places instead of
shorter type names to silence these complaints.
Note that IPC permissions (the mode bits on the socket vnode) are not
validated on SunOS systems. This change includes documentation to reflect
that.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| | |
|
| |
|
|
|
|
| |
We move the HTTP definitions out of the core nng.h and into
a supplemental header. Most of this change was trivial updates
to all of the HTTP related manual pages.
|
| |
|
|
|
|
|
| |
We enabled verbose compiler warnings, and found a lot of issues.
Some of these were even real bugs. As a bonus, we actually save
some initialization steps in the compat layer, and avoid passing
some variables we don't need.
|
| |
|
|
|
|
|
|
| |
This addresses the use of the pipe special field, and eliminates it.
The message APIs (recvmsg, sendmsg) need to be updated as well still,
but I want to handle that as part of a separate issue.
While here we fixed various compiler warnings, etc.
|
| | |
|
|
|
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
