| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
This is not quite complete, but it sets the stage for other
protocols (such as zmq or mqtt) to be added to the project.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are a few major areas in this change.
* CMake options are now located in a common cmake/NNGOptions.cmake
file. This should make it easier for folks to figure out what
the options are, and how they are used.
* Tests are now scoped with their directory name, which should
avoid possible name collisions with test names.
* A number of tests have been either moved or incorporated into
the newer testutil/acutest framework. We are moving away from
my old c-convey framework to something easier to debug.
* We use CMake directories a bit more extensively leading to a much
cleaner CMake structure. It's not complete, but a big step in the
right direction, and a preview of future work.
* Tests are now run with verbose flags, so we get more test results
in the CI/CD logs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a sweeping cleanup of the transport logic around options,
and also harmonizes the names used when setting or getting options.
Additionally, legacy methods are now moved into a separate file and
can be elided via CMake or a preprocessor define.
Fundamentally, the ability to set to transport options via the socket
is deprecated; there are numerous problems with this and my earlier
approaches to deal with this have been somewhat misguided. Further
these approaches will not work with future protocol work that is
planned (were some options need to be negotiated with peers at the
time of connection establishment.)
Documentation has been updated to reflect this. The test suites still
make rather broad use of the older APIs, and will be converted later.
|
| |
|
|
|
| |
This doesn't modularize all the tests yet, but it goes a long way
in the right direction.
|
| |
|
|
|
|
|
|
|
|
|
| |
fixes #1288 id allocation can overallocate
fixes #1126 consider removing lock from idhash
This substantially refactors the id hash code, giving a cleaner API,
and eliminating a extra locking as well as some wasteful allocations.
The ZeroTier code has it's own copy, that is 64-bit friendly, as the
rest of the consumers need only a simpler 32-bit API.
|
| | |
|
| |
|
|
|
|
| |
This also exposes an nng_thread_set_name() function for
applications to use. All NNG thread names start with "nng:".
Note that support is highly dependent on the operating system.
|
| | |
|
| | |
|
| |
|
|
|
| |
This only does it for rep, but it also has changes that should increase
the overall test coverage for the REP protocol
|
| | |
|
| | |
|
| |
|
|
|
|
| |
There is quite some package loss, which I will pursue
later (in nng / zerotier). For now it helps to tune these settings
to keep the peer relations.
|
| |
|
|
| |
nni_getopt needs extra var
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This change makes embedding nng + nggpp (or other projects depending on
nng) in cmake easier. The header files are moved to a separate include
directory. This also makes installation of the headers easier, and
allows clearer identification of private vs public heade files.
Some additional cleanups were performed by @gedamore, but the main
credit for this change belongs with @gregorburger.
|
| |
|
|
|
|
|
|
|
|
|
| |
This is a significant refactor of the library configuration.
We use the modern package configuration helper, with a template
script that also does the find_package dance for any of our
dependencies.
We also have restructured the code so that most protocols and
transports have their configuration isolated to their own CMakeLists
file, reducing the size of the global CMakeLists file.
|
| | |
|
| | |
|
| |
|
|
|
| |
These options are undocumented -- for now. We need to get some
experience with them to ensure that they are worth keeping.
|
| |
|
|
|
|
|
|
| |
While the fragment size itself fits within 16-bits, when we do math
on it (such as multiplying by a fragment count), we may need to calculate
values that exceed 16-bits (e.g. 1MB). Therefore we should treat this
as a size_t, but cast it down to 16-bits only when we write it out to
the packet field.
|
| | |
|
| |
|
|
|
|
|
| |
This changes the code to make use of a different project we have
created (libzerotiercore) that is "CMake clean". This should make
using and configuring this code *much* better. It may also have the
benefit of making this configuration work better for Windows systems.
|
| |
|
|
|
|
| |
This reverts 707, and provides a correct fix for busted
port discrimination. It also has a few minor cleanups
and reduces the listen timeout to 10 seconds.
|
| | |
|
| |
|
|
|
| |
While here we separate out the dialer and listener options, so
that options for tuning connection are only available for listeners.
|
| |
|
|
|
|
| |
This both makes new functions available to the core,
and addresses a bug which would have prevented building the
ZeroTier transport on Windows.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This changes the signature of the aio cancellation routines
to take the argument for cancellation directly, so we do not
need to lookup the argument using the nni_aio_get_prov_data.
We should probably consider eliminating nni_aio_get_prov_data,
and co, and changing the prov_extra to reflect prov_data. Later.
|
| | |
|
| |
|
|
|
| |
This changeset needs work. We are seeing errors described by
This reverts commit d7f7c896c0ede24249ef63b1e45b1878bf4bd473.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #208 pipe start should occur before connect / accept
fixes #616 Race condition closing between header & body
This refactors the transports to handle their own connection
handshaking before passing the pipe to the socket. This
changes and simplifies the setup. This also fixes a rather
challenging race condition described by #616.
|
| |
|
|
|
|
|
|
|
|
| |
fixes #573 atomic flags could help
This introduces a new atomic flag, and reduces some of the global
locking. The lock refactoring work is not yet complete, but this is
a positive step forward, and should help with certain things.
While here we also fixed a compile warning due to incorrect types.
|
| |
|
|
|
|
|
|
|
|
| |
This separates the plumbing for endpoints into distinct
dialer and listeners. Some of the transports could benefit
from further separation, but we've done some rather larger
separation e.g. for the websocket transport.
IPC would be a good one to update later, when we start looking
at exposing a more natural underlying API.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fixes #538 setopt should have an explicit chkopt routine
fixes #537 Internal TCP API needs better name separation
fixes #524 Option types should be "typed"
This is a rework of the option management code, to make it both clearer
and to prepare for further work to break up endpoints. This reduces
a certain amount of dead or redundant code, and actually saves cycles
when setting options, as some loops were not terminated that should have
been.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes #419 want to nni_aio_stop without blocking
This actually introduces an nni_aio_close() API that causes
nni_aio_begin to return NNG_ECLOSED, while scheduling a callback
on the AIO to do an NNG_ECLOSED as well. This should be called
in non-blocking close() contexts instead of nni_aio_stop(), and
the cases where we call nni_aio_fini() multiple times are updated
updated to add nni_aio_stop() calls on all "interlinked" aios before
finalizing them.
Furthermore, we call nni_aio_close() as soon as practical in the
close path. This closes an annoying race condition where the
callback from a lower subsystem could wind up rescheduling an
operation that we wanted to abort.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| | |
|
| |
|
|
|
|
|
| |
fixes #249 nngcat needs test cases
fixes #416 transports do not permit unlimited message size with 0
fixes #417 nngcat truncates input files to 4k
fixes #348 nngcat should have switch to adjust maximum receive size
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes a fundamental flaw in the way aio structures were
handled. In paticular, aio expiration could race ahead, and
fire before the aio was properly registered by the provider.
This ultimately led to the possibility of duplicate completions
on the same aio.
The solution involved breaking up nni_aio_start into two functions.
nni_aio_begin (which can be run outside of external locks) simply
validates that nni_aio_fini() has not been called, and clears certain
fields in the aio to make it ready for use by the provider.
nni_aio_schedule does the work to register the aio with the expiration
thread, and should only be called when the aio is actually scheduled
for asynchronous completion. nni_aio_schedule_verify does the same thing,
but returns NNG_ETIMEDOUT if the aio has a zero length timeout.
This change has a small negative performance impact. We have plans to
rectify that by converting nni_aio_begin to use a locklesss flag for
the aio->a_fini bit.
While we were here, we fixed some error paths in the POSIX subsystem,
which would have returned incorrect error codes, and we made some
optmizations in the message queues to reduce conditionals while holding
locks in the hot code path.
|
| | |
|
| |
|
|
|
|
|
|
| |
fixes #325 synchronous aio completion crash
fixes #327 move nni_clock() operations to outside the nni_aio_lk.
This work was done for the context tree, and is necessary to properly
enable that branch.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #302 nng_dialer/listener/pipe_getopt_sockaddr desired
This adds plumbing to pass and check the type of options
all the way through.
NNG_ZT_OPT_ORBIT is type UINT64, but you can use the untyped form to
pass two of them if needed.
No typed access for retrieving strings yet. I think this should allocate
a pointer and copy that out, but that's for later.
|
| |
|
|
| |
fixes #290 sockaddr improvements
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This does a few things. First it closes a case where a dropped
message could prevent subsequent connection attempts from getting through.
Second, it changes the rate at which we retry, and the timeout, to be
a lot more aggressive when attempting to establish a connection. We
retry every 500 ms, for up to 2 minutes, before giving up. This gives
a lot more resilience in the face of message loss that is typical of
ZeroTier in some environments when first establishing communication.
Third, makes the values for the connection attempts *tunable*, so
that applications can adjust for different deployment scenarios.
Fourth, it includes the ability to get the UDP socket name. This was
needed during some debug, and may be useful for a real UDP transport
later, so we're keeping it.
Finally, we added documentation for the above items.
|
| |
|
|
| |
fixes #267 zerotier transport should lock ZT_HOME
|
