| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
| |
fixes #573 atomic flags could help
This introduces a new atomic flag, and reduces some of the global
locking. The lock refactoring work is not yet complete, but this is
a positive step forward, and should help with certain things.
While here we also fixed a compile warning due to incorrect types.
|
| |
|
|
|
|
|
|
|
|
| |
This separates the plumbing for endpoints into distinct
dialer and listeners. Some of the transports could benefit
from further separation, but we've done some rather larger
separation e.g. for the websocket transport.
IPC would be a good one to update later, when we start looking
at exposing a more natural underlying API.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #485 Honor BUILD_SHARED_LIBS
fixes #483 Don't expose private symbols in shared library
fixes #481 Export CMake target
This is a "large" commit involving changes that don't affect the
code directly, but which have an impact on how we package and build
our project.
The most significant of these changes is that we now build only
either a shared or a static library, depending on the setting of
the BUILD_SHARED_LIBS option. We also suppress private symbols
from being exposed when the underlying toolchain lets us do so.
Minor updates to the way we version the ABI are used, and we now
have a nice exported CMake project.
To import this project in another, simply do find_package(nng)
and you can add target_link_libraries(nng::nng) to your targets.
CMake does the rest for you.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes #326 consider nni_taskq_exec_synch()
fixes #410 kqueue implementation could be smarter
fixes #411 epoll_implementation could be smarter
fixes #426 synchronous completion can lead to panic
fixes #421 pipe close race condition/duplicate destroy
This is a major refactoring of two significant parts of the code base,
which are closely interrelated.
First the aio and taskq framework have undergone a number of simplifications,
and improvements. We have ditched a few parts of the internal API (for
example tasks no longer support cancellation) that weren't terribly useful
but added a lot of complexity, and we've made aio_schedule something that
now checks for cancellation or other "premature" completions. The
aio framework now uses the tasks more tightly, so that aio wait can
devolve into just nni_task_wait(). We did have to add a "task_prep()"
step to prevent race conditions.
Second, the entire POSIX poller framework has been simplified, and made
more robust, and more scalable. There were some fairly inherent race
conditions around the shutdown/close code, where we *thought* we were
synchronizing against the other thread, but weren't doing so adequately.
With a cleaner design, we've been able to tighten up the implementation
to remove these race conditions, while substantially reducing the chance
for lock contention, thereby improving scalability. The illumos poller
also got a performance boost by polling for multiple events.
In highly "busy" systems, we expect to see vast reductions in lock
contention, and therefore greater scalability, in addition to overall
improved reliability.
One area where we currently can do better is that there is still only
a single poller thread run. Scaling this out is a task that has to be done
differently for each poller, and carefuly to ensure that close conditions
are safe on all pollers, and that no chance for deadlock/livelock waiting
for pfd finalizers can occur.
|
| |
|
|
|
|
|
|
| |
This replaces the epoll support with proper illumos/SunOS port
events. The port event support is structured so that it actually
is superior to epoll and kqueue, because it avoids a single master
lock on the poller. In the future we will explore this for macOS
and Linux pollers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This provides context support for REQ and REP sockets.
More discussion around this is in the issue itself.
Optionally we would like to extend this to the surveyor pattern.
Note that we specifically do not support pollable descriptors
for non-default contexts, and the results of using file descriptors
for polling (NNG_OPT_SENDFD and NNG_OPT_RECVFD) is undefined.
In the future, it might be nice to figure out how to factor in
optional use of a message queue for users who want more buffering,
but we think there is little need for this with cooked mode.
|
| |
|
|
| |
fixes #33
|
| | |
|
| |
|
|
|
|
| |
Basically, we have moved the compat stuff into a separate directory.
Compatibility layer users will have to update their compile flags, but
should be able to avoid changing any *source* files with this change.
|
| |
|
|
|
|
|
|
|
| |
These are incremental updates... we avoid using install() in the
subdirectories, so that we can adapt properly to them in the
single parent directory.
We have started some of the work to improve support for CPack. This
is still not yet done, but work in progress.
|
| |
|
|
|
|
|
| |
This introduces portable primitives for time, random numbers,
synchronization primitives, and threading. These are somewhat
primitive (least common denominiators), but they can help with writing
portable applications, especially our own demo apps.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces enough of the HTTP API to support fully server
applications, including creation of websocket style protocols,
pluggable handlers, and so forth.
We have also introduced scatter/gather I/O (rudimentary) for
aios, and made other enhancements to the AIO framework. The
internals of the AIOs themselves are now fully private, and we
have eliminated the aio->a_addr member, with plans to remove the
pipe and possibly message members as well.
A few other minor issues were found and fixed as well.
The HTTP API includes request, response, and connection objects,
which can be used with both servers and clients. It also defines
the HTTP server and handler objects, which support server applications.
Support for client applications will require a client object to be
exposed, and that should be happening shortly.
None of this is "documented" yet, bug again, we will follow up shortly.
|
| |
|
|
|
|
|
|
|
| |
This refactor of the file API provides a simpler and easier to use
interface for our needs (and simpler to implement) in both the
ZeroTier transport and the HTTP/TLS file accesses. It also removes
some restrictions present on the old one, although it is still not
suitable for working with large files. (It will work, just be
very inefficient as the entire file must be loaded into memory.)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces the wss:// scheme, which is available and works like
the ws:// scheme if TLS is enabled in the library.
The library modularization is refactored somewhat, to make it easier
to use. There is now a single NNG_ENABLE_TLS that enables TLS support
under the hood.
This also adds a new option for the TLS transport, NNG_OPT_TLS_CONFIG
(and a similar one for WSS, NNG_OPT_TLS_WSS_CONFIG) that offer access
to the underlying TLS configuration object, which now has a public API
to go with it as well.
Note that it is also possible to use pure HTTPS using the *private*
API, which will be exposed in a public form soon.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a rather large changeset -- it fundamentally adds websocket
transport, but as part of this changeset we added a generic framework
for both HTTP and websocket. We also made some supporting changes to
the core, such as changing the way timeouts work for AIOs and adding
additional state keeping for AIOs, and adding a common framework for
deferred finalization (to avoid certain kinds of circular deadlocks
during resource cleanup). We also invented a new initialization framework
so that we can avoid wiring in knowledge about them into the master
initialization framework.
The HTTP framework is not yet complete, but it is good enough for simple
static serving and building additional services on top of -- including
websocket. We expect both websocket and HTTP support to evolve
considerably, and so these are not part of the public API yet.
Property support for the websocket transport (in particular address
properties) is still missing, as is support for TLS.
The websocket transport here is a bit more robust than the original
nanomsg implementation, as it supports multiple sockets listening at
the same port sharing the same HTTP server instance, discriminating
between them based on URI (and possibly the virtual host).
Websocket is enabled by default at present, and work to conditionalize
HTTP and websocket further (to minimize bloat) is still pending.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a new transport (compatible with the TLS
transport from mangos), using TLS v1.2.
To use the new transport, you must have the mbed TLS library
available on your system (Xenial libmbedtls-dev). You can use
version 2.x or newer -- 1.3.x and PolarSSL versions are not
supported.
You enable the TLS transport with -DNNG_TRANSPORT_TLS=ON in the CMake
configuration.
You must configure the server certificate by default, and this can only
be done using nng options. See the nng_tls man page for details.
This work is experimental, and was made possible by Capitar IT Group BV,
and Staysail Systems, Inc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This makes all the protocols and transports optional. All
of them except ZeroTier are enabled by default, but you can
now disable them (remove from the build) with cmake options.
The test suite is modified so that tests still run as much
as they can, but skip over things caused by missing functionality
from the library (due to configuration).
Further, the constant definitions and prototypes for functions
that are specific to transports or protocols are moved into
appropriate headers, which should be included directly by
applications wishing to use these.
We have also added and improved documentation -- all of the
transports are documented, and several more man pages for
protocols have been added. (Req/Rep and Surveyor are still
missing.)
|
| |
|
|
|
|
|
| |
We added nng_inproc man page, and a header with an explicit
registration method for it. We also fixed up the markup on
several other pages, and corrected the include statements for
nng_zerotier.
|
| |
|
|
|
|
|
|
|
| |
This eliminates the "quasi-functional" notify API altogether.
The aio framework will be coming soon to replace it.
As a bonus, apps (legacy apps) that use the notification FDs
will see improved performance, since we don't have to context
switch to give them a notification.
|
| |
|
|
|
|
|
|
|
|
|
| |
This includes converting the ZeroTier transport to use these.
The new API supports file creation, retrieval, and deletion. It
also supports directory methods for traversal, creation, and
deletion. It also has a few methods to obtain well-known directories
like $TMPDIR and $HOME.
A rich test suite for this functionality is added as well.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We allow some properties to be set on endpoints after they are
started; transports now responsible for checking that. (The new
values will only apply to new connections of course!)
We added short-hand functions for pipe properties, and also added
uint64_t shorthands across the board.
The zerotier documentation got some updates (corrections). We have
also added a separate header now for the ZT stuff.
Also, dialers and listeners do not intermix anymore -- we test that
only a dialer can be used with setting dialer options, and likewise
for listeners.
|
| |
|
|
|
|
|
| |
This implements the basic UDP functionality for Windows (required
for ZeroTier for example). We have also introduced a UDP test suite
to validate that this actually works. While here a few Windows
compilation warnings / nits were fixed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The ZeroTier transport is experimental at this point, and not enabled
by default. It does not work with Windows yet (the Windows platform
needs UDP support first.)
Configure with -DNNG_ENABLE_ZEROTIER=yes -DNNG_ZEROTIER_SOUCE=<path>
The <path> must point to a dev branch of the ZeroTierOne source tree,
checked out, and built with a libzerotiercore.a in the top directory,
and a ZeroTierOne.h header located at include. The build will add
-lc++ to the compile, as the ZeroTier core functionality is written in
C++ and needs some runtime support (e.g. new, delete, etc.)
|
| |
|
|
|
|
|
|
| |
We have our versions of strdup, strlcat, and strlcpy.
This means we can avoid using snprintf() in many cases
(saving cycles), and we can get safer checks. We use
the platform supplied versions of these if they exist
(wrapping with nni_xxx versions.)
|
| |
|
|
|
|
|
|
|
|
| |
We only compile files that are appropriate for the platform. (We
still have guards in place, to allow for a future single .C file
to be built from all the sources.) We also remove the subsystem defines;
if a new platform needs to deviate from POSIX in ways beyond what we
intended here, then that platform should just copy those parts into
a new platform directory, rather than cross including portions from
POSIX.
|
| |
|
|
|
|
|
|
|
|
| |
The PAIR_V1 protocol supports both raw and cooked modes, and has loop
prevention included. It also has a polyamorous mode, wherein it allows
multiple connections to be established. In polyamorous mode (set by
an option), the sender requests a paritcular pipe by setting it on the
message.
We default to PAIR_V1 now.
|
| |
|
|
| |
fixes #23 Restore the old idhash logic for sockets
|
| |
|
|
|
|
| |
This includes async send and recv, driven from the poller. This will
be requierd to support the underlying UDP and ZeroTier transports in
the future. (ZeroTier is getting done first.)
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It turns out that I had to fix a number of subtle asynchronous
handling bugs, but now TCP is fully asynchronous. We need to
change the high-level dial and listen interfaces to be async
as well.
Some of the transport APIs have changed here, and I've elected
to change what we expose to consumers as endpoints into seperate
dialers and listeners. Under the hood they are the same, but
it turns out that its helpful to know the intended use of the
endpoint at initialization time.
Scalability still occasionally hangs on Linux. Investigation
pending.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This eliminates the two threads per pipe that were being used to provide
basic I/O handling, replacing them with a single global thread for now,
that uses poll and nonblocking I/O. This should lead to great scalability.
The infrastructure is in place to easily expand to multiple polling worker
threads. Some thought needs to be given about how to scale this to engage
multiple CPUs. Horizontal scaling may also shorten the poll() lists easing
C10K problem.
We should look into better solutions than poll() for platforms that have
them (epoll on Linux, kqueue on BSD, and event ports on illumos).
Note that the file descriptors start out in blocking mode for now, but
then are placed into non-blocking mode. This is because the negotiation
phase is not yet callback driven, and so needs to be synchronous.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
We will still need some kind of specific handling of cancellation for
msg queues, but it will be simpler to just implement that for the queues,
and not worry about cancellation in the general case around poll etc.
(The low level poll and I/O routines will get notified by their underlying
transport pipes/descriptors closing.)
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
This adds nn_device and nng_device. There were some internal changes
required to fix shutdown / close issues. Note that we shut down the
sockets when exiting from device -- this is required to make both threads
see the failure and bail, since we are not using a single event loop.
I also noticed that the bus protocol had a bug where it would send
messages back to the originator. This was specifically tested for in
the compat_device test, and we have fixed it.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This does a few things. First it closes some preexisting leaks.
Second it tightens the overall close logic so that we automatically
discard idhash resources (while keeping numeric values for next id
etc. around) when the last socket is closed. This then eliminates
the need for applications to ever explicitly terminate resources.
It turns out platform-specific resources established at nni_init()
time might still be leaked, but it's also the case that we now no
longer dynamically allocate anything at platform initialization time.
(This presumes that the platform doesn't do so under the hood when
creating critical sections or mutexes for example.)
|
| |
|
|
|
|
|
|
|
|
| |
This compiles correctly, but doesn't actually deliver events yet.
As part of this, I've made most of the initializables in nng
safe to tear-down if uninitialized (or set to zero e.g. via calloc).
This makes it loads easier to write the teardown on error code, since
I can deinit everything, without worrying about which things have been
initialized and which have not.
|
