aboutsummaryrefslogtreecommitdiff
path: root/src/transport/inproc
Commit message (Collapse)AuthorAge
* fixes #1345 Restructure the source treeGarrett D'Amore2021-01-01
| | | | | This is not quite complete, but it sets the stage for other protocols (such as zmq or mqtt) to be added to the project.
* Work for test refactoring.Garrett D'Amore2020-11-18
| | | | | | | | | | | | | | | | | | | | | | There are a few major areas in this change. * CMake options are now located in a common cmake/NNGOptions.cmake file. This should make it easier for folks to figure out what the options are, and how they are used. * Tests are now scoped with their directory name, which should avoid possible name collisions with test names. * A number of tests have been either moved or incorporated into the newer testutil/acutest framework. We are moving away from my old c-convey framework to something easier to debug. * We use CMake directories a bit more extensively leading to a much cleaner CMake structure. It's not complete, but a big step in the right direction, and a preview of future work. * Tests are now run with verbose flags, so we get more test results in the CI/CD logs.
* fixes #1071 tran_chkopt can be cleaned upGarrett D'Amore2020-11-15
| | | | | | | | | | | | | | | | | This is a sweeping cleanup of the transport logic around options, and also harmonizes the names used when setting or getting options. Additionally, legacy methods are now moved into a separate file and can be elided via CMake or a preprocessor define. Fundamentally, the ability to set to transport options via the socket is deprecated; there are numerous problems with this and my earlier approaches to deal with this have been somewhat misguided. Further these approaches will not work with future protocol work that is planned (were some options need to be negotiated with peers at the time of connection establishment.) Documentation has been updated to reflect this. The test suites still make rather broad use of the older APIs, and will be converted later.
* fixes #1239 Use after free in tlsGarrett D'Amore2020-05-23
| | | | | Also, addressed a number of Clang-tidy complaints. Potential hangs in close addressed as well.
* fixes #1156 Message cloning could help reduce copies a lotGarrett D'Amore2020-01-20
| | | | | | | | | | | | This introduces reference counting on messages to reduce the data copies. This should have a marked improvement when moving large messages through the system, or when publishing to many subscribers. For some transports, when using large messages, the copy time can be the dominant factor. Note that when a message is actually shared, inproc will still perform an extra copy in order to ensure that it can modify the headers. This will unfortunately always be the case with REQ, as the REQ protocol keeps a copy of the original message so it can retry.
* fixes #1159 inproc could skip message queueGarrett D'Amore2020-01-20
| | | | | | In addition, the message queue code is rather heavy weight, so this may make the inproc code a bit leaner. It also makes backpressure conditions much more reliable since it now is completely unbuffered.
* inproc can use msg_header_clear to save a branch.Garrett D'Amore2020-01-16
|
* fixes #1040 Convert rest of the protocols to new CMake infraGarrett D'Amore2019-12-25
|
* fixes #1032 Figure out Darwin bustednessGarrett D'Amore2019-12-24
| | | | | | | | | | | | | | | | | | | | | | | | fixes #1035 Convey is awkward -- consider acutest.h This represents a rather large effort towards cleaning up our testing and optional configuration infrastructure. A separate test library is built by default, which is static, and includes some useful utilities design to make it easier to write shorter and more robust (not timing dependent) tests. This also means that we can cover pretty nearly all the tests (protocols etc.) in every case, even if the shipped image will be minimized. Subsystems which are optional can now use a few new macros to configure what they need see nng_sources_if, nng_headers_if, and nng_defines_if. This goes a long way to making the distributed CMakefiles a lot simpler. Additionally, tests for different parts of the tree can now be located outside of the tests/ tree, so that they can be placed next to the code that they are testing. Beyond the enabling work, the work has only begun, but these changes have resolved the most often failing tests for Darwin in the cloud.
* fixes #815 Eliminate socket filters on message queuesGarrett D'Amore2019-03-13
| | | | | | | This also eliminates the enforcement of NNG_OPT_RECVMAXSZ for inproc, which never really made much sense. This helps inproc go faster. While here, also clean up the entry point for protocols to support a drain option, since we don't use that anywhere.
* fixes #848 server hang waiting for client handshakeGarrett D'Amore2019-02-23
| | | | | | | | | | | | | | | | | | | | | | | fixes #698 Need TCP stats fixes #699 Need IPC stats fixes #701 Need TLS stats This commit addresses a problem when negotiating using one of the stream based negotiation APIs -- a slow or misbehaving peer can prevent well behaved ones from establishing a connection. The fix is a fairly significant change in how these transports link up, and it does rely on the fact that the socket only has a single accept() or connect() pending at a time (on a given endpoint that is). While here, we have completely revamped the way transport statistics are done, offering a standard API for collecting these statistics. Unfortunately, this completely borks the statistics for inproc. As we are planning to change the way inproc works soon, in order to provide more control and work on performance fixes for the message queue, we feel this is an acceptable trade-off. Furthermore, almost nobody uses inproc for anything, and even fewer people are making use of the statistics at this time.
* fixes #835 inproc should use new option APIGarrett D'Amore2018-12-31
| | | | | Some more changes to use nni_type instead of nni_opt_type are included as well.
* fixes #831 Unify option structures, o_type is unusedGarrett D'Amore2018-12-29
|
* move all public headers to include/nng/ folderGregor Burger2018-11-22
| | | | | | | | | | This change makes embedding nng + nggpp (or other projects depending on nng) in cmake easier. The header files are moved to a separate include directory. This also makes installation of the headers easier, and allows clearer identification of private vs public heade files. Some additional cleanups were performed by @gedamore, but the main credit for this change belongs with @gregorburger.
* fixes #577 target library dependencies should be publicGarrett D'Amore2018-11-05
| | | | | | | | | | | This is a significant refactor of the library configuration. We use the modern package configuration helper, with a template script that also does the find_package dance for any of our dependencies. We also have restructured the code so that most protocols and transports have their configuration isolated to their own CMakeLists file, reducing the size of the global CMakeLists file.
* fixes #4 Statistics supportGarrett D'Amore2018-09-03
| | | | | | | | | | | | | | | | This introduces new public APIs for obtaining statistics, and adds some generic stats for dialers, listeners, pipes, and sockets. Also added are stats for inproc and pairv1 protocol. The other protocols and transports will have stats added incrementally as time goes on. A simple test program, and man pages are provided for this. Start by looking at nng_stat(5). Statistics does have some impact, and they can be disabled by using the advanced NNG_ENABLE_STATS (setting it to OFF, it's ON by default) if you need to build a minimized configuration.
* fixes #673 transports could benefit from access to upper layerGarrett D'Amore2018-08-27
|
* fixes #664 aio cancellation could be betterGarrett D'Amore2018-08-20
| | | | | | | | | This changes the signature of the aio cancellation routines to take the argument for cancellation directly, so we do not need to lookup the argument using the nni_aio_get_prov_data. We should probably consider eliminating nni_aio_get_prov_data, and co, and changing the prov_extra to reflect prov_data. Later.
* fixes #541 inproc does not honor maxrecvsize optionGarrett D'Amore2018-08-20
|
* fixes #522 Separate out the endpoint plumbingGarrett D'Amore2018-06-27
| | | | | | | | | | This separates the plumbing for endpoints into distinct dialer and listeners. Some of the transports could benefit from further separation, but we've done some rather larger separation e.g. for the websocket transport. IPC would be a good one to update later, when we start looking at exposing a more natural underlying API.
* fixes #540 nni_ep_opttype serves no purposeGarrett D'Amore2018-06-13
| | | | | | | | | | | | fixes #538 setopt should have an explicit chkopt routine fixes #537 Internal TCP API needs better name separation fixes #524 Option types should be "typed" This is a rework of the option management code, to make it both clearer and to prepare for further work to break up endpoints. This reduces a certain amount of dead or redundant code, and actually saves cycles when setting options, as some loops were not terminated that should have been.
* fixes #419 want to nni_aio_stop without blocking (#428)Garrett D'Amore2018-05-15
| | | | | | | | | | | | | | | | * fixes #419 want to nni_aio_stop without blocking This actually introduces an nni_aio_close() API that causes nni_aio_begin to return NNG_ECLOSED, while scheduling a callback on the AIO to do an NNG_ECLOSED as well. This should be called in non-blocking close() contexts instead of nni_aio_stop(), and the cases where we call nni_aio_fini() multiple times are updated updated to add nni_aio_stop() calls on all "interlinked" aios before finalizing them. Furthermore, we call nni_aio_close() as soon as practical in the close path. This closes an annoying race condition where the callback from a lower subsystem could wind up rescheduling an operation that we wanted to abort.
* fixes #352 aio lock is burning hotGarrett D'Amore2018-05-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fixes #326 consider nni_taskq_exec_synch() fixes #410 kqueue implementation could be smarter fixes #411 epoll_implementation could be smarter fixes #426 synchronous completion can lead to panic fixes #421 pipe close race condition/duplicate destroy This is a major refactoring of two significant parts of the code base, which are closely interrelated. First the aio and taskq framework have undergone a number of simplifications, and improvements. We have ditched a few parts of the internal API (for example tasks no longer support cancellation) that weren't terribly useful but added a lot of complexity, and we've made aio_schedule something that now checks for cancellation or other "premature" completions. The aio framework now uses the tasks more tightly, so that aio wait can devolve into just nni_task_wait(). We did have to add a "task_prep()" step to prevent race conditions. Second, the entire POSIX poller framework has been simplified, and made more robust, and more scalable. There were some fairly inherent race conditions around the shutdown/close code, where we *thought* we were synchronizing against the other thread, but weren't doing so adequately. With a cleaner design, we've been able to tighten up the implementation to remove these race conditions, while substantially reducing the chance for lock contention, thereby improving scalability. The illumos poller also got a performance boost by polling for multiple events. In highly "busy" systems, we expect to see vast reductions in lock contention, and therefore greater scalability, in addition to overall improved reliability. One area where we currently can do better is that there is still only a single poller thread run. Scaling this out is a task that has to be done differently for each poller, and carefuly to ensure that close conditions are safe on all pollers, and that no chance for deadlock/livelock waiting for pfd finalizers can occur.
* fixes #346 nng_recv() sometimes acts on null `msg` pointerGarrett D'Amore2018-04-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | This closes a fundamental flaw in the way aio structures were handled. In paticular, aio expiration could race ahead, and fire before the aio was properly registered by the provider. This ultimately led to the possibility of duplicate completions on the same aio. The solution involved breaking up nni_aio_start into two functions. nni_aio_begin (which can be run outside of external locks) simply validates that nni_aio_fini() has not been called, and clears certain fields in the aio to make it ready for use by the provider. nni_aio_schedule does the work to register the aio with the expiration thread, and should only be called when the aio is actually scheduled for asynchronous completion. nni_aio_schedule_verify does the same thing, but returns NNG_ETIMEDOUT if the aio has a zero length timeout. This change has a small negative performance impact. We have plans to rectify that by converting nni_aio_begin to use a locklesss flag for the aio->a_fini bit. While we were here, we fixed some error paths in the POSIX subsystem, which would have returned incorrect error codes, and we made some optmizations in the message queues to reduce conditionals while holding locks in the hot code path.
* fixes #336 inproc queue depths are too longGarrett D'Amore2018-04-10
|
* fixes #301 String option handling for getoptGarrett D'Amore2018-03-20
|
* fixes #296 Typed options should validate option typeGarrett D'Amore2018-03-20
| | | | | | | | | | | | | fixes #302 nng_dialer/listener/pipe_getopt_sockaddr desired This adds plumbing to pass and check the type of options all the way through. NNG_ZT_OPT_ORBIT is type UINT64, but you can use the untyped form to pass two of them if needed. No typed access for retrieving strings yet. I think this should allocate a pointer and copy that out, but that's for later.
* fixes #289 nng_sockaddr could just be a unionGarrett D'Amore2018-03-14
| | | | fixes #290 sockaddr improvements
* CMake & CPack improvements.Garrett D'Amore2018-02-21
| | | | | | | | | These are incremental updates... we avoid using install() in the subdirectories, so that we can adapt properly to them in the single parent directory. We have started some of the work to improve support for CPack. This is still not yet done, but work in progress.
* fixes #171 Refactor aio to use generic data fieldsGarrett D'Amore2018-02-08
| | | | | | | | This addresses the use of the pipe special field, and eliminates it. The message APIs (recvmsg, sendmsg) need to be updated as well still, but I want to handle that as part of a separate issue. While here we fixed various compiler warnings, etc.
* fixes #173 Define public HTTP server APIGarrett D'Amore2018-02-01
| | | | | | | | | | | | | | | | | | | | | | | This introduces enough of the HTTP API to support fully server applications, including creation of websocket style protocols, pluggable handlers, and so forth. We have also introduced scatter/gather I/O (rudimentary) for aios, and made other enhancements to the AIO framework. The internals of the AIOs themselves are now fully private, and we have eliminated the aio->a_addr member, with plans to remove the pipe and possibly message members as well. A few other minor issues were found and fixed as well. The HTTP API includes request, response, and connection objects, which can be used with both servers and clients. It also defines the HTTP server and handler objects, which support server applications. Support for client applications will require a client object to be exposed, and that should be happening shortly. None of this is "documented" yet, bug again, we will follow up shortly.
* fixes #219 transports should take URL structure instead of string addressGarrett D'Amore2018-01-22
| | | | | | | | | | This eliminates a bunch of redundant URL parsing, using the common URL logic we already have in place. While here I fixed a problem with the TLS and WSS test suites that was failing on older Ubuntu -- apparently older versions of mbedTLS were unhappy if selecting OPTIONAL verification without a validate certificate chain.
* fixes #209 NNG_OPT_TLS_VERIFIED is bustedGarrett D'Amore2018-01-17
| | | | | fixes #210 Want NNG_OPT_TLS_* options for TLS transport fixes #212 Eliminate a_endpt member of aio
* fixes #3 TLS transportGarrett D'Amore2017-11-20
| | | | | | | | | | | | | | | | | | | This introduces a new transport (compatible with the TLS transport from mangos), using TLS v1.2. To use the new transport, you must have the mbed TLS library available on your system (Xenial libmbedtls-dev). You can use version 2.x or newer -- 1.3.x and PolarSSL versions are not supported. You enable the TLS transport with -DNNG_TRANSPORT_TLS=ON in the CMake configuration. You must configure the server certificate by default, and this can only be done using nng options. See the nng_tls man page for details. This work is experimental, and was made possible by Capitar IT Group BV, and Staysail Systems, Inc.
* fixes #143 Protocols and transports should be "configurable"Garrett D'Amore2017-11-02
| | | | | | | | | | | | | | | | | | | | This makes all the protocols and transports optional. All of them except ZeroTier are enabled by default, but you can now disable them (remove from the build) with cmake options. The test suite is modified so that tests still run as much as they can, but skip over things caused by missing functionality from the library (due to configuration). Further, the constant definitions and prototypes for functions that are specific to transports or protocols are moved into appropriate headers, which should be included directly by applications wishing to use these. We have also added and improved documentation -- all of the transports are documented, and several more man pages for protocols have been added. (Req/Rep and Surveyor are still missing.)
* Various man page fixups, add inproc header.Garrett D'Amore2017-10-24
| | | | | | | We added nng_inproc man page, and a header with an explicit registration method for it. We also fixed up the markup on several other pages, and corrected the include statements for nng_zerotier.
* fixes #120 nng_sockaddr options initialized with garbage dataGarrett D'Amore2017-10-19
|
* Refactor option handling APIs.Garrett D'Amore2017-09-27
| | | | | | | | | | | | This makes the APIs use string keys, and largely eliminates the use of integer option IDs altogether. The underlying registration for options is also now a bit richer, letting protcols and transports declare the actual options they use, rather than calling down into each entry point carte blanche and relying on ENOTSUP. This code may not be as fast as the integers was, but it is more intuitive, easier to extend, and is not on any hot code paths. (If you're diddling options on a hot code path you're doing something wrong.)
* Fix a few warnings and subtle test races.Garrett D'Amore2017-08-24
|
* Eliminate legacy option settings, provide easier option IDs.Garrett D'Amore2017-08-24
| | | | | | | | | | | | | | | | | | This eliminates all the old #define's or enum values, making all option IDs now totally dynamic, and providing well-known string values for well-behaved applications. We have added tests of some of these options, including lookups, and so forth. We have also fixed a few problems; including at least one crasher bug when the timeouts on reconnect were zero. Protocol specific options are now handled in the protocol. We will be moving the initialization for a few of those well known entities to the protocol startup code, following the PAIRv1 pattern, later. Applications must therefore not depend on the value of the integer IDs, at least until the application has opened a socket of the appropriate type.
* The common endpoint code already ensures state is proper.Garrett D'Amore2017-08-23
| | | | | | | This eliminates tests for code that we cannot reach, because the upper layer endpoint code already ensures that we don't get called if we are closing, that the mode is correct, and that only one outstanding endpoint operation is in progress on any given endpoint.
* fixes #39 Transport ops vector should be versionedGarrett D'Amore2017-08-17
| | | | | | This also includes tests for some of the edge cases surrounding pluggable transports, such as version mismatch, duplication registration, and failure to initialize.
* Provide versions of mutex, condvar, and aio init that never fail.Garrett D'Amore2017-08-16
| | | | | | | | | | | | | | | | | | | | | | | If the underlying platform fails (FreeBSD is the only one I'm aware of that does this!), we use a global lock or condition variable instead. This means that our lock initializers never ever fail. Probably we could eliminate most of this for Linux and Darwin, since on those platforms, mutex and condvar initialization reasonably never fails. Initial benchmarks show little difference either way -- so we can revisit (optimize) later. This removes a lot of otherwise untested code in error cases and so forth, improving coverage and resilience in the face of allocation failures. Platforms other than POSIX should follow a similar pattern if they need this. (VxWorks, I'm thinking of you.) Most sane platforms won't have an issue here, since normally these initializations do not need to allocate memory. (Reportedly, even FreeBSD has plans to "fix" this in libthr2.) While here, some bugs were fixed in initialization & teardown. The fallback code is properly tested with dedicated test cases.
* fixes #63 NNG_FLAG_SYNCH should be the defaultGarrett D'Amore2017-08-14
| | | | | Also enables creating endpoints that are idle (first part of endpoint options API) and shutting down endpoints.
* Unify the msg API.Garrett D'Amore2017-08-10
| | | | | | | | | | | | | This makes the operations that work on headers start with nni_msg_header or nng_msg_header. It also renames _trunc to _chop (same strlen as _trim), and renames prepend to insert. We add a shorthand for clearing message content, and make better use of the endian safe 32-bit accessors too. This also fixes a bug in inserting large headers into messages. A test suite for message handling is included.
* Refactor AIO logic to close numerous races and reduce complexity.Garrett D'Amore2017-08-04
| | | | | | | | | This passes valgrind 100% clean for both helgrind and deep leak checks. This represents a complete rethink of how the AIOs work, and much simpler synchronization; the provider API is a bit simpler to boot, as a number of failure modes have been simply eliminated. While here a few other minor bugs were squashed.
* Fix close-related leak of pipes.Garrett D'Amore2017-07-18
| | | | | | | | | | | We have seen leaks of pipes causing test failures (e.g. the Windows IPC test) due to EADDRINUSE. This was caused by a case where we failed to pass the pipe up because the AIO had already been canceled, and we didn't realize that we had oprhaned the pipe. The fix is to add a return value to nni_aio_finish, and verify that we did finish properly, or if we did not then we must free the pipe ourself. (The zero return from nni_aio_finish indicates that it accepts ownership of resources passed via the aio.)
* Remove unused variables.Garrett D'Amore2017-07-17
|
* Inproc leak fixes.Garrett D'Amore2017-07-16
|
* Add missing cancellation for inproc endpoints -- the source of much woe.Garrett D'Amore2017-07-15
| | | | | | | | | Most of the races around close were probably here - the cancellation was not getting through on endpoint close, which meant that we could actually toss endpoints while they were in use. We need to fix the timeouts stuff -- especially for reconnects etc, but we are just about ready for this stuff to be reintegrated into master.
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-01-18 14:07:50 +0000