aio: separate stop / shutdown from fini (deallocate)

Probably other subsystems should get the same treatment.  We need
to basically start the process of shutting down so that subsystems
know to cease operation before we rip memory out from underneath them.

This ensures that no new operations can be started as well, once we
have begun the process of teardown.

We also enhanced the completion of sleep to avoid some extra locking
contention, since the expiration *is* the completion.

Includes a test for this case.

1 files changed, 4 insertions, 2 deletions

diff --git a/src/core/init.c b/src/core/init.c
index 6672beb1..fa07919e 100644
--- a/src/core/init.c
+++ b/src/core/init.c
@@ -133,14 +133,16 @@ nng_fini(void)
 		nni_atomic_flag_reset(&init_busy);
 		return;
 	}
+	nni_aio_sys_stop(); // no more scheduling allowed!
 	nni_sock_closeall();
 	nni_sp_tran_sys_fini();
 	nni_tls_sys_fini();
 	nni_reap_drain();
-	nni_aio_sys_fini();
 	nni_taskq_sys_fini();
-	nni_reap_sys_fini(); // must be before timer and aio (expire)
+	nni_reap_drain();
+	nni_aio_sys_fini();
 	nni_id_map_sys_fini();
+	nni_reap_sys_fini(); // must be near the end
 	nni_plat_fini();
 	nni_atomic_flag_reset(&init_busy);
 }