Prevention of Data Loss:
To prevent data loss, the NSA library caches any information that hasn't been sent in the case of network interrupt, shutdown, or similar circumstances. The cache file is circular with a maximum size equal to the value you stamped into the library. Once the maximum size has been reached, the cache file wraps around and begins overwriting the oldest data. When a successful connection to Nalpeiron is detected, the entire cache will be automatically sent.
The NSA library is threaded. Major threads include the communication thread pool, the caching thread with queue and the logging thread with queue. All access to the library MUST be started with a call to NalpLibOpen and ended with a call to NalpLibClose. NalpLibOpen creates the threads and performs some general housekeeping. NalpLibClose issues shutdown orders and waits for the threads to complete what they are doing. The shutdown should not take long. The communication thread will complete any ongoing transaction then pass off the rest of its jobs to the caching thread. The caching thread will write out all the pending jobs to cache and then shut down. Closing your program without issuing a NalpLibClose can result in data loss.
NalpLibClose is intended to be called just before the NSA library is closed. NalpLibClose is used to shut down the library's threads and cache any data that has not yet been sent to the NSA servers. As NalpLibClose does not block, it is possible to lose some data during shutdown if precautions are not taken.
The most likely scenario for data loss would be a very short-lived application. That is, an application whose run time was on the order of a few seconds. A sleep command just after NalpLibClose will prevent any data loss. When NSA is started with NalpLibOpen, it initializes its threads, a transaction thread and a cache thread among others, and immediately sends a request for location data to Nalpeiron. The location request is to ensure geographical data is current during data transmission so that local privacy laws may be followed.
Until the response to the location request reaches the library, all transactions are cached rather than sent to Nalpeiron.
The general progression of a transaction through the library is as follows:
When a data transaction is initiated, the data is first placed in the transaction thread pool to be sent to Nalpeiron's server. If location data is missing or some error occurs (lack of connectivity, for instance), the data is moved to the cache thread where it will eventually be written to disk.
During NalpLibClose, all data remaining in the transaction thread pool is moved into the cache thread and written to disk as soon as possible. NalpLibClose does not block. That is, when called it returns immediately. The decision was made that all NSA library functions should return immediately while necessary processing takes place in a thread in the background.
This means that under certain circumstances, a short-lived application, for instance, there is the possibility of data loss. This may occur if the library itself is closed (with a dlclose or similar command) before NalpLibClose actually completes.
For example, suppose your application opens the NSA library, calls NalpLibOpen, does a login, an app start, a feature start, a feature end, application end, a logout and then closes the library with NalpLibClose.
Depending on connectivity speeds, on the call to NalpLibClose there may be a total of 6 transactions pending with an ongoing request for location data being processed. When the NalpLibClose is called, it will move the six pending transactions to the cache thread and shut down the transaction thread. The cache thread will write the six pending transactions out to cache where they will be available to send to Nalpeiron on the next run of your application.
If the library itself is closed before this process is complete, you may lose some of these pending transactions. The simplest way to prevent data loss is to run NSA in offline mode.
Offline Mode is initiated by including:
in the XML string sent to NalpLibOpen. While in offline mode no data is sent to or requested from the Nalpeiron server. All data is written to the cache file. If you choose to use offline mode remember that the data collected in the cache will need to be sent to Nalpeiron via the NSASendCache command at some future time.
If you do not wish to use the offline mode, you may be able to prevent data loss with a short pause (sleep, for example) after calling NalpLibClose. Additionally, increasing the number of threads available to the network pool (with soapthreadsmax) or the length of the cache thread (with cacheqlen) may help. You can measure the effect of these parameters on your performance with calls to NSAGetStats.