Sensor Network Data Storage & Forward Schema

Sensor Network Data Storage & Forward Schema

The sensor network uses a store and forward message passing scheme.

·         A sensor takes a reading and stores it.

·         The reading is forwarded to another device such as a repeater or a receiver.

·         The sensor will repeatedly forward the same reading until it until it receives an acknowledgment from the up-stream device that the reading has been received.

·         At this point, the sensor deletes the reading from its own memory.

·         The sensors use non-volatile flash memory for safe storage.

In this way, if a sensor transmits to an access point, and receives an acknowledgment, the access point is assumed to now have the reading stored. The access point will continue to store the reading until it gets an acknowledgement from the temperature service, which runs on the server.

The temperature service is the ultimate source of acknowledgments. When a reading is received by the temperature service, several events take place, in this order:

1.      The reading is saved to disk

2.      The data is placed in a buffer for subsequent database storage.

3.      The acknowledgement is sent.

The actual database store is done by another thread and runs asynchronously with respect to these three events. Therefore, the acknowledgement can be completed before any database access is attempted because not all temperature readings are saved in the database.

Many temperature readings can be transmitted more than once if the sensor does not immediately receive an acknowledgement. The temperature service actively filters duplicate readings, preventing them from reaching the database. Additionally, some readings that cannot be stored, such as readings from a sensor whose ID is not entered in the software program. Readings of this form are dropped.

Nominally, the database writes or one of the filters activates within milliseconds of receipt. If an exception occurs, the service is designed to buffer a very large number of temperature readings in memory as necessary. This feature is intended to support automatic recovery from catastrophic database failures.

Even in this event (or if the server lost power while buffering a large amount of data), temperature readings are not irrevocably lost.  Recall that the first step, upon receipt, is to save readings to disk. The last line of recovery is to process the disk backup file into the database. We have a utility program with the Checkpoint software that will move readings for a given time range from the service's backup file into the database.