Lesson 7: Data logging application

This tutorial will discuss a fairly complete application for remote data logging and collection, called sense_and_log. This application builds on components from previous lessons: BCAST, SENSE and COMMAND. This tutorial also discusses the LOGGER component, and shows a simple multihop data propagation from the sensors to a central location.

Functionality

• START_SENSING -- this command invokes the SENSE component to collect a specified number of samples at a specified sampling rate and to store these samples in the EEPROM.
• READ_LOG -- this command will retrieve a block of data from the EEPROM and report it over the multihop network to the base station. The requests flood the network, whereas the reply travels along a well defined path. This also causes a more complex interaction with the BCAST component -- if a node detects that it should act on the message, it squashes the local re-broadcast.

Sensing

This component builds on the SENSE component from Lesson 2 and OSCOPE component from Lesson 5. In order to start sensing, an upper layer needs to invoke the START method, and specify the number of samples to be taken, as well as the rate at which they should be taken.

void TOS_EVENT(SENSE_CLOCK_EVENT)(){
    VAR(nsamples)--;
    if (VAR(nsamples)== 0) {
	// Stop the clock. 
	TOS_CALL_COMMAND(SENSE_CLOCK_INIT)(0, 255);
    }
    TOS_CALL_COMMAND(SENSE_LEDr_toggle)();
    TOS_CALL_COMMAND(SENSE_GET_DATA)();
}

Logger component, interfaces, usage, and limitations

• char APPEND_LOG(char* data); -- sequential write access, useful for storing sensor data or other in-order streams. Low regions of the EEPROM cannot be accessed with APPEND_LOG, which is useful in certain applications
• char WRITE_LOG(int line, char* data); -- random access write to the EEPROM, very useful when the size and structure of the object stored is known in advance. This functionality is very useful when it comes to network reprogramming.
• char READ_LOG(int line, char* dest); -- random access read, the component using the log passes in a buffer to be written.

All of the above operations are split phase - APPEND_LOG and WRITE_LOG will signal a APPEND_LOG_DONE(char success); upon completion, and READ_LOG will signal READ_LOG_DONE(char* packet, char success);. Only one of these operations can be performed at a time, if there is an operation is progress, the invocation of any of these commands will fail. Applications using the LOGGER must take care to either ensure a full serialization of operations or check the command return code and retry periodically.

You'll note that both READ_LOG and WRITE_LOG will address the underlying EEPROM in units of lines. Line is analogous to a block number on a harddrive -- it addresses the EEPROM not in the units of bytes but rather in the units of LOG_ENTRIES. With the 16 LOG_ENTRIES, the log will store 2048 log lines

Note on internal storage organization: The APPEND_LOG command from this release of LOGGER will not write any data to lines 0 through 3. This region of EEPROM has been set aside to store data persistent to the mote. If you're using network reprogramming components, this region will hold the TOS_LOCAL_ADDRESS for the mote, which will make it persistent across reprogramming. Also when using network programming, you may wish to set aside a larger buffer for the downloaded code, or otherwise gurantee that logging of other data will not interfere with downloaded code.

Note on tossim: At this point there is no functionality of logger in the tossim, so applications using the logger cannot be reliably simulated under tossim.

Logging performance

Another issue that comes up quite often is the performance of the logger. Read performance is proportional to the amount of data being read, and is fast enough not to be a problem in most TinyOS applications. The write speed is a different issue. Since the write speed is slower than the maximum data generation of the mote, it is the logger component that effectively limits the capture rate. The write speed is proportional to the amount of data being written, but after every log entry write there is a pause of about 5 ms. During that pause, the EEPROM transfers the data from an internal RAM buffer into nonvolatile storage. During that time, the EEPROM is completely nonresponsive. It is that internal EEPROM buffer that limits the maximum size for EEPROM log entries to 64 bytes. Moving to larger EEPROM entries will have the effect of increasing the throughput of the EEPROM. For the LOGGER in the release, we chose log entries of 16 bytes, as a reasonable compromise between logging performance, buffering requirements, and interactions between the logging components and the messaging subsystem.

As a rule of thumb, you should assume that writing a log entry to the EEPROM will take about 10 ms. This implies the logging throughput at 1600 bytes per second. If you're trying to use the EEPROM to log data from an analog sensor attached to the mote, then you can log data at up to 800 samples per second. When you exploit the fact that the ADC on the mote only produces 10 bits of data per sample, then you can log up to 1280 samples per second. At the maximum throughput, the mote is able to record about 20 seconds worth of data.

Multihop data collection

// Log message structure
typedef struct {
    short source; 
    short address;
    char log[16];
    unsigned hop_count;
} logmsg_t;

#define TOS_FRAME_TYPE COMMAND_obj_frame
TOS_FRAME_BEGIN(COMMAND_obj_frame) {
    TOS_MsgPtr msg;
    TOS_Msg log_msg;
    short parent;
    char send_pending;
    char pending;
}
TOS_FRAME_END(COMMAND_obj_frame);

...
TOS_TASK(eval_cmd) {
    cmdmsg_t * cmd = (cmdmsg_t *) VAR(msg)->data;
    char status = BCAST_FORWARD;
    // do local packet modifications: update the hop count and packet source
    cmd->hop_count++;
    VAR(parent) = cmd->source;
    cmd->source = TOS_LOCAL_ADDRESS;

    switch (cmd->action) {

    case READ_LOG:
	//Check if the message is meant for us, if so issue a split phase call
	//to the logger
	if ((cmd->args.rl_args.address == TOS_LOCAL_ADDRESS) &&
	    (VAR(send_pending) == 0)) {
	    if (TOS_CALL_COMMAND(COMMAND_READ_LOG)
		(cmd->args.rl_args.logLine, 
		 ((logmsg_t*) VAR(log_msg).data)->log)) {
		VAR(pending) ++;
	    }
	    status = BCAST_PRUNE;
	}
	break;
    }

....

}

// The log has completed the reading, and now we're ready to send out this
// message. Note a potential problem: what is send_pending set to? 

char TOS_EVENT(COMMAND_READ_LOG_DONE)(char* packet, char success) {
    if (success) {
	VAR(pending) = 0;
	VAR(send_pending) = TOS_CALL_COMMAND(COMMAND_SUB_SEND_MSG)
	    (VAR(parent), AM_MSG(LOG_MSG), &VAR(log_msg));
    }
    return 1;
}

// Routing handler for the LOG_MSG. The default handler is very simple: we
// copy the data out of the packet into the log buffer, and forward it to the
// parent. Exercise to the reader: what are the rece conditions in routing and
// how can they be fixed? 

TOS_MsgPtr TOS_MSG_EVENT(LOG_MSG)(TOS_MsgPtr msg) {
    char i;
    char *ptr1, *ptr2;
    if (VAR(send_pending) == 0) {
	VAR(send_pending) = 1;
	ptr1 = (char *) &(VAR(log_msg));
	ptr2 = (char *) msg;
	for (i =0; i < defaultMsgSize(msg); i++) {
	    *ptr1++ = *ptr2++;
	}
	if (TOS_CALL_COMMAND(COMMAND_SUB_SEND_MSG)
	    (VAR(parent), AM_MSG(LOG_MSG), &VAR(log_msg)) == 0) {
	    VAR(send_pending) = 0;
	}

    }
    return msg;
}

When any broadcast request is received, each node stores the ID of the node that forwarded that request. When a node composes the log message, it forwards it just to the parent node (unlike broadcast, which sends out the message to the entire neighborhood). The parent of the originating mote forwards such messages to its parent, until the message reaches the source of the request flood.

Mote IDs fullfill two important roles: they help us identify a specific data stream, and they are used to identify a specific path through the network. There are schemes, like directed diffusion, which do not use the mote IDs for identifying network paths; looking at the TOS_MSG_EVENT(LOG_MSG), we could have used a schemes which are more in the spirit of diffusion, e.g. the reply could have been flooded back to the requester.

Also note that the buffer management in the LOG_MSG handler is a bit different from what you've seen so far: rather than playing the pointer swapping game (like BCAST), the message being forwarded is copied to the local buffer, and the original buffer is returned to the communication stack.

PC-side tools

Exercise

java net.tinyos.tools.BcastInject groupID START_SENSING 128 4 128

 
java net.tinyos.tools.BcastInject groupID READ_LOG moteID 4