Projects‎ > ‎Previous Projects‎ > ‎

eSense System as Distributed Database for TinyDB Mobile Software

by:
B. Ariston Darmayuda
ELECTRICAL ENGINEERING OF UDAYANA UNIVERSITY

Abstract

Microtechnology in the last decade has not been mere science fiction. Microtechnology is what several scientist aim for in order to close the computation room in the case of taking some environmental data so that can take more detailed data.

Through the efforts of microtechnology, a company named Crossbow has focused their attention onto the usage of microtechnology in collecting environmental data, such as temperature, humidity, light, shock etc. This company has not only developed microtechnology but also technology that has been the aim of other scientists. The technology that they have developed is adapted from the Smart Dust system, which is a tiny sensor system (similar in size to a 500 rupiah coin) and have some advanced features.

Some problems came with this system, which is that they did not expand the Backend system. Crossbow focuses their research on only the hardware. As for the Backend software, they disposed of it after "software development environment" (not sure what this is supposed to mean. Sorry -- Koko). Within that limit, the scientist develops the system such that it can adapt with Crossbow hardware and also run at micro computation system, which in this case is Handled PC or known more widely as the Pocket PC (PDA).

Some purposes that come with the research are suggesting software developer at Indonesia indeed Bali to watch out for software development at micro computer in this case Handled PC or on their kind. The reason is now the world unceasing system development at this kind of computation system. No doubtful that Desktop PC computation system next day will be replace by micro computer. The other purpose is this system will be the platform while developing the Smart Dust technology on campus environment so that the next research does not need to build the communication system but implementing the system.

The benefit that given by this research is supply crossing line of world technology expansion into campus environment focusing at Electrical Engineering Faculty of Udayana University so that the student not only developing system that focusing on monotonous system but try to face the technology expansion that happen in this world.

Acknowledgement

First I like to say thank you to God for the gift so I can finish the system that I develop, the eSense Software, because without the gift I cannot finish the system. And I also want to say some thank you for:

My parents that support me with nutrient food at home, so I can finish my project with full health.

My beloved person that support me day and night, with her support I can full concentrate finish the software.

Dr. Ken Taylor that allows me to borrow the devices those need to develop the software and also give me the idea to develop this project so the system can achieve the main purposes.

Ir. Dayu Giriantari as the prime minister of the faculty that I studied that give me chance to finish the project and establish good relationship between Udayana University and UrRemote.

My friends, Joni, Adi Dharmadi, Tutnik, Luki, Eka Parama Darmika, and so on, that support me in many things include give new knowledge and also submit good final project title.

End User License Agreement (EULA)

SOFTWARE WORK OUT AS RESEARCH SOFTWARE PLATFORM AND THEN SOME PARTS OR ALL OF SOFTWARE IS UNDER GPL LICENSE (GNU PUBLIC LICENSE). SOME PARTS OR ALLL OF SOFTWARE CAN BE USED BY ANYONE AS RESEARCH PURPOSE WITH NO CHARGE. USING THE WHOLE SOFTWARE AS PAY SOFTWARE WILL BE CHARGE FROM PREVIOUS DEVELOPER AND VIOLENT THE RULE OF GNU PUBLIC LICENSE.

SOFTWARE CAN BE USED IN THREAD SAFE BECAUSE NOT USING POINTER OR ACCESSING RESTRICTED RESOURCE OF POCKET PC. HARDWARE DAMAGE IS OUTSIDE OF SOFTWARE CONTEXT AND DEVELOPER HAS NO RESPONSIBILITY. TO AVOID THE MISTAKES AND HARDWARE DAMAGE THE USER MUST ACKNOWLEDGE THE DOCUMENTATION AND HOW TO RUN THE SYSTEM.

FAILURE FROM THIRD PARTY, LIKE FAILURE IN DATABASE SYNCHRONIZATION BECAUSE GSM, OR FAILURE TO DO COMMUNICATION WITH MOTES BECAUSE BLUETOOTH DEVICE DAMAGE IS NOT DEVELOPER RESPONSIBILITY. YOU MUST CONTACT THE THIRD PARTY TO GET THE CONCLUSIONS.

ONE OR SOME GROUP OF PERSON THAT DEVELOPING THE SYSTEM FURTHERE HAS DUTY TO EMBEDING THE NAME OF PREVIOUS DEVELOPER AS THE DEVELOPER DOES IT NOW.

Background

The developments of Wireless Sensor Network devices at few decades have go stronger than we though. The device now more advance than before. Bet the advance of the device not also follow by the software development. As we know the TinyDB software work as the best Wireless Sensor Network software. But the problem is the software only work on Desktop PC.

We must considering this problem, because the next generation PC will not focusing on Desktop PC again. We can take a look on mobile device development. Many vendor of mobile device try to increase the device features. The processing speed is nearly same like Desktop PC. The memory support is now getting large. Operating System supports more for our work, like we can do offices job on mobile device. By these cases we must consider that the next generation of PC will be mobile PC.

With this case the research is takes out. I try to do migration of the TinyDB software from Desktop PC then can run at Pocket PC as first development. To do the research I use some tools that need to finish the project (that will explain on next section). With the tools I hope I can success do migration of the TinyDB software can run on Pocket PC device without change the TinyDB context.

Problems

As we know that the Pocket PC is nearly not same like Desktop PC. The Pocket PC has some weakness on case like multi threading. Multi threading is one feature of Desktop PC processor. The Pocket PC only supports single threading process so we cannot just translate the TinyDB sources into Pocket PC software.

So the problem that I will face will be how to translate the TinyDB sources into Pocket PC software without change the structure and easy to understand for users.

Next problem is how easy to maintenance the sense data over Pocket PC, in this case I will use software from Microsoft called Microsoft SQL Server Mobile 2005 that can do synchronization with Microsoft SQL Server 2005 over internet protocol.

Purpose and Advantages

Some purposes and advantages that can achieve by the project are:

  1. Pointing the TinyDB system more into the main purpose of Smart Dust.
  2. Easy to maintenance and synchronizing data using SQL Server Edition.
  3. Introduce the Smart Dust and Wireless Sensor Network into Udayana University environment.
  4. Supply class library for the next development of TinyDB on mobile devices.

System Design

At now I choose two kinds of system designing diagrams, those are Data Flow Diagram (DFD) and Semi UML. I choose to combine both of diagrams because every diagram has weakness and can be full fill by other diagram to get better explanation of the system.

Data Flow Diagram

I will not explain the context diagram of DFD because it’s too common.

DFD Process 1


DFD Process 1

With the diagram I can give explanation likes:

  • First user needs to supply the login data, this process prevent unauthenticated user to use the software and save invalid data for valid user. The login data will be the valid user name and password.
  • Second is if the login data is invalid then the process continuing into terminating application. If success then the authentication flag will be send into second and third process.

DFD Process 2

DFD Process 2

With the diagram I can give explanation likes:

  • After authentication flag received from first process then the second process start building communicator. After created then the communicator will be on standby mode. In this case client/user still needs to interact with system on sending query.
  • After communicator is done, user now preparing to build query depend on data that want to get. Building queries same as building query at TinyDB software on Desktop PC. After user has decided the query, process parses the query into byte array buffer. Then continue to send it out. At process 2.2, the process also saves the query for data validation on receiving buffer. If lose the query then the buffer that received from the motes will become invalid.
  • Now is receiving state. After the motes execute the query then starting collecting sense data and send it back into Pocket PC. The process that responsibility with this action is process 2.3 and the buffer parsing process will be process 2.4. At process 2.4 for parsing the sense data buffer needs the query that have been saved previously before send action. Using the valid query the sense data can be extracted into tuples data and save into database using process 3.

DFD Process 3

DFD Process 3

With the diagram I can give explanation likes:

  • In this process user only interact for requesting received data and do synchronization if synchronization schedule on demand.
  • Continue to the process, after receiving authentication flag from first process then this process will building the dumper and synchronization engine. The dumper will be used as sense data dumping into database, and synchronization engine will be use on synchronizing database of mobile server into database backend. After the dumper and synchronization built, those objects will be on standby state.
  • From process 2 this process will receive extracted sense data tuples that ready to save into database. Before saving the tuples, tuples need to be separated by index. After separated then the tuples ready to save into database.
  • For synchronization process (process 3.4) user need to give command if synchronization schedule on demand. If not then the process will check if current time must do synchronization. The details of synchronization processes not describe on this section because it already handle by Microsoft SQL Server Mobile 2005, you just need to initialize the replication object and make sure at database backend has been setup correctly, and also the connection on both side (Pocket PC and Database Backend) must online.

Activity Diagram

The activity diagram on this system design used to explain more details on each process that cannot covered by flow diagram of DFD. DFD use Flow Chart as flow diagram that definitely not supports explanation of parallel process. That’s why I choose activity diagram of UML to explain the activities. And for UML cannot explain the data flow so I choose the DFD to explain the data flow of my system.

Building Query Activities



Building Query Activities

From the diagram I can explain likes:

  • First we must check the parameters that given by user. If it’s a Query SQL then we need to parse it into sense attributes array. If already the sense attributes array then we go into simple process check the attributes on sense attributes array. If one or more attributes are invalid then throw an exception.
  • If all attributes are valid then continue by building query object. After query ready these also create a Query Message Packet that used to save the attributes into.
  • If all preparation ready, then the activity will continue saving the attributes into query and packet. After that we have a query that can send into translator.

Sense Data Translation Activities

Translation Activities

From the diagram I can explain likes:

  • At these activities we check if user needs to activate or shut down. If shut down then we kill the communication Bridger and clear the buffer.
  • If starting then first we activate the Bridger, if Bridger active then continue into next activities.
  • There are three activities that run on separated pipeline (parallel activities), that why the process separated by fork.
  • First activity is sending query. Query must supply by user on previous activity then reads by translator. The translators are parsing the query into buffer and then send it through Bridger.
  • Second activity is when Bridger reading a buffer. The buffer from Bridger parsed into packet then saved into packet collection. This activity done at Bridger event handler.
  • The last one is activity of parsing Query Result Message Packet into tuples. First the activity will read the first packet on queues. If queue empty loop back again. If exist then compare the packet query ID with query on list (note: that’s why I say save the query). If query ID of packet is registered then extract the attributes on query to get the valid attributes data from packet. After attributes data extracted save it into extracted tuples. Then the last one is pass the tuples into translator event handler.

Building Communicator Activities

Building Communicator Activities

From the diagram I can explain likes:

  • First check the state is it shut down or activate. If shut down then we kill the Bluetooth object. If activate then we create the Bluetooth by giving the Bluetooth parameters that can find on documentation chm. If fail creating Bluetooth then throw an exception. If success then do separated processes.
  • First process is sending query. Query must already parse from translator or packet. After query on byte array structure then do escaping the query buffer. After escaped then the query is ready to send through the Bluetooth.
  • Second is receiving the buffer after query execution on motes. Activity starting by read byte-per-byte. Bytes read until reach the sync flag (0x7Eh). Then read next byte, if it sync flag then we will acknowledge the buffer as packet buffer then do checking state (check size and crc). If next byte is not sync flag then save it into buffer list, do it again until reach another sync byte.

Saving and Synchronization Activities

Synchronization Activities

From the diagram I can explain likes:

  • Same as previous activities, first check the state is it shut down or activating. If shut down then clear all tuples on queue. If starting then continue into next activities.
  • In this activities also do separated activities that run on parallel state.
  • First is receiving tuples from translator, this done at translator event handler.
  • Second is reading synchronization signal from user. If user needs to synchronize the database then pulse the signal handler into true.
  • Last one is synchronization and saving tuples into database. First check the synchronization type. If on demand then check the synchronization signal handler if it’s true then do synchronization, if false then wait. If synchronization type is scheduled then do separation again, first check the synchronization signal (true or false), second is check the time differences (equal or greater than synchronization schedule). If first condition is true or second condition is true then do synchronization. If all false the pass synchronization process. After synchronization process check the delete after sync flag, if it true then delete all data on mobile device. After synchronization state then continue on saving the tuples on queue into database.

Classes

There are some classes that very important to acknowledge. The other classes will be explaining on documentation that includes on source library. The classes are:

SenseCatalog Class

SenseCatalog

The SenseCatalog class provides information about the currently supported sense attributes and sense aggregates. Using this class to build the query is most recommended way. If building without pointing the sense attributes and aggregates from this class may create invalid and unreadable sense data.

SenseAttributes Class

SenseAttributes

The SenseAttributes class provides single information about mote senses. With this class you can get detailed information about a sense. This class also provide method that used to format and crop the sense data that received by a query.

SenseAggregates Class

SenseAggregates

The SenseAggregates class provides information about a single sense aggregate. At current development now only support NOP aggregate.

Query Class

Query

The Query class provides information about the query that will send into motes. Use this class to build a TinyDB query by supply the SQL string or sense attributes collection. Object from this class is need by the translator to send the query. This class also provides a walkthrough how to separate the tuple value for corresponding query.

Commands Class

Commands

The Commands class provides information of mote commands. At now the command that currently supported is ResetMote command.

Bluetooth Class

Bluetooth

The Bluetooth class provides Connection Bridge between the Pocket PC and the Motes Gateway. With using BL521 Bluetooth Adapter we can provide Bluetooth line between MIB510 Motes Gateway and the Pocket PC. Pocket PC at now only have three way of serial communication, first is Bluetooth, second is IrDA, and the last one is USB.

To build the Bluetooth object we must consider the connection types, if it is direct then we need to supply the BL521 pair name and password, if manual using serial line then we must provide the incoming and outgoing COM port that we setup previous on control panel. To get valid COM port name, call the CreatePort method by supply the port number.

The Bluetooth object is need by the Bridger class.

Bridger Class

Bridger

The Bridger class provides the basic way to communicate with the motes. The Bridger only needs a byte array buffer to send through Bluetooth into motes gateway. NOTE: The byte array buffer must structure correctly otherwise the motes gateway will not recognize the packet.

This class also creates nested class that provides the walkthrough how to escape the byte array buffer.

Translator Class

Translator

The Translator class provides worker to translate the packet that come and go into or from motes gateway. This class needs a Bridger to start working, if Bridger is empty then when start working will throw an exception.

Dumper Class

Dumper

The Dumper class provide worker to save the tuples and do synchronization database between mobile database and database backend. On both sides must supply replicated table structure like this:

Replicated table

If anyone considering this class not efficient for their organization can change or not using this class library assembly. It’s up to your choice.

How To Execute Single Query

This section will explain the single query execution sample. By this sample you can create your own system by using the class library.

  • First step is including the namespace.
In C# you need to type syntax below to include the class library:
using eSense.protocols;
using eSense.protocols.communicators;
using eSense.protocols.parser;
using eSense.protocols.parser.commands;
using eSense.protocols.workers;
  • Put the workers as global private variables.
Do this by typing C# syntax at form private variables place:
// Put the workers as global private variables.
private Bridger bridger = null;
private Translator translator = null;

  • Create the Query object.
    To create the object first you need to define the query SQL, for this sample lets create query of SELECT (nodeid,temperature), after define the query construct Query object by using this syntax on C#:
// Create the query.
Query qry = new Query("SELECT (nodeid,temperature)");
qry.Address = Consts.ADDR_BROADCAST;
qry.EpochDuration = Consts.DEFAULT_EPOCH;
qry.GroupID = Consts.DEFAULT_GROUP_ID;

  • Create the Bluetooth object.
In this case we use the serial communication type, so first you must pair manually the BL521 with the Pocket PC and define the COM port, and for example you use COM5 as both sides communication. Then you can create Bluetooth object using syntax on C# like:
// Create Bluetooth.
Bluetooth bth = new Bluetooth(BluetoothTypes.Serial, "COM5", "COM5");

If you not sure how to define the port name you can write syntax like this:
// Create Bluetooth.
string inPort = Bluetooth.CreatePort(5);
string outPort = Bluetooth.CreatePort(5);
Bluetooth bth = new Bluetooth(BluetoothTypes.Serial, inPort, outPort);


  • A next step is creating the Bridger as Communication Bridge.
First create the Commands manager, the Bridger needs this object to send the reset command into motes so the motes on standby mode.
// Create Commands class.
Commands cmd = new Commands(Consts.ADDR_BROADCAST, Consts.DEFAULT_GROUP_ID);
Then create the Bridger object.
// Create Bridger.
this.bridger = new Bridger(bth);
this.bridger.CommandSource = cmd;


  • Creating the Translator object.
The Translator used to send and formatting received buffer from motes. If you use this class, then you create object that can control all process on sending and receiving buffer into or from motes gateway.
// Create Translator.
this.translator = new Translator();
this.translator.Bridger = this.bridger;
this.translator.WorkDumping += new WorkerDump_EventHandler(this.TranslatorDumping);
The Translator will invoke an event when dumping the translating process, that’s why you must define a method for it.
private void TranslatorDumping(object dumped)
{
   ExtractedTuple[] tuples = (ExtractedTuple[])dumped;
   // For .NET Compact Framework, other thread cannot modify main thread you need to invoke.
   if (this.InvokeRequired)
   {
     this.Invoke(new DrawExtractedTuple_Invoke(this.DrawExtractedTuple), tuples);
   }
   else
   {
     this.DrawExtractedTuple(tuples);
   }
}
Just to remember the .NET Compact Framework 2 disallows other thread to modify main thread, also vice versa. So you need to create delegator to modify or show tuples into main form.
// Delegate for drawing tuples on form.
private delegate void DrawExtractedTuple_Invoke(ExtractedTuple[] tuples);
private void DrawExtractedTuple(ExtractedTuple[] tuples)
{
   string format = "Attribute:{0} - Value:{1} - TimeStamp: {3}";
   for (int i = 0; i < tuples.Length; i++)
   {
     // Get the attribute from catalog.
     SenseAttributes attr = Commons.Catalog.GetAttribute(tuples[i].AttributeID);
     object value = attr.GetFormattedValue(tuples[i].Value);
     // Draw the results.
     this.results.Items.Add(string.Format(format, attr.FieldName, value, tuples[i].TimeStamp));
   }
}
  • Now you ready to send the query.
    To send the query first you need to activate the Translator, activating the translator also activating Bridger, and Bridger will activating the Bluetooth. After it’s active then you ready to send the query and wait for the sense data.
// Activate the Translator and send Query.
this.translator.IsActive = true;
this.translator.SendQuery(qry);
// Now wait for the data.

eSense GUI

The eSense itself like TinyDB give an interface for you so you easy to get the data. Remember to create the database backend and mobile if you choose to use the database feature.

eSense Login

You need to login to access the sense data. This will prevent unauthorized person to gathering data with your middleware.

eSense Connection Manager

Next is the main screen, at this main screen you will be navigating the sequence of collecting sense data. First you need to create the communication. Select serial port that you use then press Activate.

eSense Query Builder

Next tab used to construct the query, make sure you have been activating the connection. To construct a query first select the attribute you want to choose then press Add. After you finish for the attributes, then setup the group ID and epoch duration. After your query completely ready then press Run. After pressing Run you will be ask to save the query so you can run it next time.

eSense Live! Preview

eSense Logs

eSense Database

These three screens are user live preview. At first screen you can see the live sense data preview that currently received. Next is a current worker log. And the last one is database viewer.

eSense Quick Access

eSense QuickAccess! Menu

The menu is very important to understand especially the menu on Figure 23, the Quick Access menu give you accessibility to use database. If you not setup database backend make sure you uncheck the menu Use DB. Important note, for login name and password you can define it on file appsets.eas or build using class ApplicationSettings class. The password will be MD5 hash calculation format and must 20 characters length, if less then adds space on the end of password so the length can be exactly 20 characters.

Next Progress

The next progress will be advance into:

  1. Including and support all TinyDB aggregates.
  2. Auto pairing for the remote BL521 device by supply the name and PIN.
  3. Remote query send out using sockets.
  4. Fix synchronized database table.

Files To Download

Files that include by project that can download from the attachments:

  • Source code for the project, create and build with Microsoft Visual Studio .NET 2005 with Visual C# language. Download the file: Source Code (esense.zip)
  • White paper of the project using Indonesian language. Download the file: Report - Indonesian (ind-report.pdf)
  • Main application of the project, using CAB installer. Download the file: Application - with installer (installer.zip)
  • Main application of the project, retails mode (just copy and paste). Download the file: Application - retails (retails.zip)

ċ
esense.zip
(5545k)
tuadi@urremote.com,
14 Jul 2010, 19:57
Ċ
tuadi@urremote.com,
14 Jul 2010, 20:46
ċ
installer.zip
(1900k)
tuadi@urremote.com,
14 Jul 2010, 20:23
ċ
retails.zip
(315k)
tuadi@urremote.com,
14 Jul 2010, 20:33
Comments