Sep 10, 2007

Datalogging with NXT-G

I've been digging around for a while trying to figure out a good way to do datalogging with NXT-G. Brian's old post was a good read, but was a little bit complicated at first glance. What I was after was the simplest solution that a teacher could not only implement but also easily understand.

So using the simple file write command, I threw together the following program.


It takes a reading of a timer, uses the 'number to text' to concatenate it with a rotation sensor output with a comma in between and then places it in a log file. It repeates this as often as possible. I found I needed to put a time stamp in, as Brian found, the processors tends to wander off every now and then to proccess some other info. Having the time stamp keeps the data valid even though the sampling rate is not perfectly consistant.

I ran the program on the TI-56 arm and manually opened and closed the gripper a few times.
Data file was then uploaded from the NXT and opened in Excel.

So what did I get?
I was pretty happy with the data. Sure there are sampling rate issues, but as a cheap datalogger I was very happy with it.


Closer inspection of the sampling rate showed just what is happening. With a quick Excel formula (sample time - old sample time) I could graph the time taken between each sample.

It's pretty consistant at 200Hz sampling rate with a regular blip about every 200ms where there is a gap of 11ms, and a more infrequent but larger gap of around 60ms about once a second.

So is it usable? For a primary or middle schools implementation, I think it's great. Kids can learn about datalogging and get into the issues of "what do I log?" and "how often do I sample?" and still get vey useful data.

Stay tuned for part 2 where I have a go at sampling up to 4 sensors at once to see what the effect is. I'm also planning to dump the commas, and process the data in excel, a little more tedius but may give faster sample times.

--
Damien Kee

edit: Thanks Brian for explaining 'why' there are the delays
"1) The short spikes to 10 ms are when the FW writes the accumulated data to FLASH. It does not write every time a File Access block requests it, instead "saving up" data until a block of FLASH can be written at once (causing the delay).

2) The longer spikes are *probably* associated with copying the entire growing text file to a new place in memory. But I'm not yet sure.

3) Eliminating the commas doesn't really gain you much. Since each comma is a single character, but the data is generally 3 or 4 charecters (or more - numbers in text files are stored as, well, text), it reduces the amount that needs to be written... but not very much.

4) Ideally, you should make sure you are opening a "clean" file, and close it at the end of a session. Odd things can happen sometimes if you don't."

11 comments:

Brian Davis said...

Nicely done, Damien. A few other points (for you or the readers... feel free to add these to the body of the post if you like).
1) The short spikes to 10 ms are when the FW writes the accumulated data to FLASH. It does not write every time a File Access block requests it, instead "saving up" data until a block of FLASH can be written at once (causing the delay).

2) The longer spikes are *probably* associated with copying the entire growing text file to a new place in memory. But I'm not yet sure.

3) Eliminating the commas doesn't really gain you much. Since each comma is a single character, but the data is generally 3 or 4 charecters (or more - numbers in text files are stored as, well, text), it reduces the amount that needs to be written... but not very much.

4) Ideally, you should make sure you are opening a "clean" file, and close it at the end of a session. Odd things can happen sometimes if you don't.

--
Brian Davis

BlueToothKiwi said...

Damien - that is a great post. Well done.

I look forward to seeing the second part: I would be looking forward to seeing what the maximum sampling rate you could achieve.

I tried datalogging the sensor only a week ago when I got my NXT-G 1.1. This is my experience:

(1) We kept kicking a basket ball at very high speed at the Ultrasonic sensor set up to log the the range (the sensor was secured behind a cage to avoid the ball damaging it - facing the ball through the gaps between the wire)

(2) I read somewhere that the NXT can sample the sensor input 3000 a second. (3kHz?)

(3) I could not achieve anywhere near that. May be I need to use C to do that as opposed to NXT-G.

(3) You might also want to log to memory and dump it to file after log completed (if the log duration are relatively small)

Thanks

Tim

Anonymous said...

"Data file was then uploaded from the NXT and opened in Excel."
Excuse my ignorance but how does one upload a data file from the NXT?

Do I first need to upgrade to 1.1?

Dean Hystad said...

To upload a file from the NXT you need to open the NXT window (push the tool button with the NXT icon on it), connect to the NXT that has the file (you are probably already connected), click over to the Memory page (using the tabs), select the file you want to upload and push the upload button.

Brian Davis said...

The NXT can rip a value off a sensor fast, but the NXT-G code is slower, and will limit logging to about 300 Hz or so at best. More criticall, you may have chosen the poorest sensor for fast datalogging (the US sensor). Remember that sensor is doing a lot on-board, in addition to having to wait around for the signal return, and for the area to "quite down" before the next ping is sent out. All in all, the US sensor will update much less frequently than most other sensors. As to storing the values in "memory" and then only later writing them to FLASH, well... you can do it, for brief periods of time, but it's tricky. It might be useful if you know exactly when you need to start "high-speed" data aquisition, however.

--
Brian Davis

Guy Ziv said...

There's another datalogging solution - using NXT as bluetooth master and PC as bluetooth slave (not all BT drivers support this). I've made a program that records data streamed from NXT to PC. You can read about it here:
http://nxtasy.org/2007/08/14/tool-release-nxtlogger/
With the tightest NXT-G loop I got around 330 Hz sampling rate.

Guy Ziv

Damien Kee said...

Guy,

I had trouble setting the PC up as slave, and haven't had time to completley figure out what's happening.

I like though the ability to take the NXT somewhere outside of a classroom without having to take a PC as well.

Damien Kee said...

Tim,

I'm avoiding writing to memory, as I want the 'easiest' solution for a classroom teacher who might not necessarily have the technical know-how to implement it.

AlexD said...

My 2c on the logging:

- I think that putting the read icons closer to one another will make the sensor sample close in time to the clock read. Otherwise there is always a small discrepancy while you are converting the time to text.

- If you know how many point you will acquire (or guess upward), you can always declare a file with that size. In this case the file is not extended every now and then, and the long delays should disappear.

- most modern signal processing algorithms deal with unequal interval sampling just fine. For one, you can always interpolate to obtain a set with effective fixed interval sampling. Even Excel may be able to do this with it data processing package (although I try to avoid using Excel for any kind of serious work). For visualization purposes time/value data also works just fine.

Damien Kee said...

You perfectly right Alex. The short delay in performing the file write block will *slightly* off set the data.

Will fix for the next installment

Sudeshna said...

Dean Hystad said...
"To upload a file from the NXT you need to open the NXT window (push the tool button with the NXT icon on it), connect to the NXT that has the file (you are probably already connected), click over to the Memory page (using the tabs), select the file you want to upload and push the upload button."

I have tried doing the same, using bluetooth as well as NXT connections. But each time i get an error saying " file not found." Please help.

Related Posts Plugin for WordPress, Blogger...