Datalogging with NXT-G
I've been digging around for a while trying to figure out a good way to do datalogging with NXT-G. Brian's old post was a good read, but was a little bit complicated at first glance. What I was after was the simplest solution that a teacher could not only implement but also easily understand.
So using the simple file write command, I threw together the following program.
It takes a reading of a timer, uses the 'number to text' to concatenate it with a rotation sensor output with a comma in between and then places it in a log file. It repeates this as often as possible. I found I needed to put a time stamp in, as Brian found, the processors tends to wander off every now and then to proccess some other info. Having the time stamp keeps the data valid even though the sampling rate is not perfectly consistant.
I ran the program on the TI-56 arm and manually opened and closed the gripper a few times.
Data file was then uploaded from the NXT and opened in Excel.
So what did I get?
I was pretty happy with the data. Sure there are sampling rate issues, but as a cheap datalogger I was very happy with it.
Closer inspection of the sampling rate showed just what is happening. With a quick Excel formula (sample time - old sample time) I could graph the time taken between each sample.
It's pretty consistant at 200Hz sampling rate with a regular blip about every 200ms where there is a gap of 11ms, and a more infrequent but larger gap of around 60ms about once a second.
So is it usable? For a primary or middle schools implementation, I think it's great. Kids can learn about datalogging and get into the issues of "what do I log?" and "how often do I sample?" and still get vey useful data.
Stay tuned for part 2 where I have a go at sampling up to 4 sensors at once to see what the effect is. I'm also planning to dump the commas, and process the data in excel, a little more tedius but may give faster sample times.
--
Damien Kee
edit: Thanks Brian for explaining 'why' there are the delays
"1) The short spikes to 10 ms are when the FW writes the accumulated data to FLASH. It does not write every time a File Access block requests it, instead "saving up" data until a block of FLASH can be written at once (causing the delay).
2) The longer spikes are *probably* associated with copying the entire growing text file to a new place in memory. But I'm not yet sure.
3) Eliminating the commas doesn't really gain you much. Since each comma is a single character, but the data is generally 3 or 4 charecters (or more - numbers in text files are stored as, well, text), it reduces the amount that needs to be written... but not very much.
4) Ideally, you should make sure you are opening a "clean" file, and close it at the end of a session. Odd things can happen sometimes if you don't."
So using the simple file write command, I threw together the following program.
It takes a reading of a timer, uses the 'number to text' to concatenate it with a rotation sensor output with a comma in between and then places it in a log file. It repeates this as often as possible. I found I needed to put a time stamp in, as Brian found, the processors tends to wander off every now and then to proccess some other info. Having the time stamp keeps the data valid even though the sampling rate is not perfectly consistant.
I ran the program on the TI-56 arm and manually opened and closed the gripper a few times.
Data file was then uploaded from the NXT and opened in Excel.
So what did I get?
I was pretty happy with the data. Sure there are sampling rate issues, but as a cheap datalogger I was very happy with it.
Closer inspection of the sampling rate showed just what is happening. With a quick Excel formula (sample time - old sample time) I could graph the time taken between each sample.
It's pretty consistant at 200Hz sampling rate with a regular blip about every 200ms where there is a gap of 11ms, and a more infrequent but larger gap of around 60ms about once a second.So is it usable? For a primary or middle schools implementation, I think it's great. Kids can learn about datalogging and get into the issues of "what do I log?" and "how often do I sample?" and still get vey useful data.
Stay tuned for part 2 where I have a go at sampling up to 4 sensors at once to see what the effect is. I'm also planning to dump the commas, and process the data in excel, a little more tedius but may give faster sample times.
--
Damien Kee
edit: Thanks Brian for explaining 'why' there are the delays
"1) The short spikes to 10 ms are when the FW writes the accumulated data to FLASH. It does not write every time a File Access block requests it, instead "saving up" data until a block of FLASH can be written at once (causing the delay).
2) The longer spikes are *probably* associated with copying the entire growing text file to a new place in memory. But I'm not yet sure.
3) Eliminating the commas doesn't really gain you much. Since each comma is a single character, but the data is generally 3 or 4 charecters (or more - numbers in text files are stored as, well, text), it reduces the amount that needs to be written... but not very much.
4) Ideally, you should make sure you are opening a "clean" file, and close it at the end of a session. Odd things can happen sometimes if you don't."
Comments
1) The short spikes to 10 ms are when the FW writes the accumulated data to FLASH. It does not write every time a File Access block requests it, instead "saving up" data until a block of FLASH can be written at once (causing the delay).
2) The longer spikes are *probably* associated with copying the entire growing text file to a new place in memory. But I'm not yet sure.
3) Eliminating the commas doesn't really gain you much. Since each comma is a single character, but the data is generally 3 or 4 charecters (or more - numbers in text files are stored as, well, text), it reduces the amount that needs to be written... but not very much.
4) Ideally, you should make sure you are opening a "clean" file, and close it at the end of a session. Odd things can happen sometimes if you don't.
--
Brian Davis
I look forward to seeing the second part: I would be looking forward to seeing what the maximum sampling rate you could achieve.
I tried datalogging the sensor only a week ago when I got my NXT-G 1.1. This is my experience:
(1) We kept kicking a basket ball at very high speed at the Ultrasonic sensor set up to log the the range (the sensor was secured behind a cage to avoid the ball damaging it - facing the ball through the gaps between the wire)
(2) I read somewhere that the NXT can sample the sensor input 3000 a second. (3kHz?)
(3) I could not achieve anywhere near that. May be I need to use C to do that as opposed to NXT-G.
(3) You might also want to log to memory and dump it to file after log completed (if the log duration are relatively small)
Thanks
Tim
Excuse my ignorance but how does one upload a data file from the NXT?
Do I first need to upgrade to 1.1?
--
Brian Davis
http://nxtasy.org/2007/08/14/tool-release-nxtlogger/
With the tightest NXT-G loop I got around 330 Hz sampling rate.
Guy Ziv
I had trouble setting the PC up as slave, and haven't had time to completley figure out what's happening.
I like though the ability to take the NXT somewhere outside of a classroom without having to take a PC as well.
I'm avoiding writing to memory, as I want the 'easiest' solution for a classroom teacher who might not necessarily have the technical know-how to implement it.
- I think that putting the read icons closer to one another will make the sensor sample close in time to the clock read. Otherwise there is always a small discrepancy while you are converting the time to text.
- If you know how many point you will acquire (or guess upward), you can always declare a file with that size. In this case the file is not extended every now and then, and the long delays should disappear.
- most modern signal processing algorithms deal with unequal interval sampling just fine. For one, you can always interpolate to obtain a set with effective fixed interval sampling. Even Excel may be able to do this with it data processing package (although I try to avoid using Excel for any kind of serious work). For visualization purposes time/value data also works just fine.
Will fix for the next installment
"To upload a file from the NXT you need to open the NXT window (push the tool button with the NXT icon on it), connect to the NXT that has the file (you are probably already connected), click over to the Memory page (using the tabs), select the file you want to upload and push the upload button."
I have tried doing the same, using bluetooth as well as NXT connections. But each time i get an error saying " file not found." Please help.