I got my program to read into the file all the way past the header data up to right before the actual recorded data starts.
So far I'm getting away with using arrays of structs that are just declared on the stack (I apologize if this isn't correct terminology).
These arrays aren't unreasonably large so I'm going to let it slide.
The rest of the data is large, but not large all in once place. I think I wrote before how in order to get the data pertinent to one channel of one data type I'd have to scan through the whole file. Dynamically declaring memory to hold all the data of a channel would mean first scanning through the data to get the total size of the channel, declaring my memory space, and then rescanning and saving the data. Is this normal?!
I've decided to try making this a program that will return one channel of data called by name via the command line. For example:
./readfile filename.plx CH01
would get all the data for CH01 and return it appropriately... somehow. Appropriately might wind up being a big text file. Most data comes out in pairs - timestamp and value. Not sure how to handle that yet.
I have been reading more of the Python documentation, and the free ebook "Dive Into Python". I haven't coded much. When I have done something I've used TextWrangler to write it (does various syntax highlighting) and I've run it from the console. Nothing fancy. I actually ran into a problem that Python might be well suited for. I got into the habit of taking pictures of things and then making a copy of a dozen or so pictures into a different folder for resizing and uploading to Facebook. Resizing the pictures one-by-one is kind of annoying. I dont think I have any programs that do whole folder resizing. Anyways, I bet there are Python modules for image manipulation and I've already seen examples of viewing directory contents and executing commands form the console from Python. It should be simple to tell a Python script to apply a resizing package to all the files in a directory. I've been working with C for a while and I can't think of how to do this at all. I'm not even sure how to get a list of files in a directory or run a console command....
1) The term you're looking for is 'statically allocated' which is not stack memory in this case. When a program is run from the command line it barters with the OS saying "I need X bytes of memory for me globals... do you still love me despite all this baggage?". If the OS can handle the X bytes, it will allow the program to load and own that 'footprint' in memory.
ReplyDeleteAn experiment to observe this in action: declare another statically allocated global array with a ridiculous size and then observe the memory usage during program execution (using 'taskmgr' in windows or 'top' in *nix). Show that by commenting out the variable you can calculate the change (reduction) in the program's memory footprint. You might need to toss in an infinite loop to prevent the process from exiting before you can observe the memory footprint.
In theory, if your app doesn't use malloc() your program's memory footprint will always be static. However, your library calls might be sneaking some sweet sweet malloc() on the side. Experimentation will tell.
2) "Is this normal?!". Normally you allocate memory in the smallest chunks that are efficient. There are a number of strategies that are used for this and they all depend on the specific purpose of the program.
One common method in data format design is to include the payload size in the data header so you can allocate upfront ('framed' approach). It's more work on the front end because the capture program has to go back and fill in the size once it's done capturing data. Someone probably decided this wasn't worth the hassle for .plx files and accepted the effect this has on reading/processing the data.
In some cases it might be more efficient to allocate as you go (e.g. allocate in 1 Kbyte chunks, allocating more when you run out). Again, this is application dependent.
To that point, you'll find that when you start dynamically allocating stuff out in memory wilderness that way (chunks at a time) you'll need some extra equipment to make sure your data party sticks together (doesn't get eaten by bears, etc.). Linked list pointers are a common "2-way radio" you can use for this.
3) Perl/Python/(insert scripting language name here) is great for batch image manipulation. I think GIMP has some built-in (plug-in?) batch features too...