~jmoskie where joseph moskie dumps his crap

An Idiot and an Arduino: Power, Pause

It's been a solid two weeks since the last update, and I haven't made much progress. Here's what's going on.

I got all my wires and I (poorly) soldered one to the SENSE2 pad and then hooked it up. I ran it and it worked like a charm. The system would start up, send the temperature and relative humidity to the website, and then the WiFly shield would go to sleep for ~30 minutes before repeating the process.

Extremely happy, I hooked it up to a fresh 9v and popped it in my humidor. With the WiFly chip entering "deep sleep" mode, I was sure the battery would last longer, but I wasn't sure how long. I set it up and went to bed.

Wouldn't you know it, 10-11 hours later the battery died. That's the same time frame as all of the previous tests. I did not understand this at all, there was no change at all from having the wireless chip go into a deep sleep. I went online and looked around, and talked to my friend Will. He found the following writeup regarding Arduino and XBee Battery Test Results, which says that an Arduino doing nothing but a simple read every 500 ms will last 14 hours, 24 minutes on a 9v battery.

Despite all my efforts (and successes) attempting to make the WiFly chip draw less power, the Ardunio itself simply can't last a single day on a 9v battery. Comparatively, it seems like the WiFly running without any deep sleep at all didn't really have much of an effect compared to the power draw of the Arduino itself.

So, in order to "advance" the hardware to any degree of actual usability, I would need to ditch the Arduino entirely. My brother says that you can take the ATMega chip out of an Arduino and add some things to a board and essentially reproduce everything you need from the board and leave everything you don't out. I assume it's vastly more complicated than that, but I don't know. I haven't really investigated this too much, but I plan to talk to some of my friends about it and try to get a handle.

This seems like a level of hardware development far beyond what I believe myself capable of, but I guess I would have said that about the system I have working right now a few months ago.

In the meantime, I am thinking of buying a cheap humidor and drilling a hole in the back of it in order to run a power cable through it, so that I could work on the software / web interface with real-time data without changing the battery every 11 hours.

An Idiot and an Arduino: Ruminations on Power

I squealed in delight when my little wireless hygrometer actually started posting data that made sense to my website. I unplugged it from my laptop and attached the only battery source I had to it, a 9v battery with a barrel jack.

I put the setup in one of my humidors, and watched. Sure enough, the LEDs were all blinking as they should, and data was being pushed to my website. I had the thing run once every 5 seconds, which was hilarious, but I wanted to get a lot of data points to play around with. I left it on, and went to bed.

I woke up to no LEDs at all, and presumably a dead battery. I took the setup out and then loaded up the web interface, showing the logged data. The last update was only an hour before I woke up, and looking back to the first battery-powered sensor reading, it seems as though the device ran for a bit more than ten hours.

"Obviously, this is because I'm sensing and sending data every 5 seconds", I thought to myself. I talked to my little brother, who seemed to agree. 5 seconds is a hilarious sample time for this purpose. In a real-world setup you could easily get away with 12 hours, 24 hours, or even once every 48 hours. Unless something goes horribly wrong, your humidity in a humidor should not change significantly in any short amount of time. However, such long readings would require a long time to, you know, test.

I changed the code to sample every half hour. This means that I went from 720 samples and data transmissions per hour, to a mere 2. I plugged in a fresh 9v battery, and let it run again. To my surprise, the battery died in roughly the same amount of time as it did previously. Reducing the number of times the sensor was activated, and the number of times the WiFly chip was actually sending/receiving data seemed to have no discernable effect on the life of a battery.

I looked at the documentation for the WiFly shield, and started talking to my friend Will. The datasheet said that the WiFly chip used 40mA when it was on, and 4µA when it was in "deep sleep mode." In previous discussions with my brother, I was led to believe that the chip automatically entered some sort of "idle state" when it wasn't in use, and that was simply not true. Sitting idle, the chip was still using ~40mA to stay connected to the wireless network. This was confirmed with my brand new multimeter, which I had no idea how to use because I am a child.

I then started working on figuring out how this "deep sleep mode" worked. Sadly, it is not a simple API call or anything like that to say "go to sleep", you have to have the chip enter command mode, configure some settings and timers, and then save/reboot. Doing this "on the fly" in a way that would let the Arduino simply say "go to sleep, WiFly" wouldn't really be feasible. Setting the running device into "command mode" and rebooting it over and over interrupts normal program flow, and a full reboot probably has some power drain associated.

While I was looking at this, Will was trying to convince me to use a transistor to simply cut the power supply to the WiFly whenever I wanted to. It wouldn't be "go to sleep, WiFly" as much as it would be "THERE GOES YOUR POWER, BITCH!" I didn't like this solution much, again because powering on the chip from nothingness probably uses more power than simply waking from a sleep. Not to mention what would happen if the WiFly was trying to do something when the power went off.

So I continued reading the spec sheet and playing around. It was pretty straightforward, there were three commands that controlled how WiFly could sleep:

set sys sleep <seconds>
set sys wake <seconds>
set sys trigger <sense pin id>

The first two are purely timer based. Using the first one is effectively saying "Go to sleep after X seconds." The second says "Wake up after Y seconds." These timers would then loop over and over again, so you can make it do things like "Wake up for 30 seconds every 2 minutes" by doing the following:

set sys sleep 30
set sys wake 90

So I added that to my WiFly's configuration by modifying the WiFly library a bit, and then ran it. Sure enough, I saw the LEDs go dark after 30 seconds, and come up 90 seconds later.

Alright, that's neat. All I have to do is sync up the delays in the Arduino code with the sleep cycle of the WiFly.

"Deep sleep mode" worked, but it was a little annoying having to deal with the timers. You can't pause a timer directly, but you can reset it to it's original delay by sending it some data. Even junk data will reset it. The datasheet says:

The sleep timer is disabled if the module has an IP connection, or the module is in COMMAND mode. The timer is reset when characters are received on the UART.

So the chip will never shut itself off when it's actively connected, which is good. The sleep timer starts over whenever the connection clears, starting at it's original delay. The timer is also reset when the UART receives any data, so if it was at 1 second until going to sleep and it all of a sudden got a command to do an HTML GET from the Arduino, it would wait until the request/response finished, and then start back up at it's original delay.

The original "wake up for 30 seconds every 2 minutes" is a bit more complicated than previously explained. More accurately, it's, "wake up for a 30 second window to accept commands. For every command received, start the timer over again. Disable the timer while we have an active connection, then start it over again." Suddenly the 30 second window could have been extended to 70 seconds if an HTML request was issued late, and took a while to respond. Even though the WiFly had timers to control when to go to sleep and when to wake up, the built-in functionality to reset the timers made it extremely hard to accurately time when the chip would be awake.

This wouldn't be a problem, but there is no real way to get the state of the WiFly chip, as far as I can tell from the datasheet. You can't really know if it's currently sleeping, programmatically. I played around with sending commands to the UART and trying to get a time out, but the program kept getting stuck (hanging) waiting for a response. I thought that maybe I could open one of the pins the shield uses as input, and then try to read from it from the Arduino code, but that's pretty sloppy and could possibly cause problems later. With no way to know if the chip is currently sleeping, you can't know if it's "safe" to try to issue HTTP requests from the chip.

I say "safe", because with the library I'm using if you issue a GET request and the UART never responds, the application hangs. It doesn't just hang until the UART comes back up, either, it hangs forever. It's very hard to get around this sort of thing and still have the ability to wait for an expected response from the WiFly chip. When you send the UART a command, you'll likely want the response so that you can tell if it succeeded or something went wrong. Generally, you have to wait for the entire response to come in, and then read it out of the buffer. If you don't wait for the response to come, you'll often miss it by reading the buffer early, so the "wait until expected response" paradigm is used throughout the WiFly library. This causes a GET request to a sleeping WiFly chip to simply hang, waiting for a response that will never come. Eventually I should enhance the library to give up /time out after a while.

This, combined with the fact that you can't query for the sleep state of the chip, means you've got a dangerous situation on your hands. The sleep timer of the WiFly is messed up by simply using the chip. With no way to sync the timers, I was forced to fall back to the other way of waking the chip: Sensor data.

I should have gone this way from the start, but it involves soldering directly to the WiFly shield, so I was trying to avoid it as long as I could. There are three little areas or pads or whatever they hell they are on the shield, labeled SENSE1, SENSE2, and SENSE3, and to make use of them a wire must be soldered on.

The "SENSE" Pins

Talking with Will, I decided that I'd just hold the wire there for now and try some things out. I read the rest of the documentation concerning the "Wake on Sensor" options:

SENSE 0 to 3 inputs are available to wake the module from sleep.

SENSE 0 to 3 pins have a small current source that is activated in sleep mode.  This source is approximately 100nA, and will cause the input to float up to about 1.2VDC. If SENSE1 for example, is enabled, pulling the SENSE1 pin to GROUND will wake the device.

Sensor inputs are rated 1.2VDC maximum. You must use a resistor divider when driving a sensor pin from the other 3V pins such as RX. A resistor divider network with a minimum of 24K in series and 10K to ground from the UART RX or CTS pin should be used.

WARNING:   Under no conditions should the voltage on any sensor input exceed 1.2VDC. Permanent damage to the module will result.

I hate reading things in bold like that.

I had no idea what a "resistor divider" was, or how any of the math was supposed to work. I vaguely understood that we were trying to limit the voltage so we didn't aggravate that bold text above, and (after re-reading it several times and talking with Will) that they wanted you to GROUND the SENSE pin in order to wake the thing. Thankfully, Will did know what a resistor divider was, and watched me work on a breadboard through my webcam.

Having only the parts from the small kits I've bought, the best thing I could do with resistors was to chain up 3 of the 10k ones in series. This was easy enough to do. I ran 3.3V from the Arduino to one end, and ground to the other. I plugged in another wire to where the second resistor met the third, and tested the voltage with the multimeter. It was showing under 1.2V, so hopefully I wouldn't be shorting anything out.

Then Will instructed me to take out the tiny transistor that came with my kit (P2N2222AG) and pop it into the breadboard. I ran ground to pin 3 (emitter), a wire attached to one of the Arduino's digital out to pin 2 (base), and that wire I had put aside above to pin 1 (collector). Finally, I plugged another wire into the same node as pin 1, which would be held to the SENSE area to try to start the WiFly back up.

The basic idea is that we need to supply the SENSE pin with power (while staying under 1.2V) until we want the chip to wake up. At that point, we have to ground out the pin. By plugging in the 3.3V from the Arduino into the series of resistors, we lowered the voltage sufficiently, and hooked it up to the collector pin of the transistor. The emitter pin was grounded, and the base bin was connected to an Arduino pin that we could control programmatically. Essentially, when we send HIGH to the base, the circuit inside the transmitter is completed, and is grounded. If the SENSE2 pin is hooked up to the circuit, it also gets grounded.

My shitshow of jumper wires

I updated the sleep configuration settings to reflect the new setup:

sendCommand("set sys sleep 20");
sendCommand("set sys wake 0");
sendCommand("set sys trigger 4"); // listen on SENSE2

This configuration says "Sleep after 20 seconds have passed, and only wake up with SENSE 2 is grounded." Setting the wake to 0 is required because  simply commenting it out left the wake timer active. It's a configuration setting on the chip, and when it gets set and saved, it has to be explicitly turned off. I ran it a few times before I noticed this, and wondered why the chip was turning itself back on. I figure it's safer to just leave it in for now, in case I play around and forget to turn it off again.

I fixed that and uploaded the sketch again, and powered it on. It sat on for 20 seconds, and then turned off. I pointed my Webcam at it and we waited longer than 90 seconds to make sure that the old wake timer wasn't going to kick in, and it didn't. The WiFly was now firmly in deep sleep mode, and wasn't coming up of it's own volition. The program loop was still running, and my terminal was catching debug messages. Every 20 seconds it would read the RH/Temperature, and then send HIGH to pin 5 for 5 seconds. I waited until the program was in the 20 second wait, and then I pressed the red wire labeled "to SENSE2" to the right terminal of the SENSE2 area, and we waited.

Sure enough, when the debug output came up saying that HIGH was being sent, the WiFly chip lit up and started sending data again. 20 seconds after the data was sent (remember the timer resets and delays for active connections!) it went back to sleep. I kept holding it, and when the loop came back around the cycle repeated.

I currently don't have any actual wire in my possession, only these jumper wires for breadboarding, so I cannot solder a wire to SENSE2 and make this permanent. I will remedy this soon, but for now I'm pretty excited. Thanks to Will's help and yelling, I now have a wireless chip that actually goes to sleep, and can be woken up by the Arduino code. The main loop on the hardware can simply delay for an hour, and then read the sensor, power on wireless, fire off a request, and then go back to delay. The wireless chip itself will go back to sleep 10 seconds after it finishes its task.

I'm eager to solder that wire and then try it again and see how long the battery lasts with the WiFly chip actually sleeping. I know that a 9V battery is really ill suited for this task, but it's the only power supply I have right now. I've ordered a few other things to try as well, but for now I'd like to compare apples to apples and see how long a 9v lasts doing 30 minute updates with the WiFly sleeping between uses.

I'll post another update and maybe a video whenever that wire gets here and I get to solderin' and testing it again.

An Idiot and an Arduino: Integers are Integers, Idiot

I deserve to be slapped.

I was genuinely perplexed as to why my data was wrong. I instantly blamed the sensor, and started talking to friends about how the voltage could be wrong, or how it might not work correctly if this happened, or how it might need a "burn in" period, or how the library might not be correct, or how that one "bonus bit" I had to ignore might not be as useless as it seemed.

The last place I thought to look was my code.

This hubris has bitten me in the ass often enough. I write good code, but sometimes I make really stupid mistakes, especially when I'm moving fast. This time, I made two mistakes.

When writing my PHP script that graphs the data, I initially set it up so that Temperature was the first plotted variable, and Humidity second. By default in Google's APIs, the first line is blue and the second one is red, so I swapped the names in the legend... Without swapping the actual values in the body of the code.

So last night when I was looking at that flat line for humidity, it was actually a flat line for temperature. I even said in the video that when I breathed on the sensor, I expected the humidity to go up. Well, it did, but it was labeled temperature.

The second mistake is where the title of this entry comes from. What's nine divided by five? Some people would tell you 1.8, but they're wrong. Very very wrong. 9/5 is 1, obviously. 9.0/5.0, on the other hand, actually is 1.8.

The sensor reports the temperature in Celsius, and since I'm an American, those numbers don't make any sense to me. I added a method to convert to God-fearing Fahrenheit, forgetting about how integer math works. The reason my line was pretty flat was because the multiplier was 1 instead of 1.8, which greatly changed what numbers got spit out. 20 degrees C is 68 degrees F, but my code was reporting 52. 25C is 77F, but I was reporting 57.

Changing it made everything look right. The sensor has a high error tolerance on humidity, ±5%. It's showing 60.8% right now, and my actual hygrometer is showing 66.2%.

The new, correct graph

I eagerly await the scolding and mocking.

PHP, you have ruined me.

An Idiot and an Arduino: The Wireless Hygrometer

I left off with a WiFly chip which couldn't hit external addresses. I could ping local machines on my router, but trying to ping anything external (such as my webhost, google.com, etc) were all failing without any error code. I was using the WiFly's ping command, through their Terminal application, and nothing was happening.

Then the Superbowl happened, and I got tied up in work on Monday.

While smoking a cigar today, my brother hopped on Skype and called me up. I talked about my problem and he said that he never bothered trying the ping command, and just jumped right to HTTP GET commands. We talked about it for a while, and after I finished my Gurkha Symphony, I sat down and started playing with it again.

My brother pointed me to the WiFly HTTP Web Tutorial that came with the library itself. It was a simple affair, connect to the wireless network, send a simple request to google.com

GET /search?q=Arduino HTTP/1.0

I ran it, and it choked. Connecting to the wireless network was being flakey. Sometimes it would connect fine, other times the chip needed to be physically restarted by power cycling it or mashing the reset button. It would just hang, no error messages. My brother said something about the library trying to connect before the chip was fully powered, or something, so I added a delay. No change.

I ended up having to add a bunch of debugging to the library itself. Little bits of println("[1]"); here and there to see what part of the library was hanging. I narrowed it down to it waiting for a response from the router that only came sometimes, for some reason. The way the code was written, when it sends a request it waits for a response, which is a pre-defined set of characters (a char*), and if it doesn't get it, it just loops forever. Sometimes the router would respond with the expected string, and sometimes it would respond with a different message. it was still a success message, just not the one the library was expecting.

Simply removing the check for this particular case seemed to work just fine. After that, the example worked fine, and I was able to reach Google and do a search.

The ping command on the chip didn't report anything when I tried to ping google.com, but it had no problem doing a GET on it. The little terminal spat out a bunch of HTML that would render as a Google search in a browser, but was just an absolute mess of text in the window.

I then tried to point it at my web server, so it could hit my web site. For now, just pull down index.php and vomit the data out, later we'll work on some backend. I received a 404 error, something my brother had run into earlier. I believe it probably has something to do with the fact that my web host is on shared servers. I'm not sure how everything works behind the scenes, but something odd was happening with this raw GET request from the chip. I remember writing a Java application to hit my site when my brother first reported this problem months ago, and it worked fine. But somehow, something was different with the WiFly.

After a few moments of tinkering, I thought to try an HTTP 1.1 GET request, specifying the hostname in the request itself.

GET /index.php HTTP/1.1
Host: www.josephmoskie.com

That worked like a charm, and suddenly the little piece of silicon on my desk was talking to my web server.

Sorcery.

After that, I was in familiar territory. Working on sites PlayYourTurn and MinecraftWorlds (which I quickly abandoned and sold the domain to a Minecraft hosting company), I was pretty well versed in how to accept data from an application and dump it into an online database. I wrote a really quick PHP Script to take two values, Temperature and Humidity, add a timestamp to it, and toss it in the database. I manually tested it a few times to make sure it seemed to work, added some error handling, and went back to the hardware.

I had two separate Arduino projects: My hygrometer, and my wireless doohickey. They did not play together yet. I plucked all the wires from the hygrometer's Arduino. It wasn't complex, one wire for 5V power, one wire for ground, and one wire in pin #2 for data transfer. I took it and plugged everything into the same pins in the WiFly shield. By dumb luck, the shield didn't use pin #2, so I was all set. The physical parts were joined, now I had to combine the code.

It was simple enough. I took my WiFly project and imported the library for the RHT03. I copied over important bits of code for initialization. I examined the structure of the HTTP client code. I moved the HTTP request into a loop, added a 5 second delay, and ran that. It hit the website every 5 seconds, and added a dummy entry to the database. Excellent. Then I copied over some chunks of the hygrometer code, which grabs the temperature and humidity as floats. I changed the code to build the URL using the values from the hygrometer, and re-launched it. There were some compiler errors, and some weird infinite loops (the developers of the WiFly library seem to think hanging with no error message is an appropriate thing to do, and do it often), but I got it working!

I let it run for a while, and numbers started appearing in database. I blew on the sensor. The temperature went up. Oddly, the humidity did not. I put the rig into my humidor.

The rig in one of my humidors

I watched as the temperature went up, and... The humidity did not. I have an actual hygrometer in the humidor, and I know the humidity is 66%, and the sensor is reporting ~56%. I took a closer look at the temperature, and it was also incorrect. The hygrometer was reporting 75F, and the sensor was showing ~52. Over time, the temperature from the increased, but it was still pretty low (~61 degrees). The humidity didn't move.

However, it was gathering data, and reporting it online. I quickly whipped up a Google chart with some PHP to pull the data out of the database and show it in a pretty chart. BEHOLD!

BEHOLD THE WIRELESS HYGROMETER!

That first spike is me breathing on the sensor. The gradual rise was after I put it in the humidor. The sharp drop is when I took it out to reprogram it. Then the next rise is it being back in again.

Again, the values don't jive with what my actual hygrometer says. The numbers look good (i.e. they're not random garbage, and could actually be temperatures and humidities). I will look over the library and try to figure out if I'm missing anything, or calculating anything wrong.

I made a short video showing what's going on right now. I'm pretty happy with what I've got going so far, even if the number don't seem correct.