Over the summer, my Fitbit tracker (from before they split into multiple products) gave up the ghost. I decided it was time to explore the movement tracking space. First I participated in Phyode‘s W/Me Kickstarter, but that’s a topic for another post. More recently, I bought a Misfit Shine, followed by a Withings Pulse. This post will focus on comparing sleep analysis, the weakest point of activity trackers.
Tracking sleep via movements
There is sufficient correlation between movement and sleep quality that all the activity trackers attempt to tell you something about your sleep. You can’t hope for medically informative data from them, but you can probably compare your nights to each other and find correlations with caffeine use or exercise. Activity trackers expect you to notify them when you go to bed and when you get up, and then they analyze the time between a little differently than they do your waking time. As in the daytime, they don’t ask you where you are wearing the device, so analyses can be different based on where you put them. If they suggest anything, companies suggest you wear the tracker on your off-wrist during sleep.
In my case, I get a little annoyed with the results because most people move substantially in their sleep, on occasion. I have to wake up to make substantial movements. I can’t roll over in my sleep; I can’t even shift an arm or leg several inches without waking. I usually have discomfort dreams that inform me I need to wake up and move. But since that’s abnormal, the analyses I get don’t reflect my personal sleep movement patterns as well as they should. In my case, the activity trackers are overly willing to report me as having been asleep. Still, if I stare at the entire movement pattern for a sleep period I can infer things myself, and I find that useful.
Where and how I’m wearing them
Pulse and Shine ready for sleep!
During the day, I wear my Misfit Shine on a homemade choker. This is looser and therefore less accurate than the choker they sell, but it’s comfortable, secure, cute, and does well enough for my purposes. At night, I wear it on the watchband on my left wrist (I’m right-handed).
During the day, I wear my Withings Pulse on the clip on the front-right of my belt area. The clip itself feels very secure, but already the enclosing area is starting to feel less secure, so I’m a bit worried about that. Still, I’ve used many similar items on other electronics in the past, and they usually warp just a tiny amount and then stop. At night I wear it in the velcro wristband on my left forearm, above the Shine.
I imagine they get reasonably similar data in these positions.
Comfort and transitioning to and from night use
Transitioning the Shine between the choker and the watchband is easy: pop it out of one ring and into the other. Usually if anything is time-consuming or awkward it’s clasping the homemade necklace. I make the watchband a little more difficult by wearing it inside out, so that the knob doesn’t face my wrist while I sleep. I’m not sure why I do this; it isn’t a particularly poky knob. I think I just dislike wearing things on my wrist while I sleep and seek any potential comfort boost. Anyway, the watchband is quite comfortable in that it conforms to wrist shape, is flexible, has no stabby or lumpy bits, and secures down to my tiny wrist size without strangling.
Transitioning the Pulse between the clip and wristband is also very quick, but since the wristband is stretchy, wide, and velcro, it can take some effort to pull it taut without cutting off blood flow and without leaving velcro exposed to snag on your soft sheets and clothes. In case you can’t tell, I’m not a fan of velcro wristbands, especially for sleeping. I strongly disliked it with my Fitbit too. I’ve caught it on things a couple times (in just a week and a half!), and it even pulled off my wrist one night, screwing up half the night’s data.
Recording sleep time
Before I go to sleep, I have to click the button on the Pulse 5 times, tap the moon icon, and slide my finger in the direction indicated. This is tedious when I’m tired, but easy enough to accomplish. On the Shine, I have to tap it solidly three times. This sounds easy, but now that I have two items on my wrist the Shine gets pushed onto the bendy area, so doesn’t always have a solid item behind it ensure it receives the three taps. That can be annoying.
When I wake up, same deal with the Shine. On the Pulse, I press the button and it requests that I confirm I am getting up by sliding my finger again.
Issues with recording sleep time
The primary issue here is that I have to let it know when I am going to bed and when I am getting up. In both cases, I’m usually sleepy or have other things on my mind, so I don’t always remember. Neither of these services currently allow you to go back and say “Oh yeah! I swear I slept last night…maybe around this time to that time. Please update your analyses accordingly.” I hope both will allow it eventually. These are new products, so I don’t expect them to release with all the features I might like to see.
Now, if I do remember, I remember to do it for both devices. Yet I have no data for the Shine on one night. Did it get accidentally triple-tapped shortly before or after I went to bed and triple-tapped intentionally, thus negating the tag? Did I sleepily see some lights go off and assume it caught the triple-tap when in fact it only caught a double-tap? Or did I forget to inform either device and the Pulse inferred that I was asleep between two times? I can eliminate the last possibility because Pulse reported the amount of time “in bed” and “asleep” as different. But I still don’t know what exactly happened!
Then when I started comparing the results from each devices I realized there was often way more than a minute’s difference between the times each device recorded me as asleep. I got the impression that the Shine was, shall we say, adjusting the values I gave it. Last night I manually recorded my sleep and wake times. The Pulse reflected approximately the same times, while the Shine said I went to bed 20 minutes later. It’s true that I got up again right after I went to bed, but it makes me wonder why I bother telling it that I was in bed!
Standard deviations of the differences in sleep metrics used by both Shine and Pulse. In other words, these devices use very different algorithms!
With only a week and a half of data, I’ve determined that these services have very different algorithms for determining how long I was asleep and how much of that sleep was deep/restful. The standard deviations in differences between all comparable metrics on the two devices range between a 34 and 64 minutes, which is just huge.
Despite that activity tracking is a weak reflection of sleep quantity and quality, I already favor the Shine’s “deep sleep” algorithm. It seems to have some correlation with how rested I feel the next day. The Pulse’s sleep analyses seem to have almost no correlation with my state of mind the next day.
Interestingly, the only times Shine thinks I got more deep sleep than Pulse thinks I got, the Shine is mistakenly under the impression I was asleep much longer than I was. I looked at these data and it appears Pulse is better at trimming off awake times from the beginning and end of the period I tell it I’m asleep. So if I think I’m going to bed and then get distracted, or if I forget for an hour to tell it I’m up and just poking at my laptop/tablet/phone, Pulse is more likely to figure this out. Except that last night the Shine trimmed 20 minutes off the beginning of the time I was “in bed” because I was too active. So there’s something trickier going on here that will require more data to figure out.
Tracking sleep via movements: easy-to-get wishlist
One really simple item I miss from Fitbit’s otherwise useless sleep analysis: number of times I woke during the night. This one was handy: for example, it correlated with my caffeine intake the previous day.
Also, I haven’t seen anyone doing this, but I really want to see the longest stretch of time in “deep sleep”, or whatever they decide to call the most restful sleep. Just from eyeballing my raw sleep data, I’ve determined this is a factor in how energetic I feel the next day. I can’t imagine it’s meaningless for other people!