Android Phone Sensors Missing Data
Discussion around peculiarities of Android phone sensors, data incompleteness, known issues, work-arounds etc.
See page on phone sensors data for some documentation of the sensors and the specification documents in github:
Values not recorded if unchanged
There are two data streams, the light sensor and battery level, that only record data if the value changes. Therefore the data is collected in very irregular intervals.
Night time values may get limited by Android
There is some indication that over the night Android makes assumptions around sensor activity variability and may limit the data collected.
See issue raised in relation to the Light and Battery sensors on Android.
Details of phone sensor details for pRMT https://radar-base.org/index.php/data-sensors/
Environment Sensor → Light
Other → Battery
Other Android limitations
Android may be more aggressively limiting phone sensor data collection
RSD-13 Text Messages Contacts Status can be NULL
schemas https://github.com/RADAR-base/RADAR-Schemas/blob/master/commons/passive/phone/phone_sms.avsc # value is unknown if contact is unknown, don't assume that it is false.
it is not allowed to determine if OUTGOING messages ids are contacts (as far as we are aware, seems to be an API limitation)
see: source code https://github.com/RADAR-base/radar-android-phone/blob/master/src/main/java/org/radarcns/phone/PhoneLogManager.java#L260-L285
you see the value is not initialized for outgoing
Some of the contacts will later be know because the same hash value is seen in INCOMING, it may at that point be possible to add populate a TRUE value into some of the messages that previously were OUTGOING. See the image attached to the issue
FITBIT DATA Anomalies
Sleep stages Unknown
a bug in the sleep stage mapping, where the "wake" sleep stage is mapped to "UNKNOWN". So during analysis, any UNKNOWN sleep stage can be mapped to AWAKE.
ANDROID USAGE DATA
The time in topic "application_time_zone" is provided by the pRMT app while the timezone in "connect_fitbit_time_zone" is provided by Fitbit.
There is also a third timezone topic called “questionnaire_timezone” which is provided with every completed questionnaire from aRMT app.
It maybe worth combining the 3 and using the data which ever is available at a particular timepoint. App's timezone data should be preferred instead of Fitbit as it may be stale/ outdated (Fitbit does not sync as regularly as pRMT)
But it also depends on the source of the data.
For example For Fitbit data, using the Fitbit_time_zone makes more sense, For Questionnaire data, using questionnaire_timezone makes more sense and so on.