RADAR-CNS Significant Events Tracker

TODO collect other sources of relevant events

- of the app versions (see github or better we can get lists from the developer consoles of the aRMT and pRMT apps)
- releases of the platform back end from github
- android releases (e.g. the main schism in data completion for pRMT I think is an Android 8 release)


Very High – confirmed data loss

High  – likely data loss

Medium  – the potential for data loss

Normal  – no data loss


}{10/11/2018}​

EventDatePriorityNotesFollow up





KCL's ICE blocked the main router causing a ~2wk network loss??/03/18Very HighWhile this is within the window of data caching on the apps, there may be some associated data loss










Update aRMT App to use FCM pull notifications23/08/18HighExpected impact should be much more reliable notification and hence better compliance with questionnaires by comparison to the previous local notifications systemReview improvement in completion rates
Android 8 / 8.1 releasesee notesHigh

Varies by region and handset vendor

https://www.androidauthority.com/android-8-0-update-784308/

https://en.wikipedia.org/wiki/Android_Oreo 

March 16, 2018 – UK: The Galaxy S8 and S8 Plus in the U.K. are now receiving their Android 8.0 Oreo updates.

Moto G5  ??? 


Server Upgrade to new Signing algo ECDSA, MP 0.4.1, gateway 0.2.2, HDFS 3.0.3-alpine.MediumExpected lesser size of tokens and hence the QR codes. Should look out for any disruptions in data as this new signing algo is not extensively tested and changes all over the components. At first instance, seems to be working fine. Worked fine after a bug was fixed.





Likert Scale values in the ESM questionnaires changed from 0-7 to 1-7 in the ESM

definitions pushed to github on 03/09/18

but live on (i.e. incremented the protocol number) xx/xx/xx 


very highThis is a breaking change, so the data produced after this event (correct 7 point scale, 1-7) will be different for ESMs than prior to this date (8 point scale, 0-7). This was requested by the clinical teams to correct the error in their REDCap ESM data dictionariesAll data analyses need to take this into account (notably for LONDON_MDD where there are some 70 or so participants who have already generated data using this.
Change No multiple choice for ESM q 'what have you taken since the last beep' from single-choice radio-button to multi-choice checkbox.

 


 Updated REDCap to when note the date aRMT is updated next and close this issue  RSD-62 - Getting issue details... STATUS   cc/ Amos Folarin Yatharth Ranjan







Added FitBit connector to the production stack

Normal

Successfully pulled all historic fitbit data for 75 MDD participants. Need to check the prospective data as it is not tested very well yet.

There may be cases where the connector tries to pull data for certain dates which is not yet in the fitbit server but later ends up in the fitbit (when the fitbit is synced) and hence the connector will probably not fall back to pull that data resulting in some missing data. 


Stack may not be available as docker daemon went down. Was resolved within a day but data maybe missingstart  - resolution  MediumSome data maybe missing in this time as daemon went down. Most should be cached on devices and should eventually be uploaded.
An issue with baseline qs in aRMT was discovered in v0.5.5 which did not allow the responses to be posted to kafka. Release date of v0.5.5MediumThis was in v0.5.5 and should be fixed in v0.5.8 about to be released soon. This also adds sound:default in the notification payload in an attempt to enable sounds for notifications
FCM server updated from 0.1.1 to 0.1.4MediumThis introduced new server db mode and also uses separate disks (volumes) for database files.
Server upgrade to latest dev stack. Updates MP to 0.5.1, gateway to 0.3.1, backend to 0.4.0 NormalAlso Adds fitbit authoriser to the stack and updates radar schemas to 0.4.1. The upgrade of MP also means that the Meta QR code is now enabled and An alternative mechanism of authorisation is also implemented. 
Likert Scale on 2 fields were not changed in REDCap DDs, Medium

"field_label": "I find this virtual interaction pleasant",

"field_label": "Most people would find my current situation stressful",

These are now updated in the aRMT definition file & live


A schema change in questionnaire data


estimated start of issue: -  

resolved on:  

Very High

A schema change to the questionnaire schema (Adding the Notification Time – https://github.com/RADAR-base/RADAR-Schemas/pull/141) caused failed uploads of data on the v0.5.5 of the aRMT app (currently in production) because the optional field was not present. A hotfix for this is soon to be released by  . A major unfortunate consequence of this is that the corresponding data was neither uploaded nor cached due to a bug in gateway reporting unsuccessful uploads as successful (issue – https://github.com/RADAR-base/RADAR-Gateway/issues/29). So this may result in loss of questionnaire data in the month of november (other data like clinical tasks and thinc it should be fine).


Removed the Magnetic fields and Android step count from all studies and Smartphone accelerometer from KCL-MDD studyMediumAs requested by the WP8 and agreed upon by the clinical sites
Completion logs are found to be inaccurate? 2018Medium

Most completion logs are inaccurate. If the timestamp/notification time of a certain task has passed (at the time of opening the app), then that completion log will be sent (incomplete or not) even before the completion window of that task is over. For example if a PHQ8 has not been completed at 9AM and you open the app at 10AM, this will send a completion log saying the completion percentage of the PHQ8 at 9AM is 0, even if this is completed later in the day. However, if you open the app at 9AM, complete an RSES (scheduled at 10AM), close the app, and open it again at 10AM, this will send a correct completion log of the RSES with a completion percentage of 100.

This has been fixed and is pending testing.



Intervention testing 

  1. Removal of ESM for 6wks
  2. Increment of Completion Windows for PHQ8/RSES from 1 day to 3 days.
 High

New release of aRMT app 

Branch intervention-test used for modified versions of the protocol

Follow up may depend on results of the intervention and e.g. any notable effects on compliance rates.
REDCap: Added hidden timestamps to forms. Medium

Date forward of which individual Form Timestamps (`@NOW`@READONLY` ) can be used. Note as the @NOW field is used in redcap form it is a calculated field and needs to be set the first time the form is saved, so for retrospective forms the opportunity to correctly populate this is missed.

See the conversation on the PR

https://github.com/RADAR-base/RADAR-CNS-Studies-Data-Dictionary/pull/11

No follow up required. This significantly only affects the field value and is a note for people analysing the data no values of any of these timestamps should be guaranteed to be correct in participants enrolled before this date . The alternative to use is either other timestamps in the forms or the Event offset from the enrollment date.

Removed the Magnetic field, Smartphone step count and Smartphone accelerometer from other MDD studies. MediumThese were already disabled for KCL-MDD. Now disabled for other MDD sites too. This will help in managing the growing backend demand
aRMT Protocol Change for CNS KCL MDD Site: New audio taskNormalThe new audio task was released to RADAR-MDD-KCL-s1.
VUumc RADAR REDCap 3-Month Questionnairesissue from start of data collection till  HighThere was an issue with the 3 Month questionnaires as these survey invites were kept in the testing schedule (of 2 days post-enrollment). These will therefore not be representative of 3month for redcap IDs 1 to 19. The problem was fixed today  
VUumc RADAR REDCap "Depression Medications Adherence" Questionnairesissue from start of data collection till HighThe 6mt questionnaire was not properly scheduled for participants REDCap IDs 1 to 5. The problem was with the Survey Queue not properly chaining the Surveys. For DMA form it was scheduled to follow itself, therefore, wasn't being listed in the survey queue. The problem was fixed today  inform clinical teams and WP8 data analysis to take this into account.

aRMT Issue:

Kafka library issue causing two issues (app stuck in splash screen and `send_success` events not sending to Firebase)

First reported on 

Solved in new app version (0.7.2), released to production on  

Medium

App stuck in splash screen issue affected ~10% of participants (comparing previous Firebase event data). 

Send_success event issue affected ~80% of participants. However, data is cached and is not deleted from the cache until the `send_success` event is sent, so regardless of this being sent or not, this is still in the cache and will be sent after they update (with the fix).


aRMT Issue:


Changes in the way skipped questions in questionnaires are handled and stored in the questionnaire data payload.

First reported on start of audio data collection 

Solved in new app version (0.7.2), released to production on  

MediumIn app versions prior to 0.7.2, skipped questions in questionnaires were included as an empty entry in the answer payload. Since 0.7.2, skipped questions were not included at all in the answer payload.

aRMT Protocol Change for CNS MS Sites


Merged on  Normal

Changes are as follows: Remove PDQ, change PHQ8 from every 6 weeks to 2 weeks, and change clinical task notification from 1 week after clinical visit to 24 hours


Kafka cluster crashed (possibly due to high disk usage) MediumThe Kafka cluster crashed (isolated from other distributed components) possibly due to high disk usage (was around 97% when last time checked) even with a retention policy in place and was not able to recover. Could not reboot instance and hence has to destroy and re-create one. There should be no data loss as it should be cached and uploaded when the server was up again.
aRMT Protocol Change for CNS sites: New audio task NormalThe new audio task was released to all CNS sites (except for MDD-KCL, released in August).
aRMT Protocol Change for CNS KCL MDD Site: ESM Re-introduction NormalThe ESM (modified version: 28q) questionnaire was re-introduced to RADAR-MDD-KCL-S1.
aRMT Protocol Change for MDD Sites: COVID-19 Questionnaires NormalThe COVID-19 questionnaires: baseline and follow-up were introduced to the MDD sites (KCL, CIBER, IISPV, VUmc).
Redcap Updated (v7.4.10 → v9.9.0) NormalThe integration between redcap and RADAR was tested.
MP outage due to Kings wide internet outage

18/08/2020-19/08/2020

NormalAbout 16 hours outage of MP due to kings wide network outage possibly linked to the LINX. Started in the evening of 18th and resolved by noon on 19th.Fixed on 19th august
aRMT Protocol Change for CNS MDD VUMC Site: ESM Re-introduction NormalThe ESM (modified version: 28q) questionnaire was re-introduced to RADAR-MDD-VUmc-S1.
Management Portal Upgrade from 0.5.3 → 0.6.3, Also updated all clients including Gateway NormalThe upgrade was successful and everything was tested working fine. The aRMT, pRMT and other clients were tested as well as redcap integration was tested as working fine.
Rest Sources Authorizer updated to the latest version NormalThe upgrade was fine. Might have been some cases where logging in with the management portal posed some issues. But these were fixed by 1st March. The Fitbit data collection may be suspended for a while as it is being updated to work with the latest changes.The Fitbit data collection was resumed on 4th March 2021. There should be no missing data.
Power failure in Data Centre. Affected network connectivity and storage array High

Kafka Broker host server down for >12hrs. 
REDCap down for >12 hrs. 

There should be no issue for pRMT. aRMT data collection will have some minor data loss from notifications not issued over the downtime. Similarly, REDcap questionnaires will have a break in emailed invites for surveys.

VPN access affected access to the system.

Review and check system. 

REDCap config system check.

User services (MP, RedCap and backend Kafka) reinstated:  
Data Extraction (est.  )

Storage arrays down due to crashed switch.  HighThe notification server was down for 5 days. Other services were restored within 2 days.

All services fixed by 18th May 2021.