...
These components are core components of Radar-base and should always be active.
Name | Description | Alternatives |
Creates SSL/TLS certificates to access the web applications. Uses Letsencrypt certificates by default. | ||
Creates RADAR-base topics in Kafka, registers schemas in Schema Registry and keeps a catalog of available source types. | ||
All real-world data in RADAR-base goes through Kafka, an advanced message broker. Data is stored there for processing by other applications, like real-time analysis or long-term data storage. Data remains for a limited time (usually a month). | Confluent Cloud Amazon MSK | |
Portal to manage projects and participants throughout RADAR-base. | ||
Web server used for accessing web applications. | ||
Database used to store Management Portal data. | Amazon RDS for PostgreSQL Dedicated PostgreSQL server | |
Reads data from an intermediate object storage and converts it to a RADAR-base oriented output directory, by organizing it by project, user and collection date. | ||
Reads raw participants data from Kafka and puts them into an intermediate object storage like AWS S3 or Minio. | ||
Used by Radar Output to coordinate data processing in a parallel way. | ||
REST Gateway to Kafka, for incoming participant data. It performs authentication, authorization, content validation and decompression. |
Email server for sending registration email to researchers and participants.
Registry of Avro Schema’s used in a RADAR-base installation. Validation layer on top of Kafka to structurally verify incoming data. | Confluent Cloud | |
Use for Kafka cluster management and coordination. | Confluent Cloud |
Output components
With Essential components it is only possible to send the intermediate and output data to an already deployed object storage like AWS S3, these components expand this to allow more options.
Name | Description | Alternatives |
Storage system that behaves like a AWS S3 object storage and allows the data to be hosted on private cloud and local machines. | Amazon S3 Ceph cluster | |
Allows using Google Cloud, Microsoft Azure and a few other storage providers to store RADAR-base data by providing an AWS S3 compatible API. |
Support components
These components enable logging and monitoring of application to make sure they’re always accessible and healthy. Installation is optional but advised. For all but Fluent-bit and Velero, the components could be run in a separate server or cluster to perform external monitoring.
Name | Description |
Log management system that allows receiving and viewing application logs with search and long term storage capabilities. | |
Database system used by Graylog to store its configuration. | |
Search engine used by Graylog to store log data. | |
Data collection software that sends applications logs to Graylog. | |
For monitoring and managing Kafka cluster via a web dashboard. | |
Monitoring bundle that includes Prometheus for checking health of the applications, Alertmanager to send alerts in case an application is failing and Grafana as a dashboard to check health of the cluster via graphs. | |
Used for backing up the cluster and its data to an external object storage. |
Manual upload components
These components are needed in case data is collected by outside of RADAR-base and allow users to manually upload them to RADAR-base data pipeline. All of these components should be active for manual upload to work.
Name | Description |
Backend webservice application for upload system that stores uploaded data and its metadata in PostgreSQL for later processing. | |
Web application that provides a UI for uploading files and sending them to the upload-backend. | |
Used for reading uploaded data from backend and sending them to Kafka cluster for later processing. | |
Temporary storage for uploaded data until deleted by the researcher or by an automated deletion policy. |
Fitbit components
These components enable data collection from Fitbit devices.
Name | Description |
| |
Portal to authorize the Fitbit connector to read data from Fitbit accounts. | |
Backend for the RADAR Rest sources authorizer |
Grafana components
These component enable a dashboard mainly to visualise data collection compliance.
Name | Description |
| |
Shows a dashboard of collected data, usually by showing a heatmap. | |
Database to store time-series data from Kafka topics. TimescaleDB is an extension of PostgreSQL. |
Other components
These components are used situationally to enable more integration or customization features
Name | Description |
Used for integrating REDCap surveys with Radar-base. | |
Used to redirect incoming requests related to Schema registry to Confluent cloud Kafka cluster. | |
Mobile app configuration engine with per-project and per-user configuration. | |
Frontend web UI for App Config | |
Provides a layer to monitor and analyze streams of wearable data and write data to storage. | |
Provides a layer to monitor and analyze streams of wearable data and write data to storage. | |
Provides facilities to store user-related app information and scheduling of push notifications using Firebase Cloud Messaging. This is mostly used to support the RADAR-Questionnaire app. |