Component overview and breakdown

Descriptions on this page may not align with target audience and might be too technical or too simple for them.

Essential components

These components are core components of Radar-base and should always be active.

Name	Description	Alternatives
Cert Manager	Creates SSL/TLS certificates to access the web applications. Uses Letsencrypt certificates by default.
Catalog Server	Creates RADAR-base topics in Kafka, registers schemas in Schema Registry and keeps a catalog of available source types.
Kafka	All real-world data in RADAR-base goes through Kafka, an advanced message broker. Data is stored there for processing by other applications, like real-time analysis or long-term data storage. Data remains for a limited time (usually a month).	Confluent Cloud Amazon MSK
Management Portal	Portal to manage projects and participants throughout RADAR-base.
Nginx Ingress	Web server used for accessing web applications.
PostgreSQL	Database used to store Management Portal data.	Amazon RDS for PostgreSQL Dedicated PostgreSQL server
Radar Output	Reads data from an intermediate object storage and converts it to a RADAR-base oriented output directory, by organizing it by project, user and collection date.
Radar S3 connector	Reads raw participants data from Kafka and puts them into an intermediate object storage like AWS S3 or Minio.
Redis	Used by Radar Output to coordinate data processing in a parallel way.
RADAR Gateway	REST Gateway to Kafka, for incoming participant data. It performs authentication, authorization, content validation and decompression.
SMTP	Email server for sending registration email to researchers and participants.
Kafka Schema Registry	Registry of Avro Schema’s used in a RADAR-base installation. Validation layer on top of Kafka to structurally verify incoming data.	Confluent Cloud
Zookeeper	Use for Kafka cluster management and coordination.	Confluent Cloud

Output components

With Essential components it is only possible to send the intermediate and output data to an already deployed object storage like AWS S3, these components expand this to allow more options.

Name

Description

Alternatives

Minio

Storage system that behaves like a AWS S3 object storage and allows the data to be hosted on private cloud and local machines.

Amazon S3

Ceph cluster

S3-proxy

Allows using Google Cloud, Microsoft Azure and a few other storage providers to store RADAR-base data by providing an AWS S3 compatible API.

Support components

These components enable logging and monitoring of application to make sure they’re always accessible and healthy. Installation is optional but advised. For all but Fluent-bit and Velero, the components could be run in a separate server or cluster to perform external monitoring.

Name	Description
Graylog	Log management system that allows receiving and viewing application logs with search and long term storage capabilities.
Mongodb	Database system used by Graylog to store its configuration.
Elasticsearch	Search engine used by Graylog to store log data.
Fluent Bit	Data collection software that sends applications logs to Graylog.
Kafka Manager	For monitoring and managing Kafka cluster via a web dashboard.
Kube Prometheus Stack	Monitoring bundle that includes Prometheus for checking health of the applications, Alertmanager to send alerts in case an application is failing and Grafana as a dashboard to check health of the cluster via graphs.
Velero	Used for backing up the cluster and its data to an external object storage.

Manual upload components

These components are needed in case data is collected by outside of RADAR-base and allow users to manually upload them to RADAR-base data pipeline. All of these components should be active for manual upload to work.

Name	Description
RADAR Upload Connect Backend	Backend webservice application for upload system that stores uploaded data and its metadata in PostgreSQL for later processing.
RADAR Upload Connect Frontend	Web application that provides a UI for uploading files and sending them to the upload-backend.
RADAR Upload Source Connector	Used for reading uploaded data from backend and sending them to Kafka cluster for later processing.
RADAR Upload Postgresql	Temporary storage for uploaded data until deleted by the researcher or by an automated deletion policy.

Fitbit components

These components enable data collection from Fitbit devices.

Name	Description
RADAR Fitbit Connector	Collects data from participants via the Fitbit Web API.
RADAR REST Sources Authorizer	Portal to authorize the Fitbit connector to read data from Fitbit accounts.
RADAR REST Sources Backend	Backend for the RADAR Rest sources authorizer

Grafana components

These component enable a dashboard mainly to visualise data collection compliance.

Name	Description
RADAR JDBC Connector	Stores data from selected Kafka topics into a TimescaleDB database.
RADAR Grafana	Shows a dashboard of collected data, usually by showing a heatmap.
TimescaleDB	Database to store time-series data from Kafka topics. TimescaleDB is an extension of PostgreSQL.

Other components

These components are used situationally to enable more integration or customization features

Name	Description
RADAR Integration	Used for integrating REDCap surveys with Radar-base.
cc-schema-registry-proxy	Used to redirect incoming requests related to Schema registry to Confluent cloud Kafka cluster.
App Config	Mobile app configuration engine with per-project and per-user configuration.
App Config Frontend	Frontend web UI for App Config
RADAR Backend Monitor	Provides a layer to monitor and analyze streams of wearable data and write data to storage.
RADAR Backend Stream	Provides a layer to monitor and analyze streams of wearable data and write data to storage.
RADAR Appserver	Provides facilities to store user-related app information and scheduling of push notifications using Firebase Cloud Messaging. This is mostly used to support the RADAR-Questionnaire app.