1 What is Streamlab?

Streamlab is a sophisticated software suite specially crafted for multimodal experiments, utilizing the advanced capabilities of the lab streaming layer (LSL), as outlined in Lab Streaming Layer Documentation. This suite consists of two integral parts: StreamLive and StreamView.

StreamLab stands out as the world’s first multimodal platform based on LSL, integrating a wide range of data sources like Eye Tracking, EEG, GSR/EDA, ECG, and EMG with exceptional flexibility. This integration is particularly valuable for advancing the understanding of behavioral studies.

Together, StreamLive and StreamView support the complete workflow, from live observation and data recording to detailed data analysis. This cohesive functionality makes StreamLab an all-encompassing solution for professionals and researchers in various fields, streamlining the complexities associated with handling and interpreting multi-dimensional data.

StreamLive serves as a live observation and recording tool. It empowers users to connect to any compatible devices on local networks, then synchronize, display, and record their data streams on a unified time base. While it primarily employs LSL for synchronization, StreamLive also incorporates specific adaptations for managing video streams, ensuring comprehensive and cohesive data integration.

StreamView acts as a specialized analysis tool for recordings made with StreamLive. It provides capabilities to replay StreamLive recordings, insert custom markers, and apply filters. Additionally, it enables users to segment and trim the recordings, focusing on pertinent sections, and export the processed data in various file formats for diverse applications.

2 Streamlive

2.1 The device interface

Streamlive is structured around an extensive list of supported device models. Each compatible model is conveniently catalogued and can be easily accessed in the ‘Device’ column, ensuring users have a clear overview of all the devices compatible with Streamlive.

2.1.1 Discovery

To initiate an automatic search for any device of the specified model on your local network, simply click on the ‘Discover’ button. This search will run seamlessly in the background and is designed to complete within 10 seconds, although the exact duration may vary slightly depending on the model of the device.

Upon completion of the search, the ‘Search’ button will reactivate. If the search successfully locates any devices corresponding to the specified model, they will be automatically integrated into the Streamlive interface.

streams tab — multiple devices of the same model can be found

If no devices are detected, you may receive a model-specific message offering guidance and suggested steps to help troubleshoot common connection issues. This feature is designed to assist you in resolving potential challenges efficiently.

2.1.2 Streams

Every device is associated with several streams, which represent the various data streams emanating from that particular device. These streams may consist of either a single channel or multiple channels. Within any given stream, all channels maintain a consistent sampling rate. When a stream is sampled, a data point is generated for each channel, accompanied by a shared timestamp, ensuring synchronized data collection across all channels.

The streams and their associated channels for a specific device can be accessed from the “Stream” section located beneath the device’s listing. By unchecking a box, you can disable a stream or a channel. This action means that no data will be recorded from the disabled stream or channel, allowing for selective data collection as per your requirements.

2.1.3 Views

Views serve as visual indicators designed to showcase data from one or more streams. For each device, there is a curated list of views, accessible under its ‘View’ tab, which are specifically tailored to effectively display data from its individual streams.

views tab — The views for a device, connected to the devices streams.

Certain views within the software are designed to incorporate data from additional devices. For instance, many of the graphical displays are capable of including markers that have been transmitted from Streamlive, offering a more comprehensive data visualization experience.

To activate a view in Streamlive, simply drag and drop it into the viewpane. Users have the flexibility to move, close, resize, and rearrange these views to create a customized layout that best suits the specific needs of their experiments. This feature allows for a personalized and efficient workspace setup.

2.1.4 The “Others” device

The “Others” device category in Streamlive serves a unique purpose. It is designed to aggregate all streams that are either generated internally by Streamlive or do not correspond to any external devices. This special category ensures a comprehensive and organized collection of various data streams within the Streamlive environment.

_images/others_device.png — an “Others” device holding the streamlive marker stream and webcam streams

Currently, this feature is utilized for managing the Streamlive marker stream and webcam streams. However, there might be a need to expand its capabilities to accommodate additional types of streams in the future. When users send markers from the bottom bar, these will be prominently displayed and recorded under the “Markers” stream of this device, ensuring easy tracking and reference.

2.2 Recording

When the user initiates the recording process by pressing the ‘Record’ button, Streamlive efficiently generates a recording folder at the designated location. This folder is conveniently named using the timestamp of when the recording began. Following this, Streamlive begins the process of capturing and storing all incoming data streams in memory, ensuring a seamless and organized recording experience.

_images/recording_folders.png — different recordings inside a destination folder

When the user concludes the recording session, all the captured data is meticulously saved to a data file within the designated recording folder, formatted as per the user’s preference. Additionally, supplementary elements such as video files or PsychoPy stimuli are individually stored in their respective files, all neatly organized within the same folder for easy access and reference.

2.2.1 Folder structure

_images/inside_recording_folder.png — The uppermost directory of a recording is known as the recording’s folder. Within this folder, there’s a data file bearing an extension that reflects the format selected by the user. This file encompasses all the numeric or textual data from the streams.

Additionally, the recording’s folder includes a concealed .store subfolder. While this folder is typically not required for end-user interaction, it is essential for Streamview to access and load the recording. As such, it should not be removed.

_images/store_folder.png — content of the hidder .store folder

The main data file can be written to different formats according to the user’s need.

2.2.2 CSV

The CSV (Comma-Separated Values) file format is highly favored in psychology research due to its simplicity, wide support across all programming languages, and human readability. Its non-binary nature, however, means that it occupies significantly more space and requires more time for writing and reading compared to many other file formats, particularly when dealing with large volumes of numeric data. Consequently, for lengthy recordings with streams at very high sampling rates, such as extended EEG (Electroencephalography) recordings, the CSV format may not be the most efficient choice.

Streamlive offers two variations of its CSV format: aligned and unaligned. In the aligned version, a single timestamps column is used to represent timestamps for all streams. While this approach is more user-friendly, it results in numerous empty cells in each line, which substantially increases the file’s size and impacts the time required for writing and reading. This factor is important to consider when choosing the appropriate format for your data needs.

_images/aligned_csv.png — structure of an aligned csv recording with one common timestamps column

In the unaligned version, each stream is equipped with its own column for timestamps. This design minimizes the presence of empty spaces. However, it delegates the responsibility of realigning the timestamps to the user.

_images/unaligned_csv.png — structure of an unaligned csv recording with one timestamps column for each stream

In both version, each column correspond to a stream’s channel. Their name is formated as such.

f"{model_name} {device_name} {stream_name} {channel_name} data"

2.2.3 PKL

the .pkl format is the native format of python’s pickle module. Its a fast and lightweight binary format that conserve the hierarchical structure of the recording (model -> device -> stream -> samples) . The data contained within a .pkl file can loaded in any python programm as easily as this :

with open("my_recording.pkl","rb") as f:
    my_data = pickle.load(f)

3 Streamview

Streamview is meticulously crafted to replay, edit, and export recordings from Streamlive. Its design closely mirrors that of Streamlive, featuring a similar approach to data representation and an analogous interface for devices and streams.

Both Streamview and Streamlive utilize a consistent layout for the device column and view pane, ensuring a seamless user experience across both programs. Additionally, the majority of indicators available in StreamLive are also compatible with Streamview, further enhancing the continuity and usability between these two components of the suite.

3.1 Filters

Streamview provides the capability to selectively filter a recording, allowing users to retain only specific segments based on a variety of criteria. This feature enhances the precision and relevance of data analysis.

For exemple, streamlive markers:

_images/filter_marker.png — the user has added a filter between “marker 1” and “marker 2” to the filter list. Once applied, only parts of the recording contained between those two markers will be kept.

Psychopy stimuls:

Users have the option to select a stimulus created using the PsychoPy SDK. The software will then retain only the segment of the recording that spans from the beginning of this selected stimulus’s presentation to the onset of the subsequent stimulus’s presentation.

By hand:

Creating any kind of filter will result in the creation of bounds comprised of a start and end timestamp. Upon applying the filters. the list of all active bounds, corresponding to the applied filters, will be used to filter the recording applied and only data that fall within the bounds will be kept

Filtering the data does not affect the original recording folder. Changes will only be recorded upon export.

3.2 Export

Once filters have been applied, the user can export the modified recording with the same structure as a StreamLive recording. For now it is only possible to export the main data file to csv but other options will be added.

Exported folders behave like streamlive recordings and can be loaded normally into Streamview

_images/export.png — An exported recording. Similar to a Streamlive recording