File Types
Marple supports a wide variety of file types
CSV
Marple loves CSV files. They are simple, easy to read, and everyone uses them! Most parsing options can be selected in the import wizard, but there are a couple minimum requirements though to get your data into Marple:
Numeric data (Marple will ignore non-numeric data points)
One row of signal names
(Optional) One row of units
One column that defines the 'time' axis, can be selected in Marple
Time can be a number or date string format. (preferably
YYYY/MM/DD hh:mm:ss)
Example
Zipped CSV
Sometimes there are multiple time axes, or the data is sampled at different sample rates.
In those cases, a CSV file is not a handy format... you might think!
Typically you can solve this by creating a set of CSV files, one for each time base (eg. sampling rate) and wrapping those together in a ZIP file. Marple will recognize this and use the same algorithms to decode the individual CSV files, but organize all data so you can easily visualise signals of different sample rate all at once.
Example
Download the example-csv.zip file below:
HDF5
HDF5 files are widely known for their excellent handling of large data sets.. and we agree! We love HDF5 data sets!
Unfortunately, HDF5 files come in different shapes and formats, so we need to agree to some structure.
Marple supports 2 structures for HDF5. We call it SIGNAL based and MATRIX based structures.
SIGNAL based (flat)
In a signal-based structure, the data consists of Groups (dictionary-like objects) where the signal names are used as keys. For every signal a Dataset (array) defines the data.
We also allow for signal grouping, as can be seen in the example below
Units are also supported, by adding them as an attribute to the Dataset for the signal.
Download the example_signalbased.hdf5 file below:
MATRIX based
A matrix structure is a bit more complex to construct but allows for easier row-by-row processing. It also allows for extra information such as units.
Download the example_hdf5_matrix.h5 file below:
MAT
MAT files are used a lot when using MATLAB or certain loggers such as dSpace, Speedgoat, .. MAT files are treated similarly to HDF5 files (as described above), so the same rules apply.
If you have a MAT file of version 7.3 or higher, this will be treated as a HDF5 file. MAT files of a lower version are significantly less efficient and will require more effort to be parsed by Marple.
Download the example.mat file below:
ULOG
The ULOG format is often used when dealing with systems that operate using the PX4 software or ecosystem. We think it's a great format because it is standardized!
As long as you adhere to the ULOG structure, .ulg formats are plug & play for Marple. No configuration needed!
Download the example.ulg file below:
ROS / Rosbag
In Robotics the ROS - Robot Operating System is often used. These systems log files with extension .bag.
Marple is able to parse this data as long as the values are numeric. Image, text or binary data can not be visualised. Marple will convert the different topics into different signal groups and will show this in the UI.
MDF / MF4
In the automotive industry, MDF or MF4 files are often used. Measurement Data Format version 4 (MDF 4) is a standard file format by ASAM used by the automotive industry for storing measurement data in binary file format. Marple is able to read non-binary MDF / MF4 files. Marple will recognize the different signal groups and show them in the UI. Note that a binary MDF file cannot be parsed but needs to converted using a .dbc file first.
TDMS
TDMS is a structured, binary file format developed by National Instruments. Marple is able to parse this data if channels within a group have the same length, and each group has only 1 time signal.
MAVLINK
MAVLink is a very lightweight messaging protocol for communicating with drones. The messages can be stored in a file, often with a .bin extension and can be easily imported by Marple.
AVRO
Avro is a row-oriented serialization framework that is used in Apache Hadoop. Avro files store data in binary format but it’s syntax definition resembles a JSON file. Here is an example of what a .avro file JSON structure should look like to be supported by Marple:
Note that the “Unit” field is optional and that other fields such as “Data Type” can be present but will be ignored. If you want help want with a custom Avro format that doesn’t resemble this, reach out to our technical team at support@marpledata.com.
Last updated
