Time Series Data Management System   (TSDMS)

TSDMS Solutions Blog Pricing About Support

 

Time Series Data Solutions by Application

 

ECU Log

An electronic control unit (ECU) or electronic control module (ECM) is an embedded system in various motorized vehicles such as automotive, agriculture, aerospace, and marine vessels.   Frequently they have the capability to log the data measured by sensors throughout the product's system, and the command signals from the ECU.   For example, the ECU Data Storage in the Northwest UAV (unmanned aerial vehicle) NW-88 engine records up to 1000 hours at 1 Hz recording rate   (source).  

A custom DataPlugin can be easily created using the DIAdem Wizard to read the proprietary data format.   Thereafter, the TDMS Bulk Import application makes it easy to import thousands of ECU log files in one batch, extracting metadata from the ECM log filename and the folder where it resides, and then enriching the imported files with that metadata.   Custom metadata can also be added to each batch of files, making it easy to later search and find the data of interest.  

Learn More

 

 

CAN/LIN Bus Data

Modern vehicles use a combination of Controller Area Network (CAN) bus and Local Interconnect Network (LIN) bus communication networks.   FlexRay bus is an emerging technology for high-speed synchronized data communications.   Each type of network sends data over a twisted pair of wires as messages.  

Data Loggers are available from a number of suppliers for the recording of the CAN/LIN messages transmitted on the respective networks.   High end data acquisition equipment from suppliers such as HBM/SomMat (eDAQ) and DeweSoft can also read CAN/LIN data.  

NI DataPlugins and the Bus Log Converter in DIAdem can be used to read a variety of file formats from data loggers and data acquisition (DAQ) devices.   The CAN/LIN/FlexRay bus log conversion within the TDMS Bulk Import application makes it easy to decode files with several CAN, LIN, and FlexRay buses.  

See Example

 

Sensor Data

What you are measuring has a minimum interval (or highest frequency) that the sensor will need to be measured, and the recording device will need to capture.   Nearly all mechanical systems have events with a duration of 0.01 sec or more (or less than 100 Hz), and many are less than 30 Hz.  

Most sensors have a time constant that reflects the amount of time it takes for the sensor to get to 63% of the final value when exposed to a step change.   The sensor response time is the time it takes for a sensor to reach 99.3% of the total step change in the measured values.   The response time is five times (5x) the time constant value.   Your sensor response time should be less than the duration of the event you are trying to measure.  

You need to choose a recording sample rate that has 1/10th shorter duration (or 10x higher rate in samples/sec or Hz) than what you intend to measure.   For example, you are monitoring the fluid pressure of a component in a manufacturing process.   The pressure sensor has a response time of 50 milliseconds.   The component experiences pressure changes as quickly as 100 milliseconds.   The sample rate would be 1/(0.1 sec)*10 = 100 samples per second or 100 Hz.  

The continuous streaming of a single channel of data over a distance by wire or wireless is limited by latency.   And you will find that many IoT cloud solutions prefer a message rate of every 1.0 seconds or longer.   In general, continuous streaming should not be considered as an option unless the channel sample rate is on the order of 1 Hz or less (see Continuous Data Streaming.   Beyond that, you need to consider the use of a data logger or data acquisition (DAQ) device that will record the data to some sort of portable media.   If you are remotely recording data relative to where the data will be stored long term and analyzed, you will need a system for getting that data logger / DAQ device recorded data from the remote location.   Although the recorded data will likely be compressed, it can still take a considerable amount of time to transfer full day recording of many channels over the internet (and hopefully over a VPN or other secure system) from the remote location to the final destination.  

Once the sensor data gets to it's final long term storage and analysis destination, you will need an application to manage and extract value from that data.   NI DIAdem's DataPlugin technology allows you to easily import sensor data from any recording source and format.   If the recorded filename contains a date/time stamp, then this information can be extracted and added as metadata to the converted files by Bulk Import   Analyzing, and generating visual reports from many batches of sensor data files from many locations is easy to process using Bulk Analysis and Bulk Report.  

 

Field Test

Field testing a product can capture valuable data about how your product behaves in real world situations.   Preparing a product for field testing takes extensive planning and preparation in order to insure all the information desired is captured and fully documented (see Planning for Field Test Success).  

The data collected may include the logging of a vehicle's CAN and/or LIN bus.   This data is extremely valuable, but unless you plan for a way to capture what the activity the vehicle was engaged in, and relate it to the recorded data, you may miss the opportunity to differentiate clearly one activity from the next.   Recording where (GPS coordinates) the vehicle is can sometimes be very helpful.   If the weather can influence your product's behaviour, then it can also be useful to know record the date/time using a real time clock or GPS so that ambient conditions can later be imported and added to your data.   Think carefully about how often and how you will be transferring data from the field test vehicle to where the data will finally be stored and analyzed (see Big Data Big Distance Transfers).  

If you are continuously monitoring something remotely, and your sample rate requirements are low, then you may be in a position to continuously stream the data from the field.   Otherwise, it will be necessary to come up with a schedule and method for transferring the data from the field to the final long term storage and data analysis location.  

The TSDMS Bulk Import tool makes it extremely easy to decode recorded CAN/LIN bus data.   It is also an ideal tool for quickly importing batches of data from any remote monitored test specimen.   Bulk Analysis enables you to quickly analyze that imported data, and Bulk Report will help you quickly visualize the performance of your test subject in the field.  

 

Laboratory Test

The close proximity of the test subject to the final destination for recorded data from a laboratory test provides an advantage over the field test situation.   However, the laboratory test data typically consists of many test scenarios, many more channels, and some at very high sample rates.   It is critical that all of the test operating conditions, test subject identification, and other factors are documented with the data in a way that can be easily used as search critera later to find the data of interest.   Bulk Import can dramatically simplify the process of capturing the metadata of all those operating condtions and enriching the imported data files with that information.   Using the custom metadata feature in Bulk Import, you can custom write a JSON file with hundreds of details about your test, and then quickly apply those conditions to all of the files in the batch to be imported.   By customizing the filename and folder path contents, you can also automatically extract critical information per file and include it in the imported data files.  

As an example, suppose you are performing a cold room test on a device.   You could manually edit the custom metadata JSON file to include all of the general static test information for the tests performed in one day such as the date of the test, the device serial number, etc.   As you vary the environmental contions of the cold room, or the inputs to the item under test, you can customize the storage of the recorded file by adding metadata about those conditions to the folder path and filename(s).   By carefully planning the capture of the metadata about your test, you can leverage this to make it easy for Bulk Import to extract this data and automatically enrich your data files.  

 

NI LabVIEW

LabVIEW is a graphical programming environment used to develop automated test systems.   The LabVIEW TDM Streaming VIs allows you to read/write TDMS files from your LabVIEW application.   The NI Data Storage VIs write data to the TDM file format.  

Using the TSDMS applications Bulk Import, Bulk Analysis and Bulk Report makes it easy to quickly import, analyze, and generate reports from data saved from LabVIEW applications.  

 

NI FlexLogger

FlexLogger is a software application for fast sensor configuration and data logging.   The simple graphical user interface is easy to use and requires no programming.   FlexLogger is designed to capture metadata during data recording.  

Using the TSDMS applications Bulk Analysis and Bulk Report makes it easy to quickly analyze the recorded FlexLogger data and generate visualization reports.  

For End Users
For Businesses
Other

 

Copyright © 2021,2022 Mechatronic Solutions LLC, All Rights Reserved