Implementation of a High Speed Data Streaming System
Initial situation and project task
When testing applications are implemented today, a growing demand for data recording and playback at high bandwidths can be realized. Manufacturer from the telecommunications, automotive and radar industry must have the ability to test their products reproducible regarding their receiver quality. For making this possible, a robust, modular high speed data streaming system has been developed in cooperation with Hochschule Niederrhein, University of Applied Sciences. This system is suitable for laboratory use as well as for mobile applications.
Faster computers and innovative multi-channel and broad-band data acquisition systems lead to an increasing demand for storage and administration of huge amounts of data. Until now, expensive and limited onboard memory of data acquisition cards has been used for data storage, so that the duration of recorded and generated broadband streams has been highly limited. Resolutions of 14 and 16 bits as well as sampling rates of 50MS/s to not allow for continuous data streaming by means of existing PC technology without extensive modification.
Implementation and challenge
Based on the open platform PXI and StreamStor technology, an operational system for high-speed data acquisition and playback could be implemented. StreamStor technology itself is based on a hard disk controller and a raid system. Special drivers allow for direct access to data, stored to the memories of cards for data acquisition and generation. To achieve this, the integrated PXI controller has been bypassed, so that the full bandwidth of the PXI bus can be used for data transfer. At the same time, two problems are solved: Firstly, data has not to be transported over the PXI bus twice (acquisition and storage). Secondly, the CPU does not have to process all recorded data, because raw data is directly written to the raid system. Therefore, the system allows for a maximum continuous data rate of 100 MByte/s.
Data handling is rather difficult, because raw data is stored to the raid system, which can only be handled as a whole. By means of NI’s LabVIEW a File Allocation Table (FAT) as well as a file handling system have been developed, which are exactly tailored for the StreamStor system.
Due to a high degree of PXI bus utilization by real-time streaming, it is not possible to record data from any other sources. By means of a synchronized file, which is stored to the embedded controller and which includes FAT information, recoding parameters and Meta data, streaming administration is possible.
Additional GPS data can be stored to the integrated hard disk of the PXI controller. This offers numerous opportunities to detect dependencies for recorded signals and additional information such as Meta data, environment data and comments. Additional Backup, Restore and Editor Functionality for user data allow for a user-friendly, interactive administration of streaming data. By this means, it is also possible, to integrate further interfaces into the system, such as CAN, MOST, LIN.
When testing applications are implemented today, a growing demand for data recording and playback at high bandwidths can be realized. Manufacturer from the telecommunications, automotive and radar industry must have the ability to test their products reproducible regarding their receiver quality. For making this possible, a robust, modular high speed data streaming system has been developed in cooperation with Hochschule Niederrhein, University of Applied Sciences. This system is suitable for laboratory use as well as for mobile applications.
Faster computers and innovative multi-channel and broad-band data acquisition systems lead to an increasing demand for storage and administration of huge amounts of data. Until now, expensive and limited onboard memory of data acquisition cards has been used for data storage, so that the duration of recorded and generated broadband streams has been highly limited. Resolutions of 14 and 16 bits as well as sampling rates of 50MS/s to not allow for continuous data streaming by means of existing PC technology without extensive modification.
Implementation and challenge
Based on the open platform PXI and StreamStor technology, an operational system for high-speed data acquisition and playback could be implemented. StreamStor technology itself is based on a hard disk controller and a raid system. Special drivers allow for direct access to data, stored to the memories of cards for data acquisition and generation. To achieve this, the integrated PXI controller has been bypassed, so that the full bandwidth of the PXI bus can be used for data transfer. At the same time, two problems are solved: Firstly, data has not to be transported over the PXI bus twice (acquisition and storage). Secondly, the CPU does not have to process all recorded data, because raw data is directly written to the raid system. Therefore, the system allows for a maximum continuous data rate of 100 MByte/s.
Data handling is rather difficult, because raw data is stored to the raid system, which can only be handled as a whole. By means of NI’s LabVIEW a File Allocation Table (FAT) as well as a file handling system have been developed, which are exactly tailored for the StreamStor system.
Due to a high degree of PXI bus utilization by real-time streaming, it is not possible to record data from any other sources. By means of a synchronized file, which is stored to the embedded controller and which includes FAT information, recoding parameters and Meta data, streaming administration is possible.
Additional GPS data can be stored to the integrated hard disk of the PXI controller. This offers numerous opportunities to detect dependencies for recorded signals and additional information such as Meta data, environment data and comments. Additional Backup, Restore and Editor Functionality for user data allow for a user-friendly, interactive administration of streaming data. By this means, it is also possible, to integrate further interfaces into the system, such as CAN, MOST, LIN.
Conclusion and future prospects
By means of the high speed data streaming system it is possible to record and playback signals with a maximum bandwidth of 200MHz. Due to suitable mixer technologies, it is possible to choose the operational frequency range freely within a range of 250 kHz up to 2.7 GHz. This accounts for a broad field of application for the streaming system.
The system can be directly upgraded as an automotive RF tester, being optimized for testing broadcast radio receivers for automobiles. It is possible to record the whole FM range (88MHZ – 108MHz) including additional information during a test drive and to playback the recorded signal in a laboratory, so that reproducible testing scenarios can be generated.
Developers of navigation or mobile multimedia devices require real-world conditions for validating and testing their products. Up to now, the chances for achieving this have been very restricted, so that time-consuming and elaborate test drives were needed. Now, it is possible to store all characteristics and faults of a capture area into a test profile and to playback the signal real-time in a laboratory.
Offering a maximum recording time of 7.5 hours, the system allows for the generation of a significant portfolio of testing scenarios including important recording for AM, FM and DAB.
Currently, further modifications of the system are under development, in order to integrate innovative technologies such as PXIe. An increased data rate is as possible as parallel applications by means of a combination of PXI and PXIe. Further information on this subject you will find with the URT and with our latest RF record & playback systems.
By means of the high speed data streaming system it is possible to record and playback signals with a maximum bandwidth of 200MHz. Due to suitable mixer technologies, it is possible to choose the operational frequency range freely within a range of 250 kHz up to 2.7 GHz. This accounts for a broad field of application for the streaming system.
The system can be directly upgraded as an automotive RF tester, being optimized for testing broadcast radio receivers for automobiles. It is possible to record the whole FM range (88MHZ – 108MHz) including additional information during a test drive and to playback the recorded signal in a laboratory, so that reproducible testing scenarios can be generated.
Developers of navigation or mobile multimedia devices require real-world conditions for validating and testing their products. Up to now, the chances for achieving this have been very restricted, so that time-consuming and elaborate test drives were needed. Now, it is possible to store all characteristics and faults of a capture area into a test profile and to playback the signal real-time in a laboratory.
Offering a maximum recording time of 7.5 hours, the system allows for the generation of a significant portfolio of testing scenarios including important recording for AM, FM and DAB.
Currently, further modifications of the system are under development, in order to integrate innovative technologies such as PXIe. An increased data rate is as possible as parallel applications by means of a combination of PXI and PXIe. Further information on this subject you will find with the URT and with our latest RF record & playback systems.