Abstract
This document describes different mechanisms available for recording the images that an operator on a rugged system sees on their computer screen. The options are compared to emphasize the types of application and end-user requirements where each method might be most suited.
1.
Screen recording requirements
1.1 Introduction
This white paper considers alternative methods for recording what an operator is seeing on their computer monitor for whatever mission equipment they are using. The computer systems driving the monitor varies, which affects the available options for a particular system.
1.2 Reasons for wanting to record the screen
In order to decide which recording method is most suitable for a particular application, it is important to understand the underlying reasons for the requirement.
Does the user want to review the detail of the video images that the operator is seeing? Often the high-resolution video streams are already being recorded elsewhere. For instance, is it important that the HD video image is recorded in full resolution (this question is particularly pertinent when the video image is in a window on the screen as is often the case)?
Or does the user only want to review what actions the operator took, at different times, and review the stimulus for those actions? This could be the case for a training scenario, but is also true for many real mission applications, especially if the video sources are already being recorded in full resolution. Often, the screen recording is required for ‘just in case’ situations – being able to show what the operators did at various times, if that is ever needed.
Often the reasons are unclear, so there can be tendency to over-specify the requirement, and require full resolution, full frame rate video recording. That’s perfectly easy to achieve, but results in a lot of recording equipment, power dissipation, and data storage. It certainly isn’t a SWaP-C (Size Weight and Power, and Cost) optimised system.
1.2.1 Recording compressed vs uncompressed video
One factor worth considering in defining requirements is whether the video signal can be compressed for recording. Typically, compression is done using H.264 or H.265. These standards reduce the a HD signal (1920×1080 @ 25/30 Hz) down to around 2-5MBytes/s of data.
In compressing the video signal, some of the video data content is lost. However, the H.264 and
H.265 standards are designed to ensure that the content which is lost is mostly invisible to the human eye, so the impact on a viewer is negligible or in many cases invisible.
For machine analysis, an uncompressed video recording may be essential, but that type of analysis of video signal is more likely to be performed on the recording of the camera gimbal signal, rather than on the view that the operator had (incorporating overlays, windowing, etc.).
1.2.2 Video formats
To try to define what the real requirements are, it is important to consider the resolutions and frame rates being used for the different sources and for the screen image itself.
A set of likely video types are listed below for reference:
Table 1, Video formats
| Video type | Resolutions Frame rates Typical uses |
|---|---|
| HD-SDI (Full HD) | 1920x1080 25Hz / 30Hz |
| HD-SDI | 1280x720 25Hz / 30Hz |
| 3G-SDI | Uncommon, but may be 1920x1080 50Hz / 60Hz used for some camera gimbals |
| DVI1 / HDMI1 / Display port | Typically 60Hz Up to 1920x12002 Computer systems (Up to 144Hz) |
| VGA, SVG, XGA, etc. (RGBHV, 5 wire analog) | Up to 1920x12003 Typically 60Hz Computer systems |
| Composite (CVBS) NTSC / PAL (1 wire analog) | 7203x486 / 50 / 60Hz field rate, |
| S-Video NTSC / PAL (Y/C 2 wire analog) | 7203x486 / 50 / 60Hz field rate, 7203x576 25 / 30Hz frame rate Legacy cameras |
1 Note that DVI (DVI-D) and HDMI at the basic level use the same signalling wires for the digital video channel; the versions which are claimed as either HDMI or DVI in aircraft grade computers are often not actually compliant to either specification. That is because the specifications include a definition of the connector which should be used, and may systems use rugged connectors instead.
2 Note that HDMI, DVI and Display port also support higher resolutions up to 4k video signals (3820×2160 at 60Hz), but that is uncommon for rugged equipment at this time.
3 Note that the analog video standards (RGBHV, NTSC, PAL, S-Video) don’t have a fixed number of pixels per line within the video signal – the pixels are defined by the digitisation process (for display or for recording). The number of pixels per line is often (but not always) chosen to ensure that the pixels are square.
The specification of the video sources which may be viewed by the operator on the screen has an immediate impact on the recording requirements.
For instance, if the video sources are running at 25 or 30Hz frame rate, is there much real value in recording the complete DVI signal which is driving the monitor, running at 60Hz?
Another factor to consider is how many operator screens are needing to be recorded. DVI / HDMI / Display Port are not well suited to long distance connections in a rugged environment. These video standards can work well for a simple computer to display connection when the display is physically close to the computer. But they won’t work so well for trying to connect many computers to a single video recorder.
2.
Screen recording methods
There are multiple ways to record what the operator sees on their screen. The main ones are listed below, and described in more details in sections here:
- 2nd video output from the computer driving directly into a recorder
- Buffered video signal, so the recorder receives the same image as the monitor/display
- Using the computer to record its own video output. Typically a compressed video image is recorded, and at a reduced frame
- Using the computer to send a compression version of the screen output over Ethernet to a NAS or Ethernet recorder. Typically a compressed video image is recorded, and at a reduced frame
2.1 Full video signal recording
For this mode of recording, the full video signal going to the monitor/display is recorded. In many cases this is a DVI signal, running at 60Hz. The recorder will normally compress the video signal as part of the recording functionality, although uncompressed recording is also possible. Three different methods of getting the video signal to the recorder (and also to the monitor/display) are shown below. All 3 methods result in the same thing being recorded – a practically identical signal to the one which is seen by the operator on their display.
This method can work well for a single screen recording (i.e. 1 operator screen to be recorded).
But with multiple operator’s screens, there are some problems:
- Cable lengths get longer (DVI, HDVI and Display Port are not well suited to long cable lengths on rugged platforms)
- Recorder physical size increases (multiple DVI connections/connectors takes up a lot of space)
- Data storage requirements are relatively high (this is often not problem if the video signal to be compressed before storage
2.2 Local computer system recording
This involves the computer doing the recording itself, as shown below:
The computer runs an additional program which grabs the screen image periodically.
Various options for the screen ‘grab’ software/utilities – e.g. ffmpeg is free and provides all the functionality likely to be required, and has a low overhead on processor cycles. Ffmpeg can be used in conjunction with other utilities as described here: https://trac.ffmpeg.org/wiki/Capture/Desktop
The frame rate of the recording can be adjusted to suit the requirements of a particular system, so the resulting data storage requirements can be much smaller than with the full video signal recording (section 2.1 above). A higher frame rate will create more data, but will capture more high frequency activity, whereas a lower frame rate will use fewer processor cycles, and creates less data. Similarly, it is possible to select lower resolutions with similar tradeoffs between processor cycles, quality of image, and quantity of data.
The computer software (and the requirement to store the data) takes some computer cycles (depending on the processor subsystem used as well as the recording frame rate selected).
One disadvantage to this method of recording is that the computer must have some sort of removable storage module, or the ability to support fast copy of the recorded video/screen data to an external mobile storage device at the end of the mission.
Just like with the full video signal recording option (section 2.1 above), this method doesn’t scale very well for when multiple operators’ stations need to be recorded.
2.3 Ethernet-based recording
This involves the computer doing video ‘grabbing’ and compressing, and sending the resulting
data to a NAS or Ethernet recorder, as shown below:
This option uses the same screen ‘grab’ software/utilities as the local recording (section 2.2 above), but instead of the computer needing to support local data storage, the data is sent over Ethernet to a NAS.
Using a low overhead utility running the computer, the system is able to send a low bandwidth UDP Ethernet stream of the screen image into the network. By using a low frame rate (e.g.
5Hz), the data bandwidth is extremely low, and the quality of the video is still extremely good for review purposes, with full resolution (although compressed) images being stored.
This method scales very well – if multiple operator stations need to be recorded, they only need to be connected to the same Ethernet network, and a single NAS/Ethernet recorder will store all of the video streams simultaneously.
So, all of the recorded data is in one place, and provided the recorder/NAS has removable storage, all of the data can easily be taken off the platform for copying to the base station computer system for post-mission review, analysis, and archiving.
Also, this method is completely independent of the video formats and resolutions being used. The recorder/NAS is connected by Ethernet. Therefore, for longer term programs, it is relatively easy to find solutions for obsolescence and other similar issues.
2.3.1 Multicast for peer screen viewing
With the video signal being streamed on Ethernet, it is easy for a separate computer to view the images. This is useful for training scenarios, or for peers to be able to view the same thing that one screen is viewing for easy communication.
2.3.2 Dual redundancy for recording
Similarly, with the video signal being streamed on Ethernet, it is possible for 2 (or more) recorders to grab the video stream, thus introducing a simple redundancy capability, if required.
3.
Comparisons
The 3 methods described in the previous chapter are compared/contrasted in the table below.
Table 2, Comparison Chart.
| Screen Recording Method | Advantages | Disadvantages |
|---|---|---|
| Full video screen recording | No processor overhead Full frame rate and resolution Uncompressed recording option | Larger storage requirements Larger recorder systems Doesn't scale well |
| Local computer system recording | No separate recorder required Reduced storage requirements | Processor cycles required for compression and recording Challenging recorded data offload Doesn't scale well (for offload) |
| Ethernet based recording | Scales well for any number of operator stations Independent of video signal format Reduced storage requirements Multicast for peer viewing possible Recorder dual redundancy possible | Processor cycles required for compression and sending data |
4.
Questions or Comments
This white paper will be updated as video technology adoption on rugged platforms evolves.
Any questions or comments to the contents are welcome and appreciated. Please contact Hugh Tarver, at htarver@galleonec.com or send your feedback to info@galleonec.com
5.
References
Reference material and potentially useful links are listed here https://trac.ffmpeg.org/wiki/Capture/Desktop