 |
1.10 |
|
1.10 |
EGT leverages GStreamer extensively for its multimedia features, simplifying user experience by automatically configuring GStreamer for common use cases.
To get started, familiarilize yourself with GStreamer concepts through the official tutorials.
GStreamer operates using pipelines composed of interconnected elements based on their capabilities. Typically, a source element reads the video, other elements can process it, and a sink element displays it.
The gst-launch-1.0 tool provided by GStreamer is useful for understanding and debugging pipelines. This tool page explains its usage and syntax pipeline description syntax.
For example, you can test video with the following command on a PC:
On a board:
EGT offers several multimedia components:
Each component integrates an instance of GstDecoderImpl, the core of GStreamer support within EGT.
EGT utilizes two GStreamer elements to manage inputs.
uridecodebin handles most inputs, including files and streams, by selecting a suitable source element and linking it to a decodebin element.
It is used by VideoWindow and AudioPlayer.
Special files as /dev/video0 are not managed by uridecodebin, but v4l2src.
It is used by CameraWindow and CameraCapture.
EGT supports several sink elements:
appsink to display videofilesink to record videoautoaudiosink to play the audiotrackWhile filesink and autoaudiosink usage is straightforward, the usage of appsink requires additional configuration from EGT.
The appsink element enables EGT to obtain frame buffers of the video. It is essential that appsink receives frames that matches the size and format of the EGT Window.
To ensure this, a capsfilter element is placed before the appsink element, specifying the required size and format and is achieved with the assistance of the videoconvert and videoscale elements. EGT then processes these frame buffers to update the display.
EGT automatically computes the pipeline description for the basic use cases it handles. To accommodate additional scenarios, a custom pipeline description can be provided to the GstDecoderImpl using the custom_pipeline method. When a custom pipeline is provided, EGT uses it as is, making it your responsibility to ensure the correct output for the appsink element.
Even with a custom pipeline, it is assumed that the output will be displayed in a Window. Therefore, the pipeline must include an appsink element named appsink and a capsfilter element named vcaps. If the Window is resized, EGT will reconfigure the capsfilter to match the new size.
Additionally, when using a custom pipeline, EGT assumes the presence of an audio track. To manage volume, the pipeline must include a volume element named volume.
When multimedia is not functioning correctly, since EGT is essentially a wrapper around GStreamer, the first step is to determine whether the issue lies at the GStreamer level. This can be accomplished through a few diagnostic steps.
The first step is to verify that the pipeline used by EGT, or the one you intend to use, functions correctly with GStreamer alone.
To obtain the pipeline description used by EGT, set the EGT_DEBUG environment variable to 2 or lower. This will generate an info debug line that indicates the pipeline description computed by EGT and provided to GStreamer.
For example, with the egt_player application on a board:
The pipeline cannot be used directly with gst-launch-1.0 due to the presence of the appsink element, which is designed for applications that handle video buffers. Instead, another sink must be used. Although this means the configuration will not be identical, it still helps verify that the rest of the pipeline functions correctly.
You can use kmssink with force-modesetting=true to utilize the base layer of the LCD. Ensure that you specify a pixel format supported by the base layer. The frame size must match the native size of the base layer. If you require a specific size, you will need to use the render-rectangle parameter.
Here is an example of a basic approach:
To more closely match the EGT configuration, you can use the high-end overlay with the same size and position:
You can retrieve the plane-id using the modetest command.
If you do not have a working pipeline with gst-launch-1.0, it is unlikely to work with EGT. Therefore, you must first resolve any issues at the GStreamer level or at a lower level, such as kernel drivers.
Another tool that can be utilized to diagnose issues is the GStreamer debug logs, which are controlled by the GST_DEBUG environment variable.
Typically, setting this variable to 4 provides sufficient information to identify problems. It is important to note that errors may not always be reported in the error-level logs; they can also appear in the info-level logs.
For more detailed information about debug logs, please refer to the GStreamer documentation.
It's possible to obtain a visual representation of the pipeline computed by GStreamer from the provided description. This can be useful for verifying the configuration of the elements.
This feature is available with both gst-launch-1.0 and EGT. To enable it, set the GST_DEBUG_DUMP_DOT_DIR environment variable to the directory where you want to store the graphs.
For example:
These files correspond to the pipeline graph at various pipeline state transitions, such as when the state changes from paused to playing. You can use the xdot software as a viewer for these dot files to visualize the pipeline.
© Copyright 2020 Microchip Technology Inc. All rights reserved.