GST pipelines prototype
The i.MX processors family from NXP supports GStreamer as the main multimedia framework. For more information about the GStreamer Framework, please refer to the following links:
To get started, please be sure the camera is connected to the carrier board. See a list of camera products:
All the following examples were extracted from the listed documentation above, if you want to learn more, please read those documents.
Camera
Testing Camera Preview on Display
The following GStreamer pipeline example is the simplest way to open and test a camera device:
# gst-launch-1.0 v4l2src device=/dev/video<X> ! video/x-raw,width=<WIDTH>,height=<HEIGHT> ! autovideosink sync=false
There are some parameters within <> that must be replaced in the above GStreamer pipeline. The first parameter is the device ID (/dev/video0, /dev/video1, …), which can be easily found by using the following V4L2 tool that outputs the available devices:
# v4l2-ctl --list-devices
The next parameters can be found with the following GStreamer tool, which is a very useful tool to get information such as camera resolution, framerate, and supported formats:
# gst-device-monitor-1.0
Quick Examples for Different Resolutions
- VGA 480p 640x480@30fps
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! autovideosink sync=false
- NTSC 720x480@30fps
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=720,height=480 ! autovideosink sync=false
- HD 720p 1280x720@30fps
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! autovideosink sync=false
- FULL HD 1080p: 1920x1080@30fps
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! autovideosink sync=false
- QSXGA 2592x1944@15fps
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=2592,height=1944 ! autovideosink sync=false
Testing Camera Snapshot
The following GStreamer pipeline example is the simplest way to capture JPEG snapshots:
# gst-launch-1.0 v4l2src device=/dev/video<X> num-buffers=1 ! video/x-raw,width=<WIDTH>,height=<HEIGHT> ! <PLUGIN> ! filesink location=/tmp/<file_name>.jpg
To fill the regular parameters within <>, use the same approach from the previous section. Replace <PLUGIN> to choose which snapshot format you want, such as JPG (jpegenc), PNG (pngenc), or JPEG (jpegenc).
Quick Examples for Snapshots (JPEG)
- VGA 480p 640x480
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! video/x-raw,width=640,height=480 ! jpegenc ! filesink location=/tmp/test_640x480.jpg
- NTSC 720x480
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! video/x-raw,width=720,height=480 ! jpegenc ! filesink location=/tmp/test_720x480.jpg
- HD 720p 1280x720
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=/tmp/test_1280x720.jpg
- FULL HD 1080p: 1920x1080
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! video/x-raw,width=1920,height=1080 ! jpegenc ! filesink location=/tmp/test_1920x1080.jpg
- QSXGA 2592x1944
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! video/x-raw,width=2592,height=1944 ! jpegenc ! filesink location=/tmp/test_2592x1944.jpg
Streaming Video Section
Test Pattern Stream
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer:
- Default Test Pattern
# gst-launch-1.0 videotestsrc ! autovideosink
- Test Pattern with Specific Parameters on the Input Element
# gst-launch-1.0 videotestsrc pattern=circular ! autovideosink
Camera Loopback
Stream Video from a Camera to a Display
# gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink
Video Recording
Encoding and Saving a Video Stream from a Camera to an MJPEG-encoded AVI
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=/tmp/record.avi
Video Playback
Video File Playback
- Download the Big Buck Bunny video
# wget https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4
- Run the video
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! autovideosink
RTP Streaming
Video Streaming from Module to VLC
On the client-side, create a file (example.sdp), then paste the following content into the file:
v=0 m=video 5000 RTP/AVP 96 c=IN IP4 10.0.0.100 a=rtpmap:96 H264/90000 a=fmtp:96 sprop-parameter-sets=Z2QAMqxyhEBQBbsBEAAAAwAQAAADA8DxgxhG\,aOhDgzLIsA\=\=
On line 3: Replace the IP with the server IP;
On line 5: Replace the sprop-parameter-sets value with the respective one. To get this value, run the pipeline below on the server-side.
Open the VLC tool:
vlc example.sdp
Then, just run the pipeline on the server-side:
gst-launch-1.0 -v filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux ! rtph264pay ! udpsink host=<PC_CLIENT_IP> port=5000
NOTE: Replace <PC_CLIENT_IP>.
Multiple Cameras
To display more the one camera at the same display output, use the following GStreamer pipeline changing accordingly. See a generic example below:
gst-launch-1.0 imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=640 sink_0::height=480 sink_1::xpos=0 sink_1::ypos=480 sink_1::width=640 sink_1::height=480 sink_2::xpos=640 sink_2::ypos=0 sink_2::width=640 sink_2::height=480 sink_3::xpos=640 sink_3::ypos=480 sink_3::width=640 sink_3::height=480 ! video/x-raw,format=RGB16 ! waylandsink v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! comp.sink_0 v4l2src device=/dev/video1 ! video/x-raw,width=640,height=480 ! comp.sink_1 v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480 ! comp.sink_2 v4l2src device=/dev/video3 ! video/x-raw,width=640,height=480 ! comp.sink_3
Extra Section
Video Scaling
To display different scaling results use the glimagesink - render_rectangle property.
- For VGA Resolution
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink render-rectangle='<0, 0, 720, 480>'
- For Full HD Resolution
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink render-rectangle='<0, 0, 1920, 1080>'
Video Rotation
To rotate the video results use the glimagesink - rotate-method property:
- To Rotate 90 Degrees
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink rotate-method=1
- To Rotate 180 Degrees
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink rotate-method=2
- To Rotate 270 Degrees
# gst-launch-1.0 filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! glimagesink rotate-method=3
Debugging
This section describes some debug tool functionalities.
- Buffer Movement
# GST_DEBUG=GST_BUFFER:5 gst-launch-1.0 -v filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! decodebin ! waylandsink sync=false
As you may notice, the video performance was affected by the console log return. To avoid it, keep the debug values in a file:
- Buffer Movement
# GST_DEBUG_FILE=gst_debug.log GST_DEBUG=GST_BUFFER:5 gst-launch-1.0 -v filesrc location=/home/root/Big_Buck_Bunny_720_10s_1MB.mp4 ! decodebin ! waylandsink sync=false
Zero Copy
This section approaches the zero-copy operations using GStreamer pipelines. The zero-copy support is an operation in which the CPU uses the data produced by one element but without requiring any type of transformation. One way of using buffers to operate a zero-copy pipeline is to use properties such as io-mode. This process can be very helpful in order to improve the execution speed of a video processing pipeline.