DART-MX8M GSTREAMER: Difference between revisions
From Variscite Wiki
| Line 5: | Line 5: | ||
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.<br> | It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.<br> | ||
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.<br> | This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.<br> | ||
This release uses the [https://source.codeaurora.org/external/imx/gstreamer/ gstreamer-imx] set of GStreamer 1.0 plugins for | This release uses the [https://source.codeaurora.org/external/imx/gstreamer/ gstreamer-imx] set of GStreamer 1.0 plugins for NXP's i.MX platform, which make use of the i.MX multimedia capabilities. | ||
== Major GStreamer commands == | == Major GStreamer commands == | ||
Revision as of 17:23, 14 November 2018
DART-MX8M - Streaming Video
Introduction to GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for NXP's i.MX platform, which make use of the i.MX multimedia capabilities.
Major GStreamer commands
There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.
gst-inspect
Lists all available elements:
$ gst-inspect-1.0
Lists all available sink elements:
$ gst-inspect-1.0 | grep sink
For example:
root@imx8m-var-dart:~# gst-inspect-1.0 | grep sink alsa: alsasink: Audio sink (ALSA) waylandsink: waylandsink: wayland video sink shm: shmsink: Shared Memory Sink decklink: decklinkaudiosink: Decklink Audio Sink decklink: decklinkvideosink: Decklink Video Sink ximagesink: ximagesink: Video sink soup: souphttpclientsink: HTTP client sink autodetect: autovideosink: Auto video sink autodetect: autoaudiosink: Auto audio sink gdkpixbuf: gdkpixbufsink: GdkPixbuf sink playback: playsink: Player Sink video4linux2: v4l2sink: Video (video4linux2) Sink fbdevsink: fbdevsink: fbdev video sink tcp: tcpclientsink: TCP client sink tcp: tcpserversink: TCP server sink …
Detailed help information on a specific element:
$ gst-inspect-1.0 <name of the element>
For example, the output for kmssink:
root@imx8m-var-dart:~# gst-inspect-1.0 kmssink
Factory Details:
Rank secondary (128)
Long-name KMS video sink
Klass Sink/Video
Description Video sink using the Linux kernel mode setting API
Author Víctor Jáquez <vjaquez@igalia.com>
Plugin Details:
Name kms
Description Video sink using the Linux kernel mode setting API
Filename /usr/lib/gstreamer-1.0/libgstkms.so
Version 1.12.2
License LGPL
Source module gst-plugins-bad
Source release date 2017-07-14
Binary package GStreamer Bad Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSink
+----GstVideoSink
+----GstKMSSink
Implemented Interfaces:
GstVideoOverlay
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)BGRA, (string)BGRx, (string)RGBA, (string)RGBx, (string)UYVY, (string)YUY2, (string)YVYU,
(string)I420, (string)YV12, (string)Y42B, (string)NV12, (string)NV12_10LE, (string)NV21, (string)NV16 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element Flags:
no flags set
Element Implementation:
Has change_state() function: gst_kms_sink_change_state
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "kmssink0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
sync : Sync on the clock
flags: readable, writable
Boolean. Default: true
max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
flags: readable, writable
Integer64. Range: -1 - 9223372036854775807 Default: 20000000
qos : Generate Quality-of-Service events upstream
flags: readable, writable
Boolean. Default: true
async : Go asynchronously to PAUSED
flags: readable, writable
Boolean. Default: true
ts-offset : Timestamp offset in nanoseconds
flags: readable, writable
Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
enable-last-sample : Enable the last-sample property
flags: readable, writable
Boolean. Default: true
last-sample : The last sample received in the sink
flags: readable
Boxed pointer of type "GstSample"
blocksize : Size in bytes to pull per buffer (0 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
render-delay : Additional render delay of the sink in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
throttle-time : The time to keep between rendered buffers (0 = disabled)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
max-bitrate : The maximum bits per second to render (0 = disabled)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
show-preroll-frame : Whether to render video frames during preroll
flags: readable, writable
Boolean. Default: true
driver-name : DRM device driver name
flags: readable, writable
String. Default: null
connector-id : DRM connector id
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
plane-id : DRM plane id
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
force-modesetting : When enabled, the sink try to configure the display mode
flags: readable, writable
Boolean. Default: false
global-alpha : global alpha
flags: readable, writable
Integer. Range: 0 - 255 Default: 0
force-hantrotile : When enabled, the sink propose hantro tile modifier to VPU
flags: readable, writable
Boolean. Default: false
The i.MX VPU video decoder element
root@imx8m-var-dart:~# gst-inspect-1.0 vpudec
Factory Details:
Rank primary + 1 (257)
Long-name VPU-based video decoder
Klass Codec/Decoder/Video
Description Decode compressed video to raw data
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name vpu.imx
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.3.4
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstVpuDec
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h265
video/x-vp9
video/x-vp8
video/x-vp6-flash
video/x-h264
video/mpeg
systemstream: false
mpegversion: { (int)1, (int)2 }
video/mpeg
mpegversion: 4
video/x-h263
video/x-flash-video
flvversion: 1
video/x-divx
divxversion: 3
video/x-divx
divxversion: 4
video/x-divx
divxversion: { (int)5, (int)6 }
video/x-xvid
video/x-cavs
video/x-wmv
wmvversion: 3
format: WMV3
video/x-wmv
wmvversion: 3
format: WVC1
video/x-pn-realvideo
image/jpeg
image/webp
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element Flags:
no flags set
Element Implementation:
Has change_state() function: gst_video_decoder_change_state
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "vpudec0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
output-format : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG)
flags: readable, writable
Enum "GstVpuDecOutputFormat" Default: 0, "auto"
(0): auto - enable chroma interleave. (default)
(1): NV12 - NV12 format
(2): I420 - I420 format
(3): YV12 - YV12 format
(4): Y42B - Y42B format
(5): NV16 - NV16 format
(6): Y444 - Y444 format
(7): NV24 - NV24 format
frame-drop : enable adaptive frame drop for smoothly playback
flags: readable, writable
Boolean. Default: true
frame-plus : set number of addtional frames for smoothly playback
flags: readable, writable
Unsigned Integer. Range: 0 - 16 Default: 3
use-vpu-memory : use vpu allocate video frame buffer
flags: readable, writable
Boolean. Default: true
Test Pattern Stream
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:
$ gst-launch-1.0 videotestsrc ! kmssink
Test pattern with specific parameters on the input element
$ gst-launch-1.0 videotestsrc pattern=circular ! kmssink
Camera Loopback
Stream video from a camera to a display
$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
Video Recording
Save video stream from camera as an MJPEG-encoded AVI file
$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test0.avi
Video Playback
Example of a video file playback using the playbin element (making use of the i.MX8M VPU decoder)
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4
Example of a video file playback while explicitly specifying a pipeline (making use of the i.MX8M VPU decoder)
$ gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! kmssink
RTP Video streaming
Encode video stream from camera using a software jpeg encoder and stream it over RTP
$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.20.60 port=1223
References
- The GStreamer website
- NXP BSP Linux Users Guide, Multimedia section download link