DART-MX8M GSTREAMER: Difference between revisions
No edit summary |
|||
(14 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
{{INIT_RELEASE_PARAM|RELEASE_SUMO_V1.4_DART-MX8M-MINI}}<!-- | |||
--> {{#lst:Yocto_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!-- | --> {{#lst:Yocto_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!-- | ||
--> {{#lst:B2QT_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!-- | --> {{#lst:B2QT_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!-- | ||
--> {{#lst:Debian_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!-- | |||
--> {{PageHeader|{{#var:HARDWARE_NAME}} Streaming Video}} {{DocImage|category1=Yocto|category2=Debian}}[[Category:{{#var:HARDWARE_NAME}}]] __toc__ | --> {{PageHeader|{{#var:HARDWARE_NAME}} Streaming Video}} {{DocImage|category1=Yocto|category2=Debian}}[[Category:{{#var:HARDWARE_NAME}}]] __toc__ | ||
{{GSTREAMER_INTRO}} | |||
=== gst-inspect === | === gst-inspect === | ||
Line 21: | Line 12: | ||
Lists all available sink elements: | Lists all available sink elements: | ||
# gst-inspect-1.0 | grep sink | # gst-inspect-1.0 | grep sink | ||
For example: | For example: | ||
<pre> | <pre> | ||
Line 691: | Line 683: | ||
= Test Pattern Stream = | = Test Pattern Stream = | ||
{{#vardefine:DEFAULT_SINK | {{#varexists:USE_KMSSINK | kmssink can-scale=false | autovideosink }} }} | |||
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.<br> | Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.<br> | ||
Default test pattern: | Default test pattern: | ||
# gst-launch-1.0 videotestsrc ! | # gst-launch-1.0 videotestsrc ! {{#var:DEFAULT_SINK}} | ||
}} | |||
[[File:gst1.jpg]]<br> | [[File:gst1.jpg]]<br> | ||
<br> | <br> | ||
Test pattern with specific parameters on the input element: | Test pattern with specific parameters on the input element: | ||
# gst-launch-1.0 videotestsrc pattern=circular ! | # gst-launch-1.0 videotestsrc pattern=circular ! {{#var:DEFAULT_SINK}} | ||
[[File:gst2.jpg]] | [[File:gst2.jpg]] | ||
Line 725: | Line 715: | ||
Example of encoding and saving a short video stream from a camera to an H.265-encoded TS file: | Example of encoding and saving a short video stream from a camera to an H.265-encoded TS file: | ||
# gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! | # gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_hevc ! mpegtsmux ! filesink location=./test_h265.ts | ||
}} | }} | ||
Line 745: | Line 735: | ||
{{#ifeq: {{#var:HARDWARE_NAME}} | VAR-SOM-MX8M-NANO | | | {{#ifeq: {{#var:HARDWARE_NAME}} | VAR-SOM-MX8M-NANO | | | ||
{{#vardefine:PLAYBACK_VIDEO_CMD1 | {{#varexists:USE_KMSSINK | gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 video-sink="imxvideoconvert_g2d ! queue ! kmssink can-scale=false" audio-sink="alsasink" | gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 }} }} | |||
{{#vardefine:PLAYBACK_VIDEO_CMD2 | {{#varexists:USE_KMSSINK | gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! imxvideoconvert_g2d ! video/x-raw,width=800,height=480 ! kmssink can-scale=false | gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! autovideosink }} }} | |||
<br> | <br> | ||
Example of a video file playback using the playbin element (making use of the VPU decoder): | Example of a video file playback using the playbin element (making use of the VPU decoder): | ||
# | # {{#var:PLAYBACK_VIDEO_CMD1}} | ||
Example of a video file playback while explicitly specifying a pipeline (making use of the VPU decoder): | Example of a video file playback while explicitly specifying a pipeline (making use of the VPU decoder): | ||
# | # {{#var:PLAYBACK_VIDEO_CMD2}} | ||
}} | }} | ||
= RTP Video streaming = | = RTP Video streaming = | ||
{{GSTREAMER_RTP}} | |||
= References = | = References = | ||
{{GSTREAMER_REF}} | |||
Latest revision as of 15:07, 21 March 2023
This page is using the default release RELEASE_SUMO_V1.4_DART-MX8M-MINI.
To view this page for a specific Variscite SoM and software release, please follow these steps:
- Visit variwiki.com
- Select your SoM
- Select the software release
Introduction to GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for NXP's i.MX platform, which make use of the i.MX multimedia capabilities.
Major GStreamer commands
There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.
gst-inspect
Lists all available elements:
# gst-inspect-1.0
Lists all available sink elements:
# gst-inspect-1.0 | grep sink
For example:
# gst-inspect-1.0 | grep sink alsa: alsasink: Audio sink (ALSA) waylandsink: waylandsink: wayland video sink shm: shmsink: Shared Memory Sink decklink: decklinkaudiosink: Decklink Audio Sink decklink: decklinkvideosink: Decklink Video Sink ximagesink: ximagesink: Video sink soup: souphttpclientsink: HTTP client sink autodetect: autovideosink: Auto video sink autodetect: autoaudiosink: Auto audio sink gdkpixbuf: gdkpixbufsink: GdkPixbuf sink playback: playsink: Player Sink video4linux2: v4l2sink: Video (video4linux2) Sink fbdevsink: fbdevsink: fbdev video sink tcp: tcpclientsink: TCP client sink tcp: tcpserversink: TCP server sink …
Detailed help information on a specific element:
# gst-inspect-1.0 <name of the element>
For example, the output for waylandsink:
# gst-inspect-1.0 waylandsink Factory Details: Rank primary + 2 (258) Long-name wayland video sink Klass Sink/Video Description Output to wayland surface Author Sreerenj Balachandran <sreerenj.balachandran@intel.com>, George Kiagiadakis <george.kiagiadakis@collabora.com> Plugin Details: Name waylandsink Description Wayland Video Sink Filename /usr/lib/gstreamer-1.0/libgstwaylandsink.so Version 1.14.0 License LGPL Source module gst-plugins-bad Source release date 2018-03-19 Binary package GStreamer Bad Plug-ins source release Origin URL Unknown package origin GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseSink +----GstVideoSink +----GstWaylandSink Implemented Interfaces: GstVideoOverlay GstWaylandVideo Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR, (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU, (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9, (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] video/x-raw(memory:DMABuf) format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR, (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU, (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9, (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' Element Properties: name : The name of the object flags: readable, writable String. Default: "waylandsink0" parent : The parent of the object flags: readable, writable Object of type "GstObject" sync : Sync on the clock flags: readable, writable Boolean. Default: true max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited) flags: readable, writable Integer64. Range: -1 - 9223372036854775807 Default: 20000000 qos : Generate Quality-of-Service events upstream flags: readable, writable Boolean. Default: true async : Go asynchronously to PAUSED flags: readable, writable Boolean. Default: true ts-offset : Timestamp offset in nanoseconds flags: readable, writable Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 enable-last-sample : Enable the last-sample property flags: readable, writable Boolean. Default: true last-sample : The last sample received in the sink flags: readable Boxed pointer of type "GstSample" blocksize : Size in bytes to pull per buffer (0 = default) flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 4096 render-delay : Additional render delay of the sink in nanoseconds flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 throttle-time : The time to keep between rendered buffers (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 max-bitrate : The maximum bits per second to render (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 show-preroll-frame : Whether to render video frames during preroll flags: readable, writable Boolean. Default: true window-width : Wayland sink preferred window width in pixel flags: readable, writable Integer. Range: -1 - 2147483647 Default: -1 window-height : Wayland sink preferred window height in pixel flags: readable, writable Integer. Range: -1 - 2147483647 Default: -1 display : Wayland display name to connect to, if not supplied via the GstContext flags: readable, writable String. Default: null alpha : Wayland surface alpha value, apply custom alpha value to wayland surface flags: readable, writable Float. Range: 0 - 1 Default: 0 enable-tile : When enabled, the sink propose VSI tile modifier to VPU flags: readable, writable Boolean. Default: false
The i.MX8M-MINI VPU video decoder element
# gst-inspect-1.0 vpudec Factory Details: Rank primary + 1 (257) Long-name IMX VPU-based video decoder Klass Codec/Decoder/Video Description Decode compressed video to raw data Author Multimedia Team <shmmmw@freescale.com> Plugin Details: Name vpu Description VPU video codec Filename /usr/lib/gstreamer-1.0/libgstvpu.so Version 4.4.4 License LGPL Source module imx-gst1.0-plugin Binary package Freescle Gstreamer Multimedia Plugins Origin URL http://www.freescale.com GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstVideoDecoder +----GstVpuDec Pad Templates: SRC template: 'src' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] SINK template: 'sink' Availability: Always Capabilities: video/x-h265 video/x-vp9 video/x-vp8 video/x-h264 Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: name : The name of the object flags: readable, writable String. Default: "vpudec0" parent : The parent of the object flags: readable, writable Object of type "GstObject" output-format : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG) flags: readable, writable Enum "GstVpuDecOutputFormat" Default: 0, "auto" (0): auto - enable chroma interleave. (default) (1): NV12 - NV12 format (2): I420 - I420 format (3): YV12 - YV12 format (4): Y42B - Y42B format (5): NV16 - NV16 format (6): Y444 - Y444 format (7): NV24 - NV24 format frame-drop : enable adaptive frame drop for smoothly playback flags: readable, writable Boolean. Default: true frame-plus : set number of addtional frames for smoothly playback flags: readable, writable Unsigned Integer. Range: 0 - 16 Default: 3 use-vpu-memory : use vpu allocate video frame buffer flags: readable, writable Boolean. Default: true
The i.MX8M-MINI VPU H.264 video encoder element
# gst-inspect-1.0 vpuenc_h264 Factory Details: Rank primary + 1 (257) Long-name IMX VPU-based AVC/H264 video encoder Klass Codec/Encoder/Video Description Encode raw data to compressed video Author Multimedia Team <shmmmw@freescale.com> Plugin Details: Name vpu Description VPU video codec Filename /usr/lib/gstreamer-1.0/libgstvpu.so Version 4.4.4 License LGPL Source module imx-gst1.0-plugin Binary package Freescle Gstreamer Multimedia Plugins Origin URL http://www.freescale.com GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstVideoEncoder +----vpuenc_h264 Implemented Interfaces: GstPreset Pad Templates: SRC template: 'src' Availability: Always Capabilities: video/x-h264 stream-format: { (string)avc, (string)byte-stream } alignment: { (string)au, (string)nal } SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)I420, (string)YUY2, (string)UYVY, (string)RGBA, (string)RGBx, (string)RGB16, (string)RGB15, (string)BGRA, (string)BGRx, (string)BGR16 } width: [ 64, 1920, 8 ] height: [ 64, 1088, 8 ] framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: name : The name of the object flags: readable, writable String. Default: "vpuenc_h264-0" parent : The parent of the object flags: readable, writable Object of type "GstObject" qos : Handle Quality-of-Service events from downstream flags: readable, writable Boolean. Default: false bitrate : set bit rate in kbps (0 for automatic) flags: readable, writable Unsigned Integer. Range: 0 - 2147483647 Default: 0 gop-size : How many frames a group-of-picture shall contain flags: readable, writable Unsigned Integer. Range: 0 - 32767 Default: 30 quant : set quant value: H.264(0-51) (-1 for automatic) flags: readable, writable Integer. Range: -1 - 51 Default: -1
Test Pattern Stream
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:
# gst-launch-1.0 videotestsrc ! autovideosink
Test pattern with specific parameters on the input element:
# gst-launch-1.0 videotestsrc pattern=circular ! autovideosink
Camera Loopback
Stream video from a camera to a display:
# gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink
Video Recording
The i.MX8M-MINI can encode videos to AVC(H.264) & VP8 at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).
See the src pad capabilities in the vpuenc_264 and vpuenc_vp8 elements details for the complete list of features supported by the i.MX8M-MINI VPU encoder.
Example of encoding and saving a short video stream from a camera to an H.264-encoded AVI file:
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_h264 ! avimux ! filesink location=./test_h264.avi
Example of encoding and saving a short video stream from a camera to an MJPEG-encoded AVI file (using the SW based jpegenc element - without using the VPU):
# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test_jpeg.avi
Video Playback
The i.MX8M-MINI can decode and playback HEVC(H.265), AVC(H.264), VP9, VP8, etc. encoded videos at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).
See the sink pad capabilities in the vpudec element details for the complete list of codecs supported by the i.MX8M-MINI VPU decoder.
Example of a video file playback using the playbin element (making use of the VPU decoder):
# gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4
Example of a video file playback while explicitly specifying a pipeline (making use of the VPU decoder):
# gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! autovideosink
RTP Video streaming
Encode video stream from camera using a software based JPEG encoder and stream it over RTP:
# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! \ jpegenc ! rtpjpegpay ! udpsink host=192.168.20.60 port=1223
References
- The GStreamer website
- NXP BSP Linux Users Guide, Multimedia section download link