VAR-SOM-MX6 GSTREAM FSLC: Difference between revisions
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
{{PageHeader|VAR-SOM-MX6 - Streaming Video}} {{DocImage|category1=VAR-SOM-MX6|category2=Yocto}} __toc__ | {{PageHeader|VAR-SOM-MX6 - Streaming Video}} {{DocImage|category1=VAR-SOM-MX6|category2=Yocto}} __toc__ | ||
= Introduction to GStreamer = | = Introduction to GStreamer = | ||
Line 14: | Line 5: | ||
<br/>GStreamer is a powerful pipeline-based multimedia framework. | <br/>GStreamer is a powerful pipeline-based multimedia framework. | ||
<br/>GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products. | <br/>GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products. | ||
<br>This release is based on [https://github.com/Freescale/gstreamer-imx/blob/master/README.md GStreamer-I.MX] community project that allows utilizing internal I.MX VPU and GPU capabilities. | |||
== Major GStreamer commands == | == Major GStreamer commands == |
Revision as of 16:55, 6 April 2017
Introduction to GStreamer
GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.
This release is based on GStreamer-I.MX community project that allows utilizing internal I.MX VPU and GPU capabilities.
Major GStreamer commands
There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.
gst-inspect
Lists all available elements:
$ gst-inspect-1.0
Lists all available i.MX6 specific elements:
$ gst-inspect-1.0 | grep imx
For example:
root@var-som-mx6:~# gst-inspect-1.0 | grep imx aiur.imx: webm: [Invalid UTF-8] 8\x91\xddq aiur.imx: aiurdemux: Aiur universal demuxer vpu.imx: vpuenc_h264: VPU-based AVC/H264 video encoder vpu.imx: vpuenc_mpeg4: VPU-based MPEG4 video encoder vpu.imx: vpuenc_h263: VPU-based H263 video encoder vpu.imx: vpuenc_jpeg: VPU-based JPEG video encoder vpu.imx: vpudec: VPU-based video decoder overlaysink.imx: overlaysink: IMX Video (video compositor) Sink imxvideoconvert.imx: imxvideoconvert_ipu: IMX ipu Video Converter imxvideoconvert.imx: imxvideoconvert_g2d: IMX g2d Video Converter imxv4l2.imx: imxv4l2sink: IMX Video (video4linux2) Sink imxv4l2.imx: imxv4l2src: IMX Video (video4linux2) Source beep.imx: ac3: [Invalid UTF-8] 4J\xa0t beep.imx: 3ca: [Invalid UTF-8] 4J\xa0t beep.imx: beepdec: Beep universal decoder imxcompositor.imx: imxcompositor_ipu: IMX ipu Video Compositor imxcompositor.imx: imxcompositor_g2d: IMX g2d Video Compositor imxmp3enc.imx: imxmp3enc: imx mp3 audio encoder
Detailed help information on a specific element:
$ gst-inspect-1.0 imxv4l2sink
For example:
root@var-som-mx6:~# gst-inspect-1.0 imxv4l2sink ====== IMXV4L2SINK: 4.0.9 build on Aug 8 2016 12:47:08. ====== Factory Details: Rank primary + 1 (257) Long-name IMX Video (video4linux2) Sink Klass Sink/Video Description Displays frames on IMX SoC video4linux2 device Author Multimedia Team <shmmmw@freescale.com> Plugin Details: Name imxv4l2.imx Description IMX SoC v4l2-based video source/sink Filename /usr/lib/gstreamer-1.0/libgstimxv4l2plugins.so Version 4.0.9 License LGPL Source module imx-gst1.0-plugin Binary package Freescle Gstreamer Multimedia Plugins Origin URL http://www.freescale.com GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseSink +----GstVideoSink +----GstImxV4l2Sink Implemented Interfaces: GstVideoOverlay Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition) format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] Element Flags: no flags set Element Implementation: Has change_state() function: 0x76a0b808 Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' Element Properties: name : The name of the object flags: readable, writable String. Default: "imxv4l2sink0" parent : The parent of the object flags: readable, writable Object of type "GstObject" sync : Sync on the clock flags: readable, writable Boolean. Default: true max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited) flags: readable, writable Integer64. Range: -1 - 9223372036854775807 Default: 20000000 qos : Generate Quality-of-Service events upstream flags: readable, writable Boolean. Default: true async : Go asynchronously to PAUSED flags: readable, writable Boolean. Default: true ts-offset : Timestamp offset in nanoseconds flags: readable, writable Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 enable-last-sample : Enable the last-sample property flags: readable, writable Boolean. Default: true last-sample : The last sample received in the sink flags: readable Boxed pointer of type "GstSample" blocksize : Size in bytes to pull per buffer (0 = default) flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 4096 render-delay : Additional render delay of the sink in nanoseconds flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 throttle-time : The time to keep between rendered buffers (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 max-bitrate : The maximum bits per second to render (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 show-preroll-frame : Whether to render video frames during preroll flags: readable, writable Boolean. Default: true device : Device location flags: readable, writable String. Default: "/dev/video17" overlay-top : The topmost (y) coordinate of the video overlay; top left corner of screen is 0,0 flags: readable, writable Integer. Range: -2147483648 - 2147483647 Default: 0 overlay-left : The leftmost (x) coordinate of the video overlay; top left corner of screen is 0,0 flags: readable, writable Integer. Range: -2147483648 - 2147483647 Default: 0 overlay-width : The width of the video overlay; default is equal to screen width flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 0 overlay-height : The height of the video overlay; default is equal to screen height flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 0 crop-top : The topmost (y) coordinate of the video crop; top left corner of image is 0,0 flags: readable, writable Unsigned Integer. Range: 0 - 2147483647 Default: 0 crop-left : The leftmost (x) coordinate of the video crop; top left corner of image is 0,0 flags: readable, writable Unsigned Integer. Range: 0 - 2147483647 Default: 0 crop-width : The width of the video crop; default is equal to negotiated image width flags: readable, writable Unsigned Integer. Range: 0 - 2147483647 Default: 0 crop-height : The height of the video crop; default is equal to negotiated image height flags: readable, writable Unsigned Integer. Range: 0 - 2147483647 Default: 0 rotate : The orientation degree of the video; default is 0 degree flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 0 force-aspect-ratio : When enabled, scaling will respect original aspect ratio flags: readable, writable Boolean. Default: false deinterlace : set deinterlace enabled; can't be configed on fly flags: readable, writable Boolean. Default: true motion : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion. flags: readable, writable Unsigned Integer. Range: 0 - 2 Default: 2 reconfig : Change V4L2 configuration while running; overlay position/size/rotation changed. flags: readable, writable Boolean. Default: false composition-meta-enable: Enable overlay composition meta processing flags: readable, writable Boolean. Default: false
gst-launch
For Example launch the camera and display on the monitor:
$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink
i.MX6 major elements
Prefix |
Element Name |
description |
---|---|---|
imxmp3enc.imx | imxmp3enc | imx mp3 audio encoder |
imxv4l2.imx | imxv4l2sink | IMX Video (video4linux2) Sink |
overlaysink.imx | overlaysink | IMX Video (video compositor) Sink |
imxv4l2.imx | imxv4l2src | IMX Video (video4linux2) Source |
vpu.imx | vpudec | VPU-based video decoder |
vpu.imx | vpuenc_h264 | VPU-based AVC/H264 video encoder |
vpu.imx | vpuenc_h263 | VPU-based H263 video encoder |
vpu.imx | vpuenc_mpeg4 | VPU-based MPEG4 video encoder |
vpu.imx | vpuenc_jpeg | VPU-based JPEG video encoder |
imxvideoconvert.imx | imxvideoconvert_ipu | IMX ipu Video Converter |
imxvideoconvert.imx | imxvideoconvert_g2d | IMX g2d Video Converter |
imxcompositor.imx | imxcompositor_ipu | IMX ipu Video Compositor |
imxcompositor.imx | imxcompositor_g2d | IMX g2d Video Compositor |
Test Pattern Stream
Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:
$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink
Test pattern with specific paramters on the input element
$ gst-launch-1.0 videotestsrc pattern=circular ! imxv4l2sink
Test pattern with specific paramters on the output element
$ gst-launch-1.0 videotestsrc ! imxv4l2sink overlay-width=1280 overlay-height=720
Test pattern with specific paramters on the input and output elements
$ gst-launch-1.0 videotestsrc pattern=circular ! imxv4l2sink overlay-width=1280 overlay-height=720
Camera Loopback
Stream video from a camera to a display
Command:
gst-launch-1.0 imxv4l2src ! 'video/x-raw, format=(string)$FORMAT, width=$WIDTH, height=$HEIGHT, framerate=(fraction)30/1' ! imxv4l2sink
For example:
$ gst-launch-1.0 imxv4l2src device=/dev/video0 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1'! imxv4l2sink overlay-width=1920 overlay-height=1080
This is a 3 elements pipe,the element in the middle is a GStreamer capsfilter element.
Parameter comments:
- • Get the camera support format and resolution using gst-inspect-1.0 imxv4l2src.
- • Set capsfilter according to the camera's supported capabilities if the user needs other format or resolution.
- • Ensure that the right caps filter has been set, which also needs to be supported by imxv4l2sink.
ov5640 example resolutions:
- width=320,height=240
- width=640,height=480
- width=720,height=480
- width=720,height=576
- width=1280,height=720
- width=1920,height=1080
Video Record
command:
gst-launch-1.0 imxv4l2src device=$DEVICE num-buffers=300 ! $INPUT_CAPS ! queue ! vpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION
Parameter comments:
- • $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.
- • num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)
- • INPUT_CAPS should be set according to camera format and resolution
- • vpuenc_XXX can be vpuenc_mpeg4,vpuenc_h263, vpuenc_h264, and vpuenc_jpeg
- • MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux
- • EXTENSION is filename extension according to the muxer type.
Record video from a camera into a file.
720p:
$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1280,height=720,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi
1080p:
$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi
Simple Movie Play
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4