VAR-SOM-MX6 GSTREAM JETHRO: Difference between revisions

From Variscite Wiki
 
(26 intermediate revisions by the same user not shown)
Line 199: Line 199:
=== gst-launch ===
=== gst-launch ===
For Example launch the camera and display on the monitor:
For Example launch the camera and display on the monitor:
<pre>gst-launch-1.0 -v videotestsrc ! imxv4l2sink</pre>
<pre>$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink</pre>


=== i.MX6 major elements ===
=== i.MX6 major elements ===
Line 208: Line 208:
! scope="col" | description<br/>
! scope="col" | description<br/>
|-
|-
| mp3enc.imx
| imxmp3enc.imx
| mfw_mp3encoder
| imxmp3enc
| mp3 audio encoder
| imx mp3 audio encoder
|-
|-
| v4lsink.imx
| imxv4l2.imx
| imxv4l2sink  
| imxv4l2sink  
| v4l2 video sink
| IMX Video (video4linux2) Sink
|-
|-
| isink.imx
| overlaysink.imx
| mfw_isink
| overlaysink
| IPU-based video sink
| IMX Video (video compositor) Sink
|-
|-
| v4lsrc.imx
| imxv4l2.imx
| imxv4l2src
| imxv4l2src
| v4l2 based camera src
| IMX Video (video4linux2) Source
|-
|-
| vpu.imx
| vpu.imx
Line 228: Line 228:
| VPU-based video decoder
| VPU-based video decoder
|-
|-
|vpu.imx  
| vpu.imx  
|vpuenc
| vpuenc_h264
|VPU-based video encoder
| VPU-based AVC/H264 video encoder
|-
| vpu.imx
| vpuenc_h263
| VPU-based H263 video encoder
|-
|-
| vpu.imx
| vpuenc_mpeg4
| VPU-based MPEG4 video encoder
|-
|-
| vpu.imx
| vpuenc_jpeg
| VPU-based JPEG video encoder
|-
|-
| imxvideoconvert.imx
| imxvideoconvert_ipu
| IMX ipu Video Converter
|-
|-
| ipucsc.imx
|-
| mfw_ipucsc
| imxvideoconvert.imx
| IPU-based video converter
| imxvideoconvert_g2d
| IMX g2d Video Converter
|-
|-
| imxcompositor.imx
| imxcompositor_ipu
| IMX ipu Video Compositor
|-
|-
| imxcompositor.imx
| imxcompositor_g2d
| IMX g2d Video Compositor
|-
|-
|}
|}
Line 250: Line 280:
[[File:gst4.jpg]]
[[File:gst4.jpg]]
<br/><br/>Test pattern with specific paramters on the input and output elements
<br/><br/>Test pattern with specific paramters on the input and output elements
<pre>$ gst-launch videotestsrc pattern=circular ! imxv4l2sink overlay-width=1280 overlay-height=720</pre>
<pre>$ gst-launch-1.0 videotestsrc pattern=circular ! imxv4l2sink overlay-width=1280 overlay-height=720</pre>
[[File:gst3.jpg]]
[[File:gst3.jpg]]


= Camera Loopback  =
= Camera Loopback  =
Stream video from a camera to a display
Stream video from a camera to a display</br>
<pre>$ gst-launch imxv4l2src device=/dev/video0 capture-mode=4 fps-n=30 queue-size=7 ! queue max-size-buffers=3 ! imxv4l2sink disp-width=1280 disp-height=720</pre>
Command:</br>
:capture-mode ov5460 options:
gst-launch-1.0 imxv4l2src ! 'video/x-raw, format=(string)$FORMAT, width=$WIDTH, height=$HEIGHT, framerate=(fraction)30/1' ! imxv4l2sink</br>
:::ov5640_mode_VGA_640_480 = 0,
For example:
:::ov5640_mode_QVGA_320_240 = 1,
<pre>$ gst-launch-1.0 imxv4l2src device=/dev/video0 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1'! imxv4l2sink overlay-width=1920 overlay-height=1080</pre>
:::ov5640_mode_NTSC_720_480 = 2,
 
:::ov5640_mode_PAL_720_576 = 3,
This is a 3 elements pipe,the element in the middle is a GStreamer capsfilter element.</br>
:::ov5640_mode_720P_1280_720 = 4,
Parameter comments:</br>
:::ov5640_mode_1080P_1920_1080 = 5 </br>
:• Get the camera support format and resolution using gst-inspect-1.0 imxv4l2src.</br>
This is a 3 elements pipe,the queue in the middle is a GStreamer element.
:• Set capsfilter according to the camera's supported capabilities if the user needs other format or resolution.</br>
:• Ensure that the right caps filter has been set, which also needs to be supported by imxv4l2sink.</br>
 
ov5640 example resolutions:
:width=320,height=240
:width=640,height=480
:width=720,height=480
:width=720,height=576
:width=1280,height=720
:width=1920,height=1080 </br>
 
= Video Record =
command:</br>
gst-launch-1.0 imxv4l2src device=$DEVICE num-buffers=300 ! $INPUT_CAPS ! queue ! vpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION </br>


= Video Encode =
Parameter comments:
Encode video from a camera into a file.</br>
:• $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.
:• num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)
:• INPUT_CAPS should be set according to camera format and resolution
:• vpuenc_XXX can be vpuenc_mpeg4,vpuenc_h263, vpuenc_h264, and vpuenc_jpeg
:• MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux
:• EXTENSION is filename extension according to the muxer type.
</br>
Record video from a camera into a file.</br>
720p:
720p:
<pre>$ gst-launch imxv4l2src capture-mode=4 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi</pre>
<pre>$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1280,height=720,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi
</pre>
1080p:
1080p:
<pre>$ gst-launch imxv4l2src capture-mode=5 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi</pre>
<pre>$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi</pre>
:The codec type for encoding
:::flags: readable, writable
:::Enum "vpuenc_codec" Default: 0, "mpeg4"
::::(0): mpeg4            - video/mpeg,
::::(5): h263            - video/x-h263
::::(6): avc              - video/x-h264
::::(12): mjpg            - image/jpeg


= Simple Movie Play =  
= Simple Movie Play =  
<pre>$ gst-launch playbin2 uri=file:/run/media/sda1/big_buck_bunny.mp4  
<pre>$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4  
</pre>
</pre>

Latest revision as of 17:41, 9 August 2016

VAR-SOM-MX6 - Streaming Video

Under construction!!!

Stream Camera with imxcamera application

On your host machine install Video Lan VLC VideoLan

  • Find out your host IP address (Windows:ipconfig, Linux:ifconfig)
  • Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and press "Play".
  • On target, start the imxcamera application.
    • Press the Settings button. Select "Enable RTP" box and set the IP to you host machine. Press OK.
    • Press the Camera button to get a Video camera Image.
    • Press the blue button and the camera will be streamed to you host VLC.

Introduction to GStreamer

GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements: 

$ gst-inspect-1.0

Lists all available i.MX6 specific elements: 

$ gst-inspect-1.0 | grep imx

For example: 

root@var-som-mx6:~# gst-inspect-1.0 | grep imx
aiur.imx: webm: [Invalid UTF-8] 8\x91\xddq
aiur.imx:  aiurdemux: Aiur universal demuxer
vpu.imx:  vpuenc_h264: VPU-based AVC/H264 video encoder
vpu.imx:  vpuenc_mpeg4: VPU-based MPEG4 video encoder
vpu.imx:  vpuenc_h263: VPU-based H263 video encoder
vpu.imx:  vpuenc_jpeg: VPU-based JPEG video encoder
vpu.imx:  vpudec: VPU-based video decoder
overlaysink.imx:  overlaysink: IMX Video (video compositor) Sink
imxvideoconvert.imx:  imxvideoconvert_ipu: IMX ipu Video Converter
imxvideoconvert.imx:  imxvideoconvert_g2d: IMX g2d Video Converter
imxv4l2.imx:  imxv4l2sink: IMX Video (video4linux2) Sink
imxv4l2.imx:  imxv4l2src: IMX Video (video4linux2) Source
beep.imx: ac3: [Invalid UTF-8] 4J\xa0t
beep.imx: 3ca: [Invalid UTF-8] 4J\xa0t
beep.imx:  beepdec: Beep universal decoder
imxcompositor.imx:  imxcompositor_ipu: IMX ipu Video Compositor
imxcompositor.imx:  imxcompositor_g2d: IMX g2d Video Compositor
imxmp3enc.imx:  imxmp3enc: imx mp3 audio encoder

Detailed help information on a specific element: 

$ gst-inspect-1.0 imxv4l2sink

For example: 

root@var-som-mx6:~# gst-inspect-1.0 imxv4l2sink
====== IMXV4L2SINK: 4.0.9 build on Aug  8 2016 12:47:08. ======
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                IMX Video (video4linux2) Sink
  Klass                    Sink/Video
  Description              Displays frames on IMX SoC video4linux2 device
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     imxv4l2.imx
  Description              IMX SoC v4l2-based video source/sink
  Filename                 /usr/lib/gstreamer-1.0/libgstimxv4l2plugins.so
  Version                  4.0.9
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstImxV4l2Sink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
                 format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: 0x76a0b808

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "imxv4l2sink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  device              : Device location
                        flags: readable, writable
                        String. Default: "/dev/video17"
  overlay-top         : The topmost (y) coordinate of the video overlay; top left corner of screen is 0,0
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  overlay-left        : The leftmost (x) coordinate of the video overlay; top left corner of screen is 0,0
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  overlay-width       : The width of the video overlay; default is equal to screen width
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  overlay-height      : The height of the video overlay; default is equal to screen height
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  crop-top            : The topmost (y) coordinate of the video crop; top left corner of image is 0,0
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-left           : The leftmost (x) coordinate of the video crop; top left corner of image is 0,0
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-width          : The width of the video crop; default is equal to negotiated image width
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-height         : The height of the video crop; default is equal to negotiated image height
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  rotate              : The orientation degree of the video; default is 0 degree
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  force-aspect-ratio  : When enabled, scaling will respect original aspect ratio
                        flags: readable, writable
                        Boolean. Default: false
  deinterlace         : set deinterlace enabled; can't be configed on fly
                        flags: readable, writable
                        Boolean. Default: true
  motion              : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion.
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2 Default: 2
  reconfig            : Change V4L2 configuration while running; overlay position/size/rotation changed.
                        flags: readable, writable
                        Boolean. Default: false
  composition-meta-enable: Enable overlay composition meta processing
                        flags: readable, writable
                        Boolean. Default: false

gst-launch

For Example launch the camera and display on the monitor: 

$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix
 Element Name
 description
imxmp3enc.imx imxmp3enc imx mp3 audio encoder
imxv4l2.imx imxv4l2sink IMX Video (video4linux2) Sink
overlaysink.imx overlaysink IMX Video (video compositor) Sink
imxv4l2.imx imxv4l2src IMX Video (video4linux2) Source
vpu.imx vpudec VPU-based video decoder
vpu.imx vpuenc_h264 VPU-based AVC/H264 video encoder
vpu.imx vpuenc_h263 VPU-based H263 video encoder
vpu.imx vpuenc_mpeg4 VPU-based MPEG4 video encoder
vpu.imx vpuenc_jpeg VPU-based JPEG video encoder
imxvideoconvert.imx imxvideoconvert_ipu IMX ipu Video Converter
imxvideoconvert.imx imxvideoconvert_g2d IMX g2d Video Converter
imxcompositor.imx imxcompositor_ipu IMX ipu Video Compositor
imxcompositor.imx imxcompositor_g2d IMX g2d Video Compositor

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern: 

$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink

Gst1.jpg

Test pattern with specific paramters on the input element 

$ gst-launch-1.0 videotestsrc pattern=circular ! imxv4l2sink

Gst2.jpg

Test pattern with specific paramters on the output element 

$ gst-launch-1.0 videotestsrc ! imxv4l2sink overlay-width=1280 overlay-height=720

Gst4.jpg

Test pattern with specific paramters on the input and output elements 

$ gst-launch-1.0 videotestsrc pattern=circular ! imxv4l2sink overlay-width=1280 overlay-height=720

Gst3.jpg

Camera Loopback

Stream video from a camera to a display
Command:
gst-launch-1.0 imxv4l2src ! 'video/x-raw, format=(string)$FORMAT, width=$WIDTH, height=$HEIGHT, framerate=(fraction)30/1' ! imxv4l2sink
For example: 

$ gst-launch-1.0 imxv4l2src device=/dev/video0 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1'! imxv4l2sink overlay-width=1920 overlay-height=1080

This is a 3 elements pipe,the element in the middle is a GStreamer capsfilter element.
Parameter comments:

• Get the camera support format and resolution using gst-inspect-1.0 imxv4l2src.
• Set capsfilter according to the camera's supported capabilities if the user needs other format or resolution.
• Ensure that the right caps filter has been set, which also needs to be supported by imxv4l2sink.

ov5640 example resolutions:

width=320,height=240
width=640,height=480
width=720,height=480
width=720,height=576
width=1280,height=720
width=1920,height=1080

Video Record

command:
gst-launch-1.0 imxv4l2src device=$DEVICE num-buffers=300 ! $INPUT_CAPS ! queue ! vpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION

Parameter comments:

• $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.
• num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)
• INPUT_CAPS should be set according to camera format and resolution
• vpuenc_XXX can be vpuenc_mpeg4,vpuenc_h263, vpuenc_h264, and vpuenc_jpeg
• MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux
• EXTENSION is filename extension according to the muxer type.


Record video from a camera into a file.
720p: 

$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1280,height=720,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi

1080p: 

$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi

Simple Movie Play

$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4
Retrieved from "https://variwiki.com/index.php?title=VAR-SOM-MX6_GSTREAM_JETHRO&oldid=6304"
Latest Modules
Support
Variscite Support Center
Contact Support
Product Longevity
Product Compliance Policy
Blog
Variscite Training
Variscite Website
This page was last edited on 9 August 2016, at 17:41.

© 2024 Variscite. All rights reserved