Revision as of 12:15, 9 August 2016

VAR-SOM-MX6 - Streaming Video

Under construction!!!

Stream Camera with imxcamera application

On your host machine install Video Lan VLC VideoLan

Find out your host IP address (Windows:ipconfig, Linux:ifconfig)
Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and press "Play".
On target, start the imxcamera application.
- Press the Settings button. Select "Enable RTP" box and set the IP to you host machine. Press OK.
- Press the Camera button to get a Video camera Image.
- Press the blue button and the camera will be streamed to you host VLC.

Introduction to GStreamer

GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect-1.0

Lists all available i.MX6 specific elements:

$ gst-inspect-1.0 | grep imx

For example:

root@var-som-mx6:~# gst-inspect-1.0 | grep imx
aiur.imx: webm: [Invalid UTF-8] 8\x91\xddq
aiur.imx:  aiurdemux: Aiur universal demuxer
vpu.imx:  vpuenc_h264: VPU-based AVC/H264 video encoder
vpu.imx:  vpuenc_mpeg4: VPU-based MPEG4 video encoder
vpu.imx:  vpuenc_h263: VPU-based H263 video encoder
vpu.imx:  vpuenc_jpeg: VPU-based JPEG video encoder
vpu.imx:  vpudec: VPU-based video decoder
overlaysink.imx:  overlaysink: IMX Video (video compositor) Sink
imxvideoconvert.imx:  imxvideoconvert_ipu: IMX ipu Video Converter
imxvideoconvert.imx:  imxvideoconvert_g2d: IMX g2d Video Converter
imxv4l2.imx:  imxv4l2sink: IMX Video (video4linux2) Sink
imxv4l2.imx:  imxv4l2src: IMX Video (video4linux2) Source
beep.imx: ac3: [Invalid UTF-8] 4J\xa0t
beep.imx: 3ca: [Invalid UTF-8] 4J\xa0t
beep.imx:  beepdec: Beep universal decoder
imxcompositor.imx:  imxcompositor_ipu: IMX ipu Video Compositor
imxcompositor.imx:  imxcompositor_g2d: IMX g2d Video Compositor
imxmp3enc.imx:  imxmp3enc: imx mp3 audio encoder

Detailed help information on a specific element:

$ gst-inspect-1.0 imxv4l2sink

For example:

root@var-som-mx6:~# gst-inspect-1.0 imxv4l2sink
====== IMXV4L2SINK: 4.0.9 build on Aug  8 2016 12:47:08. ======
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                IMX Video (video4linux2) Sink
  Klass                    Sink/Video
  Description              Displays frames on IMX SoC video4linux2 device
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     imxv4l2.imx
  Description              IMX SoC v4l2-based video source/sink
  Filename                 /usr/lib/gstreamer-1.0/libgstimxv4l2plugins.so
  Version                  4.0.9
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstImxV4l2Sink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
                 format: { RGB16, BGR, RGB, RGBx, BGRx, NV12, UYVY, YUY2, Y42B, I420, YV12, TNVP, TNVF, Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: 0x76a0b808

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "imxv4l2sink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  device              : Device location
                        flags: readable, writable
                        String. Default: "/dev/video17"
  overlay-top         : The topmost (y) coordinate of the video overlay; top left corner of screen is 0,0
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  overlay-left        : The leftmost (x) coordinate of the video overlay; top left corner of screen is 0,0
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  overlay-width       : The width of the video overlay; default is equal to screen width
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  overlay-height      : The height of the video overlay; default is equal to screen height
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  crop-top            : The topmost (y) coordinate of the video crop; top left corner of image is 0,0
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-left           : The leftmost (x) coordinate of the video crop; top left corner of image is 0,0
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-width          : The width of the video crop; default is equal to negotiated image width
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  crop-height         : The height of the video crop; default is equal to negotiated image height
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  rotate              : The orientation degree of the video; default is 0 degree
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  force-aspect-ratio  : When enabled, scaling will respect original aspect ratio
                        flags: readable, writable
                        Boolean. Default: false
  deinterlace         : set deinterlace enabled; can't be configed on fly
                        flags: readable, writable
                        Boolean. Default: true
  motion              : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion.
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2 Default: 2
  reconfig            : Change V4L2 configuration while running; overlay position/size/rotation changed.
                        flags: readable, writable
                        Boolean. Default: false
  composition-meta-enable: Enable overlay composition meta processing
                        flags: readable, writable
                        Boolean. Default: false

gst-launch

For Example launch the camera and display on the monitor:

gst-launch -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix	Element Name	description
mp3enc.imx	mfw_mp3encoder	mp3 audio encoder
v4lsink.imx	imxv4l2sink	v4l2 video sink
isink.imx	mfw_isink	IPU-based video sink
v4lsrc.imx	imxv4l2src	v4l2 based camera src
vpu.imx	vpudec	VPU-based video decoder
vpu.imx	vpuenc	VPU-based video encoder
ipucsc.imx	mfw_ipucsc	IPU-based video converter

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ gst-launch -v videotestsrc ! imxv4l2sink

Test pattern with specific paramters on the input element

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink

Test pattern with specific paramters on the output element

$ gst-launch videotestsrc ! imxv4l2sink disp-width=1280 disp-height=720

Test pattern with specific paramters on the input and output elements

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink  disp-width=1280 disp-height=720

Camera Loopback

Stream video from a camera to a display

$ gst-launch imxv4l2src device=/dev/video0 capture-mode=4 fps-n=30 queue-size=7 ! queue max-size-buffers=3 ! imxv4l2sink  disp-width=1280 disp-height=720

capture-mode ov5460 options:

ov5640_mode_VGA_640_480 = 0,

ov5640_mode_QVGA_320_240 = 1,

ov5640_mode_NTSC_720_480 = 2,

ov5640_mode_PAL_720_576 = 3,

ov5640_mode_720P_1280_720 = 4,

ov5640_mode_1080P_1920_1080 = 5

This is a 3 elements pipe,the queue in the middle is a GStreamer element.

Video Encode

Encode video from a camera into a file.
720p:

$ gst-launch imxv4l2src capture-mode=4 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi

1080p:

$ gst-launch imxv4l2src capture-mode=5 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi

The codec type for encoding

flags: readable, writable

Enum "vpuenc_codec" Default: 0, "mpeg4"

(0): mpeg4 - video/mpeg,

(5): h263 - video/x-h263

(6): avc - video/x-h264

(12): mjpg - image/jpeg

Simple Movie Play

$ gst-launch playbin2 uri=file:/run/media/sda1/big_buck_bunny.mp4

VAR-SOM-MX6 GSTREAM JETHRO: Difference between revisions