GstBaseVideoDecoder

GstBaseVideoDecoder — Base class for video decoders

Synopsis

#define             GST_BASE_VIDEO_DECODER_SINK_NAME
#define             GST_BASE_VIDEO_DECODER_SRC_NAME
#define             GST_BASE_VIDEO_DECODER_FLOW_NEED_DATA
#define             GST_BASE_VIDEO_DECODER_FLOW_DROPPED
#define             GST_BASE_AUDIO_DECODER_ERROR        (el,
                                                         w,
                                                         domain,
                                                         code,
                                                         text,
                                                         debug,
                                                         ret)
struct              GstBaseVideoDecoder;
struct              GstBaseVideoDecoderClass;
void                gst_base_video_decoder_class_set_capture_pattern
                                                        (GstBaseVideoDecoderClass *klass,
                                                         guint32 mask,
                                                         guint32 pattern);
GstVideoFrame *     gst_base_video_decoder_get_frame    (GstBaseVideoDecoder *coder,
                                                         int frame_number);
GstVideoFrame *     gst_base_video_decoder_get_oldest_frame
                                                        (GstBaseVideoDecoder *coder);
void                gst_base_video_decoder_add_to_frame (GstBaseVideoDecoder *base_video_decoder,
                                                         int n_bytes);
void                gst_base_video_decoder_lost_sync    (GstBaseVideoDecoder *base_video_decoder);
GstFlowReturn       gst_base_video_decoder_have_frame   (GstBaseVideoDecoder *base_video_decoder);
void                gst_base_video_decoder_set_sync_point
                                                        (GstBaseVideoDecoder *base_video_decoder);
gboolean            gst_base_video_decoder_set_src_caps (GstBaseVideoDecoder *base_video_decoder);
GstBuffer *         gst_base_video_decoder_alloc_src_buffer
                                                        (GstBaseVideoDecoder *base_video_decoder);
GstFlowReturn       gst_base_video_decoder_alloc_src_frame
                                                        (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);
GstVideoState *     gst_base_video_decoder_get_state    (GstBaseVideoDecoder *base_video_decoder);
GstClockTimeDiff    gst_base_video_decoder_get_max_decode_time
                                                        (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);
GstFlowReturn       gst_base_video_decoder_finish_frame (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);

Description

This base class is for video decoders turning encoded data into raw video frames.

GstBaseVideoDecoder and subclass should cooperate as follows.

  1. Configuration

    • Initially, GstBaseVideoDecoder calls start when the decoder element is activated, which allows subclass to perform any global setup.

    • GstBaseVideoDecoder calls set_format to inform subclass of caps describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

    • GstBaseVideoDecoder calls stop at end of all processing.

  2. Data processing

    • Base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.

    • Input frame is provided to subclass' handle_frame.

    • If codec processing results in decoded data, subclass should call gst_base_video_decoder_finish_frame to have decoded data pushed downstream.

  3. Shutdown phase

    • GstBaseVideoDecoder class calls stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' set_format function, though it might be delayed until calling gst_base_video_decoder_finish_frame.

Subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, baseclass provides limited framerate based interpolation.

Similarly, the baseclass provides some limited (legacy) seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Baseclass provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, subclass should then be prepared for the parsing and frame processing stage to occur separately (rather than otherwise the latter immediately following the former), and should ensure the parsing stage properly marks keyframes or rely on upstream to do so properly for incoming data.

Things that subclass need to take care of:

  • Provide pad templates

  • Set source pad caps when appropriate

  • Configure some baseclass behaviour parameters.

  • Optionally parse input data, if it is not considered packetized. Parse sync is obtained either by providing baseclass with a mask and pattern or a custom scan_for_sync. When sync is established, parse_data should invoke gst_base_video_decoder_add_to_frame and gst_base_video_decoder_have_frame as appropriate.

  • Accept data in handle_frame and provide decoded results to gst_base_video_decoder_finish_frame.

Details

GST_BASE_VIDEO_DECODER_SINK_NAME

#define GST_BASE_VIDEO_DECODER_SINK_NAME    "sink"

The name of the templates for the sink pad.


GST_BASE_VIDEO_DECODER_SRC_NAME

#define GST_BASE_VIDEO_DECODER_SRC_NAME     "src"

The name of the templates for the source pad.


GST_BASE_VIDEO_DECODER_FLOW_NEED_DATA

#define GST_BASE_VIDEO_DECODER_FLOW_NEED_DATA GST_FLOW_CUSTOM_SUCCESS

Returned while parsing to indicate more data is needed.


GST_BASE_VIDEO_DECODER_FLOW_DROPPED

#define GST_BASE_VIDEO_DECODER_FLOW_DROPPED GST_FLOW_CUSTOM_SUCCESS_1

Returned when the event/buffer should be dropped.


GST_BASE_AUDIO_DECODER_ERROR()

#define             GST_BASE_AUDIO_DECODER_ERROR(el, w, domain, code, text, debug, ret)

struct GstBaseVideoDecoder

struct GstBaseVideoDecoder {
  GstBaseVideoCodec base_video_codec;
};

The opaque GstBaseVideoDecoder data structure.


struct GstBaseVideoDecoderClass

struct GstBaseVideoDecoderClass {
  GstBaseVideoCodecClass base_video_codec_class;

  gboolean      (*start)          (GstBaseVideoDecoder *coder);

  gboolean      (*stop)           (GstBaseVideoDecoder *coder);

  int           (*scan_for_sync)  (GstBaseVideoDecoder *decoder, gboolean at_eos,
                                   int offset, int n);

  GstFlowReturn (*parse_data)     (GstBaseVideoDecoder *decoder, gboolean at_eos);

  gboolean      (*set_format)     (GstBaseVideoDecoder *coder, GstVideoState * state);

  gboolean      (*reset)          (GstBaseVideoDecoder *coder);

  GstFlowReturn (*finish)         (GstBaseVideoDecoder *coder);

  GstFlowReturn (*handle_frame)   (GstBaseVideoDecoder *coder, GstVideoFrame *frame);
};

Subclasses can override any of the available virtual methods or not, as needed. At minimum handle_frame needs to be overridden, and set_format and likely as well. If non-packetized input is supported or expected, parse needs to be overridden as well.

GstBaseVideoCodecClass base_video_codec_class;

start ()

Optional. Called when the element starts processing. Allows opening external resources.

stop ()

Optional. Called when the element stops processing. Allows closing external resources.

scan_for_sync ()

Optional. Allows subclass to obtain sync for subsequent parsing by custom means (above an beyond scanning for specific marker and mask).

parse_data ()

Required for non-packetized input. Allows chopping incoming data into manageable units (frames) for subsequent decoding.

set_format ()

Notifies subclass of incoming data format (caps).

reset ()

Optional. Allows subclass (codec) to perform post-seek semantics reset.

finish ()

Optional. Called to request subclass to dispatch any pending remaining data (e.g. at EOS).

handle_frame ()

Provides input data frame to subclass.

gst_base_video_decoder_class_set_capture_pattern ()

void                gst_base_video_decoder_class_set_capture_pattern
                                                        (GstBaseVideoDecoderClass *klass,
                                                         guint32 mask,
                                                         guint32 pattern);

gst_base_video_decoder_get_frame ()

GstVideoFrame *     gst_base_video_decoder_get_frame    (GstBaseVideoDecoder *coder,
                                                         int frame_number);

base_video_decoder :

a GstBaseVideoDecoder

frame_number :

system_frame_number of a frame

Returns :

pending unfinished GstVideoFrame identified by frame_number.

gst_base_video_decoder_get_oldest_frame ()

GstVideoFrame *     gst_base_video_decoder_get_oldest_frame
                                                        (GstBaseVideoDecoder *coder);

Sets the mask and pattern that will be scanned for to obtain parse sync. Note that a non-zero mask implies that scan_for_sync will be ignored.

base_video_decoder_class :

a GstBaseVideoDecoderClass

gst_base_video_decoder_add_to_frame ()

void                gst_base_video_decoder_add_to_frame (GstBaseVideoDecoder *base_video_decoder,
                                                         int n_bytes);

gst_base_video_decoder_lost_sync ()

void                gst_base_video_decoder_lost_sync    (GstBaseVideoDecoder *base_video_decoder);

Advances out-of-sync input data by 1 byte and marks it accordingly.

base_video_decoder :

a GstBaseVideoDecoder

gst_base_video_decoder_have_frame ()

GstFlowReturn       gst_base_video_decoder_have_frame   (GstBaseVideoDecoder *base_video_decoder);

Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e. handle_frame.

base_video_decoder :

a GstBaseVideoDecoder

Returns :

a GstFlowReturn

gst_base_video_decoder_set_sync_point ()

void                gst_base_video_decoder_set_sync_point
                                                        (GstBaseVideoDecoder *base_video_decoder);

Marks current frame as a sync point, i.e. keyframe.

base_video_decoder :

a GstBaseVideoDecoder

gst_base_video_decoder_set_src_caps ()

gboolean            gst_base_video_decoder_set_src_caps (GstBaseVideoDecoder *base_video_decoder);

Sets src pad caps according to currently configured GstVideoState.

base_video_decoder :

a GstBaseVideoDecoder

gst_base_video_decoder_alloc_src_buffer ()

GstBuffer *         gst_base_video_decoder_alloc_src_buffer
                                                        (GstBaseVideoDecoder *base_video_decoder);

Helper function that uses gst_pad_alloc_buffer_and_set_caps to allocate a buffer to hold a video frame for base_video_decoder's current GstVideoState.

base_video_decoder :

a GstBaseVideoDecoder

Returns :

allocated buffer

gst_base_video_decoder_alloc_src_frame ()

GstFlowReturn       gst_base_video_decoder_alloc_src_frame
                                                        (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);

Helper function that uses gst_pad_alloc_buffer_and_set_caps to allocate a buffer to hold a video frame for base_video_decoder's current GstVideoState. Subclass should already have configured video state and set src pad caps.

base_video_decoder :

a GstBaseVideoDecoder

frame :

a GstVideoFrame

Returns :

result from pad alloc call

gst_base_video_decoder_get_state ()

GstVideoState *     gst_base_video_decoder_get_state    (GstBaseVideoDecoder *base_video_decoder);

base_video_decoder :

a GstBaseVideoDecoder

Returns :

GstVideoState describing format of video data.

gst_base_video_decoder_get_max_decode_time ()

GstClockTimeDiff    gst_base_video_decoder_get_max_decode_time
                                                        (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);

Determines maximum possible decoding time for frame that will allow it to decode and arrive in time (as determined by QoS events). In particular, a negative result means decoding in time is no longer possible and should therefore occur as soon/skippy as possible.

base_video_decoder :

a GstBaseVideoDecoder

frame :

a GstVideoFrame

Returns :

max decoding time.

gst_base_video_decoder_finish_frame ()

GstFlowReturn       gst_base_video_decoder_finish_frame (GstBaseVideoDecoder *base_video_decoder,
                                                         GstVideoFrame *frame);

Removes next n_bytes of input data and adds it to currently parsed frame.

base_video_decoder :

a GstBaseVideoDecoder

n_bytes :

an encoded GstVideoFrame

See Also

GstBaseTransform