vp9: recon: Use emulated edge to prevent buffer overflows

The arm/aarch64 horizontal filter reads one additional pixel beyond what
the filter uses. This can become an issue if the application does not
allocate larger buffers than what's required for the pixel data. If the
motion vector points to the bottom right edge of the picture this
becomes a read buffer overflow. This triggers segfaults in Firefox for
video resolutions which result in a page aligned picture size like
1280x640.
Prevent this by using emulated edge in this case.

Fixes: https://bugzilla.mozilla.org/show_bug.cgi?id=1881185
Signed-off-by: Janne Grunau <janne-ffmpeg@jannau.net>
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
This commit is contained in:
Janne Grunau 2025-01-03 01:54:40 +01:00 committed by Ronald S. Bultje
parent f366256215
commit 060464105b

View file

@ -319,7 +319,11 @@ static av_always_inline void mc_luma_unscaled(VP9TileData *td, const vp9_mc_func
// The arm/aarch64 _hv filters read one more row than what actually is // The arm/aarch64 _hv filters read one more row than what actually is
// needed, so switch to emulated edge one pixel sooner vertically // needed, so switch to emulated edge one pixel sooner vertically
// (!!my * 5) than horizontally (!!mx * 4). // (!!my * 5) than horizontally (!!mx * 4).
// The arm/aarch64 _h filters read one more pixel than what actually is
// needed, so switch to emulated edge if that would read beyond the bottom
// right block.
if (x < !!mx * 3 || y < !!my * 3 || if (x < !!mx * 3 || y < !!my * 3 ||
((ARCH_AARCH64 || ARCH_ARM) && (x + !!mx * 5 > w - bw) && (y + !!my * 5 + 1 > h - bh)) ||
x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) { x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {
s->vdsp.emulated_edge_mc(td->edge_emu_buffer, s->vdsp.emulated_edge_mc(td->edge_emu_buffer,
ref - !!my * 3 * ref_stride - !!mx * 3 * bytesperpixel, ref - !!my * 3 * ref_stride - !!mx * 3 * bytesperpixel,
@ -358,7 +362,11 @@ static av_always_inline void mc_chroma_unscaled(VP9TileData *td, const vp9_mc_fu
// The arm/aarch64 _hv filters read one more row than what actually is // The arm/aarch64 _hv filters read one more row than what actually is
// needed, so switch to emulated edge one pixel sooner vertically // needed, so switch to emulated edge one pixel sooner vertically
// (!!my * 5) than horizontally (!!mx * 4). // (!!my * 5) than horizontally (!!mx * 4).
// The arm/aarch64 _h filters read one more pixel than what actually is
// needed, so switch to emulated edge if that would read beyond the bottom
// right block.
if (x < !!mx * 3 || y < !!my * 3 || if (x < !!mx * 3 || y < !!my * 3 ||
((ARCH_AARCH64 || ARCH_ARM) && (x + !!mx * 5 > w - bw) && (y + !!my * 5 + 1 > h - bh)) ||
x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) { x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {
s->vdsp.emulated_edge_mc(td->edge_emu_buffer, s->vdsp.emulated_edge_mc(td->edge_emu_buffer,
ref_u - !!my * 3 * src_stride_u - !!mx * 3 * bytesperpixel, ref_u - !!my * 3 * src_stride_u - !!mx * 3 * bytesperpixel,