72 Commits
Author SHA1 Message Date
Wan-Teh Chang 428f3104fa Include "gtest/gtest.h" using the shorter path
Rely on the -I or -system compiler option to find "gtest/gtest.h". This
makes it easier to build our tests against a copy of gtest outside the
libvpx source tree.

Bug: webm:42330726
Change-Id: I3b189c6345e13b36b236d1eedc6ee091bfa71f48
2024-08-02 22:42:20 +00:00
James Zern cdf8da4c03 vp8: fix OOB access in x->MVcount
Motion vectors are now clamped in
vp8_find_best_sub_pixel_step_iteratively, vp8_find_best_sub_pixel_step,
vp8_find_best_half_pixel_step, vp8_full_search_sad,
vp8_refining_search_sadx4 and vp8_refining_search_sad_c (the rtcd for
other optimizations are redirects to vp8_refining_search_sadx4).

The difference of valid motion vectors may still go beyond the range of
the MVcount array, however, so additional checks are added to
rd_update_mvcount() and update_mvcount().

Note the test source and settings (speed 1 and GOOD quality mode) come
from the issue report; additional coverage is added for realtime. The
realtime path does not trigger the error without the fix, but as it's
similar to the rd path, the same clamp is done to be safe.

Fixes:
vp8/encoder/rdopt.c:1579:5: runtime error: index 17467 out of bounds for
  type 'unsigned int[2047]'

Bug: oss-fuzz:69906
Change-Id: Ia8bd087cfe4475ab09ba711ed806fbcbaa72e552
2024-07-25 15:08:02 -07:00
James Zern f9120b789d vp8,calc_iframe_target_size: clamp kf_boost
cpi->output_framerate may be as large as 10M. Previously this would
cause kf_boost to be ~20M which would overflow an int when multiplied by
values in kf_boost_qadjustment[].

Fixes:
vp8/encoder/ratectrl.c:340:25: runtime error: signed integer overflow:
  19999984 * 220 cannot be represented in type 'int'

Bug: oss-fuzz:69100
Change-Id: I2d77c9d2912412f6265f6a8dc0e6b361b63b8242
2024-07-25 19:43:53 +00:00
James Zern efe615f804 add repro for crbug.com/352414650
+ fix an additional double -> int overflow warning (chrome's fuzzers do
  not have the float-cast-overflow sanitizer enabled)

Bug: chromium:352414650
Change-Id: I634bb421a74236eac434df138ed71dadf197596a
2024-07-18 13:36:10 -07:00
Wan-Teh Chang 25540b3c12 Fix some UBSan errors in vp8_new_framerate()
Fix some UBSan errors in the calculations of cpi->av_per_frame_bandwidth
and cpi->min_frame_bandwidth in vp8_new_framerate() and in the
calculation of cpi->per_frame_bandwidth in encode_frame_to_data_rate().

A port of the VP9 changes in
https://chromium-review.googlesource.com/c/webm/libvpx/+/4944271 and
https://chromium-review.googlesource.com/c/webm/libvpx/+/5565157 to VP8.
Similar to the libaom CL
https://aomedia-review.googlesource.com/c/aom/+/190462.

Bug: aomedia:3509
Change-Id: I77b0e0b2f9fe667428daa9c4ceec0a35aafbfa81
2024-05-23 18:05:54 -07:00
Wan-Teh Chang 495c4b596c Add a #endif comment for CONFIG_VP9_HIGHBITDEPTH
Change-Id: Idc388e722e2579ce6935b52b1786038bdf2d5d47
2024-05-23 23:50:28 +00:00
Wan-Teh Chang 1f65facb63 Fix a UBSan error in vp9_rc_update_framerate()
Fix a UBSan error in the calculation of rc->min_frame_bandwidth in
vp9_rc_update_framerate().

A follow-up to
https://chromium-review.googlesource.com/c/webm/libvpx/+/4944271.
Similar to the libaom CL
https://aomedia-review.googlesource.com/c/aom/+/190462.

Bug: aomedia:3509
Change-Id: I36168a6d00cd81e60ae19a7d74c21f2e6c2f0caf
2024-05-23 14:38:50 -07:00
James Zern 108f5128e2 encode_api_test.cc: assert encoder is initialized
Before proceeding with Encode(). This avoids some static analysis
warnings about uninitialized `cfg_` members.

Change-Id: Ib67b278d6706ab1034219e8c1ad9ba0c5b574ba8
2024-05-03 22:00:56 +00:00
Wan-Teh Chang f65aff7b99 Remove vpx_ports/msvc.h
The vpx_ports/msvc.h header provides snprintf() and round() for MSVC
older than Visual Studio 2015 and Visual Studio 2013, respectively.

Since configure now requires vs14 (Visual Studio 2015) or later, it is
safe to remove vpx_ports/msvc.h.

Change-Id: I2fe4c41eaa126f4cf17639c11895f1e464294c76
2024-05-02 20:01:11 +00:00
James Zern e9be4f607b encode_api_test.cc: apply iwyu
add missing <cstdio> and <cstdlib> and delete some unused headers.

Change-Id: I6c66368f557e6df896bffb2aa90228811f14f027
2024-05-02 02:25:43 +00:00
James Zern 63b9c2c0e2 VP9: add vpx_codec_get_global_headers() support
This returns the contents of CodecPrivate described in:
https://www.webmproject.org/docs/container/#vp9-codec-feature-metadata-codecprivate

The value for 4:2:0 is 1 (colocated) to match the default given for the
codec parameter string:
https://www.webmproject.org/vp9/mp4/#codecs-parameter-string

Bug: b:332052663
Change-Id: Ie50dd8d76e2d7389ac01bf4dbec801f9c8ea0e21
2024-04-25 15:20:18 -07:00
Casey Smalley 588beb020b Unit test config changes for Chromium
This CL will help run libvpx tests under Chromium against its partition
allocator. The allocator does not support single allocations above
3.998GiB. Because of this tests related to large video sizes that
Chromium is configured for are expected to fail.

Chromium also only supports the CONFIG_REALTIME_ONLY option,
some changes are scoped behind this flag.

Change-Id: I80e8743c0619ce502688109ce0be01cb252d5f92
2024-04-03 20:20:44 +00:00
Marco Paniconi 3f8f19372b vp9: Fix to alloc for row_base_thresh_freq_fac
Issue happens for real-time nonrd pickmode.
Due to speed feature: sf->adaptive_rd_thresh_row_mt,
enabled for speed >= 8, and for speed >= 7 svc only.

Issue occurs where resolution (sb_rows) changes and
row_base_thresh_freq_fact needs to be re-allocated.

Fix is to add sb_rows to TileDataEnc and check for
re-alloc of row_base_thresh_freq_fac.

Bug: b:331108922
Change-Id: I1a1ca94c14f343200c180725e4cb8d91d3c55b83
2024-03-28 16:47:45 +00:00
Wan-Teh Chang 34277e53ad Free row mt memory before freeing cpi->tile_data
In vp9_init_tile_data(), call vp9_row_mt_mem_dealloc(cpi) to free the
row mt memory in cpi->tile_data before freeing cpi->tile_data.

Bug: b:331086799, b:331108729
Change-Id: Idc79984ce7e0110e6858139b2ed286492a2e8622
2024-03-26 23:38:09 +00:00
Marco Paniconi c29e637283 Fix to buffer alloc for vp9_bitstream_worker_data
The code was using the bitstream_worker_data when it
wasn't allocated for big enough size. This is because
the existing condition was to only re-alloc the
bitstream_worker_data when current dest_size was larger
than the current frame_size. But under resolution change
where frame_size is increased, beyond the current dest_size,
we need to allow re-alloc to the new size.

The existing condition to re-alloc when dest_size is
larger than frame_size (which is not required) is kept
for now.

Also increase the dest_size to account for image format.

Added tests, for both ROW_MT=0 and 1, that reproduce
the failures in the bugs below.

Note: this issue only affects the REALTIME encoding path.

Bug: b/329088759, b/329674887, b/329179808

Change-Id: Icd65dbc5317120304d803f648d4bd9405710db6f
2024-03-15 21:43:28 +00:00
Wan-Teh Chang 6c0bf97a98 Detect integer overflows related to pts & duration
A port of the following two libaom CLs:
https://aomedia-review.googlesource.com/c/aom/+/187902
https://aomedia-review.googlesource.com/c/aom/+/188161

Bug: webm:1828
Change-Id: Id25039b000c3d04e7a4c8d71579a6932e9fd65ef
2024-03-14 11:14:25 -07:00
Wan-Teh Chang 8c36d36bcc Add high bit depths, 4:2:2, 4:4:4 to VP9Encoder
A port of the changes to vp9_encoder_fuzz_test.cc in
https://chromium-review.googlesource.com/c/chromium/src/+/5292940.

Change-Id: Ie143ffd9cffbd6a8639812c72e85c9a017aa554e
2024-03-13 13:45:41 -07:00
Wan-Teh Chang d63efe0679 Handle EINTR from sem_wait(&cpi->h_event_end_lpf)
sem_wait() may be interrupted by a signal and fail with EINTR:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_wait.html

Retry the sem_wait(&cpi->h_event_end_lpf) call if it fails with EINTR.

Bug: chromium:324459561
Change-Id: Icc957e8b9f21f25ec3c95e22cab502af417443f2
2024-02-16 00:14:40 +00:00
James Zern 7ad5f4f695 vp9_scale_and_extend_frame_ssse3: fix uv width/height
Use uv_crop_(width|height). This fixes an issue with 1 to 2 scaling from
1x1 where the unrounded value would go to zero, resulting in a heap
overflow. This path is only executed when the library is built without
--enable-vp9-highbitdepth.

Bug: b:319964497
Change-Id: I9cb6632f864ec54c045608af86aede20657d6253
2024-01-30 20:50:56 +00:00
James Zern 43e1c8bf10 encode_api_test,RandomPixelsVp8: fix stack overflow
Observed when built using Visual Studio 2019.

Move 720P image allocation to the heap.

Bug: webm:1831
Change-Id: I4e343af08d2f282618ad1b328a39d7dba5e79654
2024-01-22 18:04:03 +00:00
Marco Paniconi 43bd567950 vp8: Fix to integer division by zero and overflow
This can happen in the setting of the frame
target size for delta frames, for non-CBR mode
(end_usage != USAGE_STREAM_FROM_SERVER) and with
temporal layers.

In calc_pframe_target_size(): the percent_high
(factor to adjust the target_size) may end up dividing
bits_off_target by total_byte_count. The total_byte_count
is define per layer for temporal layers, so it will be zero
for delta frames if the enhancement layer has never been
encoded before.

Since percent_high is capped to over_shoot_pct, the proposed
fix is to apply this cap if total_byte_count is zero.
Also this CL fixes a few integer overflow issues in setting
the layer target_bandwidth, the recale function, and in
setting target_bits_per_mb.

Unittest is added by Wan-Teh which triggers this issue.

Bug: chromium:1514684

Change-Id: I091158e720ece75d7ab9b7c4d18d30a5783102ab
2024-01-11 23:05:47 +00:00
Wan-Teh Chang aef73b22cb Make encoder know frame size increase from config
Equivalent to the change to av1_change_config() in the libaom CL
https://aomedia-review.googlesource.com/c/aom/+/182413.

Because we call alloc_compressor_data() only if
cm->mi_alloc_size < new_mi_size, this change won't cause
alloc_compressor_data() to be called unnecessarily, unlike the libaom
bug https://crbug.com/aomedia/3526.

Bug: b:317105128
Change-Id: I8a772a1d5c4766846641a6d541a6d861bf76c60f
2024-01-10 21:42:45 +00:00
Jerome Jiang 41ced868a6 Remove VP9E_GET_TPL_STATS
This is never used.
A callback in external rc func was added and used instead.

Change-Id: Iade6f361072f0c28af98904baf457d2f0e9ca904
2023-12-16 20:12:34 +00:00
Marco Paniconi 193b151195 Fix to integer overflow in vp8 encodeframe.c
Unit test added.

Bug:webm:1831

Change-Id: Ib85f4f0fbdbebc0b49555f206a36376cea687df6
2023-12-12 22:49:46 +00:00
Cheng Chen 1ed56a46b3 Update frame size in actual encoding
Issue explanation:
The unit test calls set_config function twice after encoding the
first frame.
The first call of set_config reduces frame width, but is still within
half of the first frame.
The second call reduces frame width even more, making is less than
half of the first frame, which according to the encoder logic,
there is no valid ref frames, and this frame should be set as a
forced keyframe. This leads to null pointer access in scale_factors
later.

Solution:
To make sure the correct detection of a forced key frame,
we need to update the frame width and height only when the actual
encoding is performed.

Bug: b/311985118

Change-Id: Ie2cd3b760d4a4b399845693d7421c4eb11a12775
2023-12-06 21:46:54 -08:00
Yunqing Wang 50ed636e49 Fix a bug in simple motion search
This change fixed a bug revealed by b/311294795.
In simple motion search, the reference buffer pointer needs to be
restored after the search. Otherwise, it causes problems while the
reference frame scaling happens. This CL fixes the bug.

Bug: b/311294795
Change-Id: I093722d5888de3cc6a6542de82a6ec9d601f897d
2023-12-06 16:50:33 -08:00
Jerome Jiang 585798f756 Set pred buffer stride correctly
Bug: b/312875957
Change-Id: I2eb5ab86d5fe30079b3ed1cbdb8b45bb2dc72a1d
2023-12-06 23:54:31 +00:00
Jerome Jiang 4c2435c33e Fix several clang-tidy complaints
Change-Id: I78721d6b7ed692ad9363b5cac4e3324a3136d5b6
2023-12-05 22:08:56 +00:00
Marco Paniconi 12e928cb34 Add unittest for issue b/314857577
Bug: b/314857577

Change-Id: I591036c1ad3362023686d395adb4783c51baa62d
2023-12-05 22:02:45 +00:00
Yunqing Wang 8bf3649d41 Fix a bug in frame scaling
This change fixed a corner case bug reealed by b/311394513.
During the frame scaling, vpx_highbd_convolve8() and vpx_scaled_2d()
requires both x_step_q4 and y_step_q4 are less than or equal to a
defined value. Otherwise, it needs to call vp9_scale_and_extend_
frame_nonnormative() that supports arbitrary scaling.

The fix was done in LBD and HBD funnctions.

Bug: b/311394513
Change-Id: Id0d34e7910ec98859030ef968ac19331488046d4
2023-12-04 19:39:08 -08:00
Bohan Li 9ad598f249 Improve test comments.
Change-Id: I42dddb946193e30cf07e39b43eaad051c5da479a
2023-12-04 23:32:08 +00:00
Angie Chiang f10481dc0a Set skip_recode=0 in nonrd_pick_sb_modes
Need to set skip_recode properly so that
vp9_encode_block_intra() can work properly when it is
called by block_rd_txfm(). We can not skip "recode" because
it is still at the rd search stage.

Bug: b/310340241
Change-Id: I7d7600ef72addd341636549c2dad1868ad90e1cb
2023-12-02 07:18:11 +00:00
Wan-Teh Chang 478df94cd2 Merge "Define vpx_enc_deadline_t type for encode deadline" into main 2023-12-01 22:57:58 +00:00
Bohan Li a9f1bfdb8e Fix edge case when downsizing to one.
BUG: b/310329177
Change-Id: I2ebf4165adbc7351d6cc73554827812dedc4d362
2023-12-01 13:44:56 -08:00
Bohan Li 845a817c05 Fix scaled reference offsets.
Since the reference frame is already scaled, do not scale the offsets.

BUG: b/311489136, b/312656387
Change-Id: Ib346242e7ec8c4d3ed26668fa4094271218278ed
2023-12-01 09:52:59 -08:00
Wan-Teh Chang 15c2a9a02f Add a test for b/312517065
Bug: b/312517065
Change-Id: I6b5529a8e034fb0468f110e420fafb4944a19d0f
2023-11-30 16:10:04 -08:00
Wan-Teh Chang b027590c30 Define vpx_enc_deadline_t type for encode deadline
The deadline parameter of vpx_codec_encode() is of the unsigned long
type. The cpplint runtime/int check and the clang-tidy
google-runtime-int warn about the use of the unsigned long type. Adding
a type alias works around this issue.

Note: vpx_codec_decode() also has a deadline parameter, but it is of the
long type. So unfortuntely this type alias cannot be simply named
vpx_codec_deadline_t and the name must suggest it is encoder-specific.

Change-Id: I27b6b25730b620b328422ec3f91e63fdc55b377a
2023-11-29 17:36:24 -08:00
Wan-Teh Chang 57b72fe807 Add VP9Encoder class to simplify fuzz test cases
Bug: b:306422625
Change-Id: I8344cb7fb4e1aee87d46f683746517cdcddf5c5d
2023-11-29 14:22:10 -08:00
Wan-Teh Chang 73e38df5d5 Merge "Adding "const" to vpx_codec_iface_t is redundant" into main 2023-11-29 20:04:24 +00:00
Marco Paniconi adebf364cb rtc: Set nonrd keyframe under dynamic change of deadline
For realtime mode: if the deadline mode (good/best/realtime)
is changed on the fly (via codec_encode() call), force a
key frame and set the speed feature nonrd_keyframe = 1 to
avoid entering the rd pickmode.

nonrd_pickmode=0/off is the only feature in realtime mode that
involves rd pickmode, so by forcing it on/1 we can cleanly
separate nonrd (realtime) from rd (good/best), so we can
avoid possible issues on this dynamic mode switch, such as in
bug listed below.

Dynamic change of deadline, in particular for realtime mode,
involves a lot of coding/speed feature changes, so best to
also force reset with keyframe.

Added unitest that triggers the issue in the bug.
Bug: b/310663186

Change-Id: Idf8fd7c9ee54b301968184be5481ee9faa06468d
2023-11-27 20:08:57 -08:00
Marco Paniconi d7358ed53a Unitest for issue: b/310477034
Fix is made here:
https://chromium-review.googlesource.com/c/webm/libvpx/+/5055827

Bug: b/310477034
Change-Id: Id1cc7a6a95e1ea5d1a022f36d7971915c9918339
2023-11-27 09:35:10 -08:00
Wan-Teh Chang 635eba3319 Adding "const" to vpx_codec_iface_t is redundant
vpx_codec_iface_t is defined as follows:

  typedef const struct vpx_codec_iface vpx_codec_iface_t;

Since vpx_codec_iface_t is already a const struct, it is redundant to
add "const" to vpx_codec_iface_t.

Note: I think vpx_codec_iface_t should not have been defined as a const
struct, but it is too late to change that now.

Change-Id: Ifbd3f8a63c1d48e9169ff77fa0b505ea1e65519d
2023-11-22 15:38:04 -08:00
Jerome Jiang 741b8f6228 Check null ptr before use
prev_mi is a pointer to pointer

Bug: b/310401647
Bug: b/310590556
Change-Id: Ic3c39a7eec14693357bd2485a5451d4b7f031b5e
2023-11-21 16:33:50 -05:00
Wan-Teh Chang f05122d35c Fix ClangTidy warnings
Most are related to include-what-you-use. One is to avoid using the
unsigned long type explicitly (by passing VPX_DL_REALTIME directly to
vpx_codec_encode).

Change-Id: Ieaf3418382ad8516cb4b172f7678893286fcb8cf
2023-11-09 18:35:51 -08:00
Wan-Teh Chang 7ab673a9f6 Fix float-cast-overflow in vp8_change_config()
Bug: b:309716574
Change-Id: I9c523d5e9211f895c7497a9e3674b55f6be6c742
2023-11-09 12:01:13 -08:00
Jerome Jiang 352f9f64df Reduce memory usage of test with large frame size
- Use smaller frame size that still triggers the overflow
 - Do not run encoder as the encoder init also triggers the overflow

Bug: chromium:1492864
Change-Id: I392549abf69f1cfb3754cc847a214513ec9bedc5
2023-10-20 15:40:43 -04:00
Jerome Jiang 424723dc02 Run bitrate overflow test only on 64bit systems
Frame size caps the target bitrate internally, so the frame size needs
to be large enough to reproduce the target bitrate overflow in the
fuzzing test.

However the frame size needed exceeds the max buffer allowed on 32bit
system defined by VPX_MAX_ALLOCABLE_MEMORY

Bug: chromium:1492864

Change-Id: Ia3a9a78cd35516373897039a7769b492e29e8450
2023-10-19 11:36:47 -04:00
Jerome Jiang e4db6c3aac Cap avg_frame_bandwidth at INT_MAX
avg_frame_bandwidth = target_bandwidth / framerate

If target_bandwidth is too big and/or framerate is too small (< 1),
avg_frame_bandwidth could be overflow

Bug: chromium:1492864
Change-Id: I32314da1414b472ae4bf2acdcd81b8a948286146
2023-10-17 17:06:28 -04:00
Wan-Teh Chang b729684b05 Use big cfg.g_w in ConfigResizeChangeThreadCount
vp8cx_create_encoder_threads() caps the thread count at
(cm->mb_cols / cpi->mt_sync_range) - 1. If cfg.g_w is 16, cm->mb_cols is
only 1 (see vp8_alloc_frame_buffers: mb_cols = width >> 4), so we won't
be using multiple threads. To reproduce bug chromium:1486441, the test
just needs to increase cfg.g_h sufficiently.

Bug: chromium:1486441
Change-Id: Ie6b2da2e31cfa1717a481f55eebc8c875db94d87
2023-10-02 13:55:16 -07:00
James Zern 3fbd1dca6a VP8: disallow thread count changes
Currently allocations are done at encoder creation time. Going from
threaded to non-threaded would cause a crash.

Bug: chromium:1486441
Change-Id: Ie301c2a70847dff2f0daae408fbef1e4d42e73d4
2023-09-25 19:33:07 -07:00