XGboost test 'TestXGBoostLib' contains container-overflow error

[PhoebeHui]

When I run xgboost test with vs2022 msvc, the test 'TestXGBoostLib' failed with 'container-overflow' error, could you please take a look?

Steps to reproduce:

1. Open VS2022 x64 Native Command Prompt tools
2. Clone repo and checkout https://github.com/dmlc/xgboost/commit/3868b5fb1ce14bcdf2c11c5ac6c63de600041777
3. set _CL_=/fsanitize=address /GS- /wd5072 & set _LINK_=/InferASanLibs /incremental:no /debug
4. cd to build folder
5. cmake -G "Visual Studio 17 2022" -A x64 -DCMAKE_SYSTEM_VERSION=10.0.26100.0 -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON 
6. msbuild /m /p:Platform=x64 /p:Configuration=Release xgboost.sln /t:Rebuild
7. ctest  -R TestXGBoostLib  --build-config Release --output-on-failure

Memory safety issue reported by Address Sanitizer:

==14896==ERROR: AddressSanitizer: container-overflow on address 0x01ae056f2820 at pc 0x7ffb4a988457 bp 0x004c552fdf60 sp 0x004c552fd6f0
READ of size 258048 at 0x01ae056f2820 thread T0
#0 0x7ffb4a988456 in __asan_memcpy C:\repos\msvc\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_interceptors_memintrinsics.cpp:66
#1 0x7ff69993e4b5 in xgboost::tree::MultiTargetTreeView::SplitIndex(int) const (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x14074e4b5)
#2 0x7ff699906efd in xgboost::tree::QuantileHist_HistUpdaterPartitionerOverrun_Test::TestBody(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140716efd)
#3 0x7ff69a0cb276 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb276)
#4 0x7ff69a0cadcb in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edadcb)
#5 0x7ff69a1231f3 in testing::TestInfo::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f331f3)
#6 0x7ff69a12361d in testing::TestSuite::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f3361d)
#7 0x7ff69a1243bd in testing::internal::UnitTestImpl::RunAllTests(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f343bd)
#8 0x7ff69a0cb2d6 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb2d6)
#9 0x7ff69a0cb1c9 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb1c9)
#10 0x7ff69a123a0f in testing::UnitTest::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f33a0f)
#11 0x7ff6997cec07 in main (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x1405dec07)
#12 0x7ff69a21bd63 in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
#13 0x7ff69a21bd63 in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#14 0x7ffbf7c5259c (C:\Windows\System32\KERNEL32.DLL+0x18001259c)
#15 0x7ffbf850af77 (C:\Windows\SYSTEM32\ntdll.dll+0x18005af77)

0x01ae056f2820 is located 4128 bytes inside of 262183-byte region [0x01ae056f1800,0x01ae05731827)
allocated by thread T0 here:
#0 0x7ff69a21a7e5 in operator new(unsigned __int64) C:\repos\msvc\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_win_new_scalar_thunk.cpp:40
#1 0x7ff6992815e8 in std::_Allocate<16, struct std::_Default_allocate_traits>(unsigned __int64) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x1400915e8)
#2 0x7ff6994cc120 in std::vector<int, class std::allocator>::_Resize_reallocate(unsigned __int64, int const &) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x1402dc120)
#3 0x7ff6994cbf79 in std::vector<int, class std::allocator>::_Resize(unsigned __int64, int const &) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x1402dbf79)
#4 0x7ff699b25483 in xgboost::HostDeviceVector::Resize(unsigned __int64) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140935483)
#5 0x7ff69993e242 in xgboost::tree::MultiTargetTreeView::SplitIndex(int) const (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x14074e242)
#6 0x7ff699906efd in xgboost::tree::QuantileHist_HistUpdaterPartitionerOverrun_Test::TestBody(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140716efd)
#7 0x7ff69a0cb276 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb276)
#8 0x7ff69a0cadcb in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test , void (__cdecl testing::Test::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edadcb)
#9 0x7ff69a1231f3 in testing::TestInfo::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f331f3)
#10 0x7ff69a12361d in testing::TestSuite::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f3361d)
#11 0x7ff69a1243bd in testing::internal::UnitTestImpl::RunAllTests(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f343bd)
#12 0x7ff69a0cb2d6 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb2d6)
#13 0x7ff69a0cb1c9 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl , bool (__cdecl testing::internal::UnitTestImpl::)(void), char const *) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140edb1c9)
#14 0x7ff69a123a0f in testing::UnitTest::Run(void) (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x140f33a0f)
#15 0x7ff6997cec07 in main (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x1405dec07)
#16 0x7ff69a21bd63 in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
#17 0x7ff69a21bd63 in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#18 0x7ffbf7c5259c (C:\Windows\System32\KERNEL32.DLL+0x18001259c)
#19 0x7ffbf850af77 (C:\Windows\SYSTEM32\ntdll.dll+0x18005af77)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow (C:\gitP\dmlc\xgboost\build_amd64\testxgboost.exe+0x14074e4b5) in xgboost::tree::MultiTargetTreeView::SplitIndex(int) const
Shadow bytes around the buggy address:
0x01ae056f2580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x01ae056f2600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x01ae056f2680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x01ae056f2700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x01ae056f2780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x01ae056f2800: 00 00 00 00[fc]fc fc fc fc fc fc fc fc fc fc fc
0x01ae056f2880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
0x01ae056f2900: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
0x01ae056f2980: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
0x01ae056f2a00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
0x01ae056f2a80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==14896==ABORTING

[trivialfis]

Thank you for sharing. We run asan on Linux, but not Windows. Not sure why it's failing with MSVC. Could you please share how you generated the VS solution file? I will try to reproduce it.

[PhoebeHui]

@trivialfis, thanks for your quick response! I added the missed cmake command to the reproduces steps.

[trivialfis]

Got it. The test was written in a way to access the reserved memory instead of the allocated memory. I will try to rewrite the test later.