TMiniKQLWideLastCombinerPerfTest::TestTpch Crashes With Spilling Enabled

Mar 12, 2025 by ADMIN 73 views

Introduction

In the realm of database performance testing, the TMiniKQLWideLastCombinerPerfTest::TestTpch test case is a crucial benchmark for evaluating the efficiency of query execution in YDB. However, recent reports have surfaced indicating that this test case crashes when spilling is enabled. In this article, we will delve into the root cause of this issue and explore a potential solution.

Background

The TMiniKQLWideLastCombinerPerfTest::TestTpch test case is designed to evaluate the performance of the WideLastCombiner in YDB's MiniKQL engine. This test case is particularly relevant for large-scale query execution scenarios, where the WideLastCombiner plays a critical role in optimizing query performance. However, when spilling is enabled, the test case crashes, resulting in a VERIFY failure.

Symptoms

The crash occurs when writing out values in the mkql_wide_combine.cpp file, specifically at line 481. The error message indicates that the value is empty, and the Get() function fails to retrieve the embedded markers. The backtrace reveals a complex chain of function calls, ultimately leading to the NPrivate::InternalPanicImpl function.

Analysis

To understand the root cause of this issue, we need to examine the code in the mkql_wide_combine.cpp file. Specifically, we need to focus on the WriteWideItem function, which is responsible for writing out values when spilling is enabled. The error message suggests that the value is empty, which implies that the Get() function is failing to retrieve the embedded markers.

Upon closer inspection, we notice that the Get() function is called with the EMarkers::Embedded requirement. This requirement is not met, resulting in the Get() function failing to retrieve the embedded markers. This failure, in turn, causes the WriteWideItem function to crash.

Solution

To resolve this issue, we need to modify the code in the mkql_wide_combine.cpp file to ensure that the Get() function is called with the correct requirement. Specifically, we need to change the Get() function call to Get(EMarkers::None).

However, this change alone may not be sufficient to resolve the issue. We also need to apply the patch from the 0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c commit, which addresses a related issue in the WideUnboxedValuesSpillerAdapter class.

Implementation

To implement the solution, we need to follow these steps:

Apply the patch from the 0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c commit.
Modify the WriteWideItem function in the mkql_wide_combine.cpp file to call the Get() function with the EMarkers::None requirement.
Rebuild the TMiniKQLWideLastCombinerPerfTest::TestTpch test case using the ya make command.

Conclusion

In conclusion, the TMiniKQLWideLastCombinerPerfTest::TestTpch test case crashes when spilling is enabled due to a failure in the Get() function. By modifying the code in the mkql_wide_combine.cpp file and applying the patch from the 0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c commit, we can resolve this issue and ensure that the test case runs successfully.

Additional Information

For those interested in reproducing the issue, we provide the following additional information:

Apply the patch from the 0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c commit.
Run the cd yql/essentials/minikql/comp_nodes && ya make -r -ttt -F 'TMiniKQLWideLastCombinerPerfTest::TestTpch+LLVM' command to rebuild the test case.

References

Acknowledgments

Introduction

In our previous article, we delved into the root cause of the TMiniKQLWideLastCombinerPerfTest::TestTpch test case crashing when spilling is enabled. In this article, we will provide a comprehensive Q&A section to address common questions and concerns related to this issue.

Q: What is the root cause of the crash?

A: The root cause of the crash is a failure in the Get() function, which is called with the EMarkers::Embedded requirement. This requirement is not met, resulting in the Get() function failing to retrieve the embedded markers.

Q: What is the solution to this issue?

A: To resolve this issue, we need to modify the code in the mkql_wide_combine.cpp file to ensure that the Get() function is called with the correct requirement. Specifically, we need to change the Get() function call to Get(EMarkers::None). Additionally, we need to apply the patch from the 0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c commit, which addresses a related issue in the WideUnboxedValuesSpillerAdapter class.

Q: How do I apply the patch?

A: To apply the patch, you need to follow these steps:

Clone the YDB GitHub repository.
Checkout the commit with the patch (0075ea64e241e990d24c4ef2f298ca7e0f4f7d3c).
Apply the patch using the git apply command.
Rebuild the TMiniKQLWideLastCombinerPerfTest::TestTpch test case using the ya make command.

Q: What are the benefits of applying the patch?

A: Applying the patch addresses a related issue in the WideUnboxedValuesSpillerAdapter class, which can improve the performance of the TMiniKQLWideLastCombinerPerfTest::TestTpch test case.

Q: Can I reproduce the issue?

A: Yes, you can reproduce the issue by following these steps:

Clone the YDB GitHub repository.
Checkout the commit without the patch.
Run the cd yql/essentials/minikql/comp_nodes && ya make -r -ttt -F 'TMiniKQLWideLastCombinerPerfTest::TestTpch+LLVM' command to rebuild the test case.
Observe the crash when spilling is enabled.

Q: What are the implications of this issue?

A: This issue can impact the performance of the TMiniKQLWideLastCombinerPerfTest::TestTpch test case, which is a critical benchmark for evaluating the efficiency of query execution in YDB.

Q: How can I get help with this issue?

A: You can get help with this issue by:

Posting a question on the YDB GitHub issues page.
Reaching out to the YDB community on the YDB forum.
Contacting the YDB support team.