text diffs on huge files are slow #12406
Labels
topic: rewrite
related to the assertion rewrite mechanism
type: performance
performance or memory problem/improvement
Ok, of course they are... But usually* when a dev commits the test, the test is passing, so the dev may not notice how preposterous a diff they are inadvertently asking for.
In my case, a 5 year old test happened to be comparing text files about 2million lines long as strings, functionally:
This was fine, until the order of some fields changed and the test started hanging in CI for hours. The right thing to do is to fix this annoying test, but I thought that it might also make sense to push a fix up to pytest.
tl;dr -
_diff_text()
already knows the verbosity level - would it make sense to truncate the length of the diff calculated in "non verbose" mode? By default, the diff is truncated to the first 10 lines, so_diff_text()
is doing extra computation that the caller will never see or use.The text was updated successfully, but these errors were encountered: