You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the unit tests for the document chunkers primarily revolve around class instantiation, etc.
We should definitely add tests for the .chunk_documents() method in each chunker class. Additionally, it would be good to add tests around some of the custom logic we have for building chunks based on docling's .json outputs.
On the other hand, the DocumentChunker interface may change as a result of #334, so we should decide whether or not we want to hold off and do that first. Once that refactor is complete, we will only need to test one class and it should be easier for us to parametrize and test different filetypes.
The text was updated successfully, but these errors were encountered:
Currently, the unit tests for the document chunkers primarily revolve around class instantiation, etc.
We should definitely add tests for the
.chunk_documents()
method in each chunker class. Additionally, it would be good to add tests around some of the custom logic we have for building chunks based on docling's .json outputs.On the other hand, the
DocumentChunker
interface may change as a result of #334, so we should decide whether or not we want to hold off and do that first. Once that refactor is complete, we will only need to test one class and it should be easier for us to parametrize and test different filetypes.The text was updated successfully, but these errors were encountered: