Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PythonDataSource is creating a repr #5633

Open
vyeevani opened this issue Oct 3, 2024 · 1 comment
Open

PythonDataSource is creating a repr #5633

vyeevani opened this issue Oct 3, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@vyeevani
Copy link

vyeevani commented Oct 3, 2024 •
edited
Loading

PythonDataSource has repr defined on it by the data class. This results in the flat examples getting serialized. This can be a huge performance penalty. The other sources have their repr's defined by the base class. Therefore, it's relatively safe to do this for tensor flow dataset as well.
@vyeevani vyeevani added the bug Something isn't working label Oct 3, 2024
@vyeevani vyeevani changed the title repr- PythonDataSource is creating a repr Oct 3, 2024
@vyeevani
Copy link
Author

vyeevani commented Oct 3, 2024

This is the root cause of: google/grain#593

@vyeevani vyeevani mentioned this issue Oct 3, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vyeevani