Proposal for AISdb Integration with TensorFlow and PyTorch for Custom Data-Loader Development
This issue is raised to propose the integration of AISdb with TensorFlow and PyTorch to develop a custom, high-speed data loader. This aims to streamline setting up a database, importing the data loader, and training sequence-to-sequence models with the provided data. The proposed integration would enable users to directly leverage TensorFlow and PyTorch's machine-learning libraries within the AISdb environment. This would facilitate the development and training of models, particularly those that require sequence-to-sequence learning.
For the development of the custom data loader, I suggest exploring the data-loader functionalities provided by TensorFlow (https://www.tensorflow.org/api_docs/python/tf/data/Dataset) and PyTorch (https://pytorch.org/docs/stable/data.html). They already offer robust and efficient data loading and preprocessing capabilities that could be tailored to general needs, so we can specify that to our case.
Cheers!