Batch generators take a long time to initialize
I'm finding that the batch generators can be quite slow to initialise (1-2 minutes), even for a moderate size dataset.
I plan to investigate this a bit more thoroughly/systematically to determine if this is always the case or only with certain input values for the constructor.
However, I was curious to know if others @frazao @padovese have had similar experiences?
The only data that needs to be loaded at initialisation is the id
column in the data table, which should be pretty fast even for large datasets since they are just integers. So I'm not sure what's causing the latency ...