AI accelerator hardware like and promise to speed up model training, but because of the way the chips are architected, earlier stages of the training pipeline (like data preprocessing) don’t benefit from the boosts. That’s why scientists at Google Brain, Google’s AI research division, propose in a a technique called “data echoing,” which they say reduces the computation used by earlier pipeline stages by reusing intermediate outputs from these stages.

According to the researchers, the best-performing data echoing algorithms can match the baseline’s predictive performance using less upstream processing, in some cases compensating for a four times slower input pipeline.

“Training a neural network requires more than just the operations that run well on accelerators, so we cannot rely on accelerator improvements alone to keep

Read More At Article Source | Article Attribution