Training loss increases for each epoch

Deep Speech
#1

I am training end to end model for Hindi. For each epoch, the training loss increases from a smaller value. But the overall trend of the loss seems to be decreasing over training epochs. Is this behaviour defined?
Also, Is is because of the training utterance lengths (related to sortagrad and stuff)?

Here is the plot from tensorboard

tensorboard_plot_ds
tensorboard_plot_ds1454×462 48.6 KB

#2

Care to share a bit more of context on what you do ?

#3

That is expected. As you’ve already figured out, it’s related to the sorting of utterances by size. The average loss should decrease from epoch to epoch.