如何在tensorflow中找到train\u损失和valu损失,注意神经机器翻译

kuuvgm7e  于 2021-07-14  发布在  Java
关注(0)|答案(1)|浏览(328)

我从这个教程学习神经机器翻译https://www.tensorflow.org/tutorials/text/nmt_with_attention#restore_the_latest_checkpoint_and_test
但似乎没有 train_losses 以及 val_losses 在教程中(仅限 batch_loss ).
有没有办法像我们用另一个模型那样得到损失价值历史
前任。

train_loss = seqModel.history['loss']
val_loss   = seqModel.history['val_loss']
train_acc  = seqModel.history['acc']
val_acc    = seqModel.history['val_acc']
ljsrvy3e

ljsrvy3e1#

在这些教程中,实际上有。当他们使用

for epoch in range(EPOCHS):
  start = time.time()

  enc_hidden = encoder.initialize_hidden_state()
  total_loss = 0

  for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):
    batch_loss = train_step(inp, targ, enc_hidden)
    total_loss += batch_loss

这样,他们就可以计算出 train_step 方法。但没有验证集,因此不会显示验证丢失。
根据你的评论,你需要写下 test_step 函数并在训练循环中使用。下面是获得验证损失的最小表示。

@tf.function
def test_step(inp, targ, enc_hidden):
    loss = 0 
    enc_output, enc_hidden = encoder(inp, enc_hidden, training=False)
    dec_hidden = enc_hidden
    dec_input = tf.expand_dims([targ_lang.word_index['<start>']] * BATCH_SIZE, 1)

    for t in range(1, targ.shape[1]):
      predictions, dec_hidden, _ = decoder(dec_input, dec_hidden, 
                                           enc_output, training=False)
      loss += loss_function(targ[:, t], predictions)
      dec_input = tf.expand_dims(targ[:, t], 1)

    batch_loss = (loss / int(targ.shape[1]))
    return batch_loss

要在自定义训练循环中使用它,您将执行以下操作。注意,我用的是相同的 dataset ,但实际上我们需要创建一个单独的验证数据集。

EPOCHS = 5
history = {'loss':[], 'val_loss':[]}

for epoch in range(EPOCHS):
    start = time.time()

    enc_hidden = encoder.initialize_hidden_state()
    total_loss = 0
    for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):
        batch_loss = train_step(inp, targ, enc_hidden)
        total_loss += batch_loss
    if (epoch + 1) % 2 == 0:
        checkpoint.save(file_prefix=checkpoint_prefix)
    history['loss'].append(total_loss.numpy()/steps_per_epoch)
    print(f'Epoch {epoch+1} Loss {total_loss/steps_per_epoch:.4f}')

    total_loss = 0
    for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):
        batch_loss = test_step(inp, targ, enc_hidden)
        total_loss += batch_loss
    history['val_loss'].append(total_loss.numpy()/steps_per_epoch)
    print(f'Epoch {epoch+1} Val Loss {total_loss/steps_per_epoch:.4f}')

    print(f'Time taken for 1 epoch {time.time()-start:.2f} sec\n')

接下来,

history['loss']
history['val_loss']

相关问题