python 在Keras数据生成器的train-valid-test中应该如何分割数据集?

nnt7mjpx  于 2023-01-24  发布在  Python
关注(0)|答案(1)|浏览(137)

我应该如何分割Keras数据生成器train-valid-test中的数据集?我应该将train文件夹作为train有效子集加载,并加载test文件夹和测试以与model.predict(...).一起使用还是再次使用valid_generator

train_datagen = ImageDataGenerator(rescale=1./255)
valid_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(224, 224),
        class_mode='categorical')

valid_generator = valid_datagen.flow_from_directory(
        test_dir,
        target_size=(224, 224),
        class_mode='categorical')

history =  model.fit(train_generator,
                     epochs=10,
                     validation_data=valid_generator,
                     verbose=1)

,这里是预测步骤

pred = model.predict(valid_generator)
3pmvbmvn

3pmvbmvn1#

由于训练数据集的数据扩充需求,通常需要ImageDataGenerator的多个示例,而没有扩充的相同ImageDataGenerator可用于验证和测试数据集(以及它们对应的目录路径)。在训练模型之后,model.evaluate_generator可用于评估测试数据集上的模型性能。以下是给定模型和数据集有效目录的简单示例:

import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator

#An ImageDataGenerator with augmentation to generate images for training dataset
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=40,
    zoom_range=0.15,
    shear_range=0.1,
    height_shift_range=0.1,
    width_shift_range=0.1,
    horizontal_flip=True,
    fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(
    './train_dir',
    target_size=(150, 150),
    batch_size=32,
    class_mode='binary')

#An ImageDataGenerator to generate images for both validation and test datasets
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
    './val_dir',
    target_size=(150, 150),
    batch_size=32,
    class_mode='binary')
test_generator = test_datagen.flow_from_directory(
    './test_dir',
    target_size=(150, 150),
    batch_size=32,
    class_mode='binary')

#compiling the model
model.compile(
    loss='binary_crossentropy',
    optimizer=tf.keras.optimizers.Adam(1e-4),
    metrics=['acc'])

#Training
history = model.fit_generator(
    train_generator,
    steps_per_epoch=100,
    epochs=3,
    validation_data=validation_generator,
    validation_steps=50)

#Evaluation on test dataset
test_loss, test_acc = model.evaluate_generator(test_generator, steps=50)
print('test loss:', test_loss)
print('test acc:', test_acc)

相关问题