如何在Keras中合并LSTM和CNN模型

toe95027  于 2022-11-13  发布在  其他
关注(0)|答案(3)|浏览(222)

我有一些用户的个人资料图片和时间序列数据(由这些用户生成的事件),为了进行二元分类,我编写了两个模型:LSTM和CNN独立工作很好。但我真正想实现的是将这些模型串联起来。
下面是我的LSTM模型:

input1_length = X_train.shape[1]
input1_dim = X_train.shape[2]

input2_length = X_inter_train.shape[1]
input2_dim = X_inter_train.shape[2]

output_dim = 1

input1 = Input(shape=(input1_length, input1_dim))
input2 = Input(shape=(input2_length, input2_dim))

lstm1 = LSTM(20)(input1)
lstm2 = LSTM(10)(input2)

lstm1 = Dense(256, activation='relu')(lstm1)
lstm1 = Dropout(0.5)(lstm1)
lstm1 = Dense(12, activation='relu')(lstm1)

lstm2 = Dense(256, activation='relu')(lstm2)
#lstm2 = Dropout(0.5)(lstm2)
lstm2 = Dense(12, activation='relu')(lstm2)

merge = concatenate([lstm1, lstm2])

# interpretation model
lstm = Dense(128, activation='relu')(merge)

output = Dense(output_dim, activation='sigmoid')(lstm)

model = Model([input1, input2], output)
optimizer = RMSprop(lr=1e-3, decay=0.0)

model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
model.summary()

CNN模式:

def gen_img_model(input_dim=(75,75,3)):
    input = Input(shape=input_dim)

    conv = Conv2D(32, kernel_size=(3,3), activation='relu')(input)
    conv = MaxPooling2D((3,3))(conv)
    conv = Dropout(0.2)(conv)

    conv = BatchNormalization()(conv)

    dense = Dense(128, activation='relu', name='img_features')(conv)
    dense = Dropout(0.2)(dense)

    output = Dense(1, activation='sigmoid')(dense)

    optimizer = RMSprop(lr=1e-3, decay=0.0)

    model = Model(input, output)
    model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])

    return model

以下是CNN的培训方式:

checkpoint_name = './keras_img_checkpoint/img_model'
callbacks = [ModelCheckpoint(checkpoint_name, save_best_only=True)]

img_model = gen_img_model((75,75,3))

# batch size for img model
batch_size = 200

train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

val_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

# train gen for img model
train_generator = train_datagen.flow_from_directory(
        './dataset/train/',
        target_size=(75, 75),
        batch_size=batch_size,
        class_mode='binary')

val_generator = val_datagen.flow_from_directory(
        './dataset/val/', 
        target_size=(75, 75),
        batch_size=batch_size,
        class_mode='binary')

STEP_SIZE_TRAIN = train_generator.n // train_generator.batch_size
STEP_SIZE_VAL = val_generator.n // val_generator.batch_size

img_model.fit_generator(
        train_generator,
        steps_per_epoch=STEP_SIZE_TRAIN,
        validation_data=val_generator,
        validation_steps=800 // batch_size,
        epochs=1,
        verbose=1,
        callbacks=callbacks
)

将LSTM和CNN模型连接在一起的最佳方式是什么?

2w3rbyxf

2w3rbyxf1#

This is how you can merge two Deep learning models.

    model1 = Sequential()
    #input 
    model1.add(Dense(32, input_shape=(NUM_FEAT1,1)))
    model1.add(Activation("elu"))
    model1.add(Dropout(0.5))
    model1.add(Dense(16))
    model1.add(Activation("elu"))
    model1.add(Dropout(0.25))
    model1.add(Flatten())

    model2 = Sequential()
    #input 
    model2.add(Dense(32, input_shape=(NUM_FEAT1,1)))
    model2.add(Activation("elu"))
    model2.add(Dropout(0.5))
    model2.add(Dense(16))
    model2.add(Activation("elu"))
    model2.add(Dropout(0.25))
    model2.add(Flatten())

merged = Concatenate()([model1.output,model2.output])
z = Dense(128, activation="relu")(merged)
z = Dropout(0.25)(z)
z = Dense(1024, activation="relu")(z)
z = Dense(1, activation="sigmoid")(z)

model = Model(inputs=[model1.input, model2.input], outputs=z)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

        model.fit([x_train[train_index][:,:66], x_train[train_index][:,66:132], y_train[train_index], batch_size=100, epochs=100, verbose=2)

通过这个,你可以根据你的需要向你的模型提供2种不同类型的数据,如第一个模型中的图像和第二个模型中的文本数据。

rqdpfwrv

rqdpfwrv2#

您可以使用Keras在一个模型中添加CNN和LSTM图层。您可能会遇到形状问题。
示例:

def CNN_LSTM():
    model = Sequential()
    model.add(Convolution2D(input_shape = , filters = , kernel_size = 
    , activation = )
    model.add(LSTM(units = , )

   return model

您只需添加参数。希望这对您有所帮助。

yyyllmsg

yyyllmsg3#

我认为这并不能完全回答你的问题,但是你可以考虑在你的数据集上运行几十个ML模型,看看哪一个效果最好。你可以使用AoutML或DataRobot来完成这些任务。
https://heartbeat.fritz.ai/automl-the-next-wave-of-machine-learning-5494baac615f
https://www.forbes.com/sites/janakirammsv/2018/06/04/datarobot-puts-the-power-of-machine-learning-in-the-hands-of-business-analysts/#5e9586ea4306

相关问题