TF Keras密集网络中的Tensor大小问题,训练时展平图层不工作

oxf4rvwz  于 2023-01-09  发布在  其他
关注(0)|答案(1)|浏览(143)

我被Keras模型结构的问题困扰了一段时间,我使用的是一个大小为(181,181,1)的灰度图像数据集,图像路径和类名一起存储在PandasDF中。
下面是数据生成器:

trainData = train_generator.flow_from_dataframe(
    dataframe=train_df,
    x_col='Path',
    y_col='CLASS',
    image_size=(181,181),
    color_mode='grayscale',
    class_mode='categorical',
    batch_size=64,
    interpolation='nearest',
    shuffle=True,
    seed=42
)
valData = train_generator.flow_from_dataframe(
    dataframe=valid_df,
    x_col='Path',
    y_col='CLASS',
    image_size=(181,181),
    color_mode='grayscale',
    class_mode='categorical',
    batch_size=64,
    interpolation='nearest',
    shuffle=True,
    seed=42
)

下面是我们讨论的模型:

model = Sequential([
        layers.Rescaling(scale=1./255, offset = -1,input_shape=(181,181,1)),
        layers.Flatten(),
        Dense(units=90, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
        Dense(units=45, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
        Dense(units=20, activation='softmax',kernel_initializer='he_normal',bias_initializer=biasInitializer)
    ],name='myModel')

型号总结:

model.summary()

Model: "myModel"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 rescaling_19 (Rescaling)    (None, 181, 181, 1)       0         
                                                                 
 flatten_17 (Flatten)        (None, 32761)             0         
                                                                 
 dense_44 (Dense)            (None, 90)                2948580   
                                                                 
 dense_45 (Dense)            (None, 45)                4095      
                                                                 
 dense_46 (Dense)            (None, 20)                920       
                                                                 
=================================================================
Total params: 2,953,595
Trainable params: 2,953,595
Non-trainable params: 0

模型编制:

model.compile(optimizer=k.optimizers.Adam(learning_rate=0.001) , loss='categorical_crossentropy', metrics=['accuracy'])

模型培训:

epochs = 100
# train the model to the dataset
model.fit(x=trainData,
          epochs=epochs,
          verbose=1, 
          shuffle=True,
          validation_data=valData)

每当我在模型中使用展平层时,我得到这个错误:

Node: 'myModel/flatten_17/Reshape'
Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
     [[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]

以下是完整的错误记录,如果有帮助的话:

InvalidArgumentError                      Traceback (most recent call last)
    Input In [79], in <cell line: 3>()
          1 epochs = 65
          2 # train the model to the dataset
    ----> 3 model.fit(x=trainData,
          4           epochs=epochs,
          5           verbose=1, 
          6           shuffle=True,
          7           validation_data=valData)
    
    File /usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
         65 except Exception as e:  # pylint: disable=broad-except
         66   filtered_tb = _process_traceback_frames(e.__traceback__)
    ---> 67   raise e.with_traceback(filtered_tb) from None
         68 finally:
         69   del filtered_tb
    
    File /usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
         52 try:
         53   ctx.ensure_initialized()
    ---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
         55                                       inputs, attrs, num_outputs)
         56 except core._NotOkStatusException as e:
         57   if name is not None:
    
    InvalidArgumentError: Graph execution error:
    
    Detected at node 'myModel/flatten_17/Reshape' defined at (most recent call last):
        File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
          return _run_code(code, main_globals, None,
        File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
          exec(code, run_globals)
        File "/usr/local/lib/python3.9/dist-packages/ipykernel_launcher.py", line 17, in <module>
          app.launch_new_instance()
        File "/usr/local/lib/python3.9/dist-packages/traitlets/config/application.py", line 976, in launch_instance
          app.start()
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelapp.py", line 712, in start
          self.io_loop.start()
        File "/usr/local/lib/python3.9/dist-packages/tornado/platform/asyncio.py", line 215, in start
          self.asyncio_loop.run_forever()
        File "/usr/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
          self._run_once()
        File "/usr/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
          handle._run()
        File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
          self._context.run(self._callback, *self._args)
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
          await self.process_one()
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
          await dispatch(*args)
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
          await result
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
          reply_content = await reply_content
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
          res = shell.run_cell(
        File "/usr/local/lib/python3.9/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
          return super().run_cell(*args, **kwargs)
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2881, in run_cell
          result = self._run_cell(
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2936, in _run_cell
          return runner(coro)
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
          coro.send(None)
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3135, in run_cell_async
          has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3338, in run_ast_nodes
          if await self.run_code(code, result, async_=asy):
        File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3398, in run_code
          exec(code_obj, self.user_global_ns, self.user_ns)
        File "/tmp/ipykernel_61/2241442519.py", line 3, in <cell line: 3>
          model.fit(x=trainData,
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
          try:
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1409, in fit
          `tf.distribute.experimental.ParameterServerStrategy`.
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1051, in train_function
          self.loss_tracker.reset_states()
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1040, in step_function
          def __init__(self, *args, **kwargs):
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1030, in run_step
          def compute_loss(self, x=None, y=None, y_pred=None, sample_weight=None):
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 889, in train_step
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
          try:
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 490, in __call__
          # default.
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
          try:
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
          RuntimeError: if `super().__init__()` was not called in the
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
          def error_handler(*args, **kwargs):
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/sequential.py", line 374, in call
          def build(self, input_shape=None):
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 458, in call
          def _trackable_children(self, save_type="checkpoint", **kwargs):
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
          # Read final output shapes from layers_to_output_shapes.
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
          try:
        File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
          RuntimeError: if `super().__init__()` was not called in the
        File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
          def error_handler(*args, **kwargs):
        File "/usr/local/lib/python3.9/dist-packages/keras/layers/reshaping/flatten.py", line 98, in call
          )
    Node: 'myModel/flatten_17/Reshape'
    Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
         [[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]

我使用Python 3.9,尝试了Tensorflow-gpu 2.9.1和2.11.0 ..都有相同的问题。我尝试了各种各样的建议,我可以在网上找到,但仍然没有运气。感谢任何建议谢谢!

hl0ma9xz

hl0ma9xz1#

这个问题的原因是,我在flow_from_dataframe函数中使用关键字image_size设置图像大小,而不是正确的关键字target_size ...因此flow_from_dataframe使用target_size的默认值(256,256)
修复这个问题并将此target_size关键字设置为正确的图像大小(181,181)为我修复了这个问题

相关问题