问题类型
Bug
来源
source
Tensorflow版本
2.7.4
自定义代码
是
OS平台和发行版
Kaggle内核
移动设备
- 无响应*
Python版本
3.7.12
Bazel版本
- 无响应*
GCC/编译器版本
- 无响应*
CUDA/cuDNN版本
- 无响应*
GPU型号和内存
- 无响应*
当前行为？
`hub.KerasLayer` loses it's shape under `strategy.scope()`重现问题的独立代码
To reproduce: https://www.kaggle.com/code/maifeeulasad/tfhub-bert-with-scope/
Code:
def get_model():
bert_preprocess = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3")
bert_encoder = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4")
text_input = Input(shape=(), dtype=tf.string, name='text')
preprocessed_text = bert_preprocess(text_input)
outputs = bert_encoder(preprocessed_text)
output_sequence = outputs['sequence_output']
x = Flatten()(output_sequence)
x = Dense(NUM_CLASS, activation='sigmoid')(x)
model = Model(inputs=[text_input], outputs = [x])
return model
# 1
optimizer = Adam()
model = get_model()
model.compile(loss=CategoricalCrossentropy(from_logits=True),optimizer=optimizer,metrics=[Accuracy(), ],)
model.summary()
# 2 <--- issue here
with strategy.scope():
optimizer = Adam()
model_scoped = get_model()
model_scoped.compile(loss=CategoricalCrossentropy(from_logits=True),optimizer=optimizer,metrics=[Accuracy(), ],)### Relevant log output
```shell
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/tmp/ipykernel_19/3651651963.py in <module>
1 with strategy.scope():
2 optimizer = Adam()
----> 3 model_scoped = get_model()
4 model_scoped.compile(loss=CategoricalCrossentropy(from_logits=True),optimizer=optimizer,metrics=[Accuracy(), ],)
5
/tmp/ipykernel_19/304083462.py in get_model()
10 output_sequence = outputs['sequence_output']
11 x = Flatten()(output_sequence)
---> 12 x = Dense(NUM_CLASS, activation='sigmoid')(x)
13
14 model = Model(inputs=[text_input], outputs = [x])
/opt/conda/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
/opt/conda/lib/python3.7/site-packages/keras/layers/core/dense.py in build(self, input_shape)
137 last_dim = tf.compat.dimension_value(input_shape[-1])
138 if last_dim is None:
--> 139 raise ValueError('The last dimension of the inputs to a Dense layer '
140 'should be defined. Found None. '
141 f'Full input shape received: {input_shape}')
ValueError: The last dimension of the inputs to a Dense layer should be defined. Found None. Full input shape received: (None, None)
Related issue: https://github.com/tensorflow/hub/issues/870
3条答案
按热度按时间deikduxw1#
相关： tensorflow/hub#870
bprjcwpo2#
好的，@maifeeulasad !
感谢您分享您关于BERT的分布策略的观察。
我只能使用2.7来复现这个问题。
@SuryanarayanaY !
请问您能否查看一下这个问题？附件中的gist供参考。
谢谢！
but5z9lq3#
你好，@maifeeulasad ,
我在一台带有2个GPU的虚拟机上尝试复制你的问题，并附上了下面的错误日志。
然而在Colab上，由于我们只有一个GPU,因此如果我使用
tf.distribute.OneDeviceStrategy,就不会出现错误，如附加的gist所示。在分析错误后，我会再次回来。谢谢！