unilm 忽略了从预训练模型初始化的VisionTransformer的权重

64jmpszr  于 5个月前  发布在  其他
关注(0)|答案(2)|浏览(49)

我正在尝试加载微调过的 "beit_large_patch16_224_pt22k_ft22k.pth" 模型,但我得到了以下警告:
Ignored weights of VisionTransformer not initialized from pretrained model: ['blocks.0.attn.relative_position_index', 'blocks.1.attn.relative_position_index', 'blocks.2.attn.relative_position_index', 'blocks.3.attn.relative_position_index', 'blocks.4.attn.relative_position_index', 'blocks.5.attn.relative_position_index', 'blocks.6.attn.relative_position_index', 'blocks.7.attn.relative_position_index', 'blocks.8.attn.relative_position_index', 'blocks.9.attn.relative_position_index', 'blocks.10.attn.relative_position_index', 'blocks.11.attn.relative_position_index', 'blocks.12.attn.relative_position_index', 'blocks.13.attn.relative_position_index', 'blocks.14.attn.relative_position_index', 'blocks.15.attn.relative_position_index', 'blocks.16.attn.relative_position_index', 'blocks.17.attn.relative_position_index', 'blocks.18.attn.relative_position_index', 'blocks.19.attn.relative_position_index', 'blocks.20.attn.relative_position_index', 'blocks.21.attn.relative_position_index', 'blocks.22.attn.relative_position_index', 'blocks.23.attn.relative_position_index']
这有问题吗?我可以忽略它并继续吗?
提前感谢,
-Francisco

相关问题