我试图通过here获取代码生成模型的“last_hidden_state”（如here所述）。我无法确定如何继续，只能手动下载每个代码生成模型，并使用以下代码检查其键是否具有该属性-

import numpy as np
from datasets import load_dataset
from transformers import AutoTokenizer
from transformers import AutoModel, AutoModelForCausalLM
import torch
from sklearn.linear_model import LogisticRegression

from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("codeparrot/codeparrot")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = AutoModelWithLMHead.from_pretrained("codeparrot/codeparrot").to(device)

inputs = tokenizer("def hello_world():", return_tensors="pt")
inputs = {k:v.to(device) for k,v in inputs.items()}
with torch.no_grad():
    outputs = model(**inputs)
print(outputs.keys())

到目前为止，我在CodeParrot和InCoder上尝试了这种策略，但没有成功。也许有更好的方法来访问隐藏层的值？