我的设置如下所示:
MODEL_CHECKPOINT = "distilroberta-base"
tokenizer = AutoTokenizer.from_pretrained(PATH_TO_MY_MODEL, max_len=512, add_prefix_space=True)
model = AutoModelForTokenClassification.from_pretrained(MODEL_CHEKPOINT, num_labels=32)
ner_pipeline = pipeline(task="ner", tokenizer=tokenizer, model=model)
但是,我可以获得任意长度文档的ner预测。我想知道它是如何在内部实现的(可能是滑动窗口方法?)
暂无答案!
目前还没有任何答案,快来回答吧!