pytorch 如何在Detectron2上实现自定义FastRCNNOutputLayers?

wtlkbnrh  于 2023-03-02  发布在  其他
关注(0)|答案(1)|浏览(173)

I need to implement a new architecture on the Faster-RCNN classifier on top of Faster-RCNN model implemented in Detectron2 framework. I need to implement both a new loss function on classification and a new inference logic. As reported in the official documentations (https://detectron2.readthedocs.io/en/latest/tutorials/write-models.html#:~:text=As%20an%20example%2C%20to%20use%20custom%20loss%20function%20in%20the%20box%20head%20of%20a%20Faster%20R%2DCNN%2C%20we%20can%20do%20the%20following%3A), the most flexible way should be to register a new CustomROIHead module inheriting from StandardROIHeads, and insert as box_predictor argument a custom module which inherits from FastRCNNOutputLayers. In this last, it should be possible to overwrite the function "losses" implementing the new loss as well as the other functions of interest. The problem is that I am actually not able of doing it, since I always get an error which is likely dependent on how I inherit and instantiate objects as well as how I use the super() method. I am not an expert in OOP, that's why my feeling is that the error is in there.
Here the new Module that inherits from FastRCNNOutputLayers, in which I overwrite losses() just with a new classification loss function:

class CustomRCNNOutput(FastRCNNOutputLayers):
    def __init__(self, cfg, input_shape):
        super().__init__(cfg, input_shape)
    
    def losses(self, predictions, proposals):
        """
        Args:
            predictions: return values of :meth:`forward()`.
            proposals (list[Instances]): proposals that match the features that were used
                to compute predictions. The fields ``proposal_boxes``, ``gt_boxes``,
                ``gt_classes`` are expected.
        Returns:
            Dict[str, Tensor]: dict of losses
        """
        scores, proposal_deltas = predictions

        # parse classification outputs
        gt_classes = (
            cat([p.gt_classes for p in proposals], dim=0) if len(proposals) else torch.empty(0)
        )
        _log_classification_stats(scores, gt_classes)

        # parse box regression outputs
        if len(proposals):
            proposal_boxes = cat([p.proposal_boxes.tensor for p in proposals], dim=0)  # Nx4
            assert not proposal_boxes.requires_grad, "Proposals should not require gradients!"
            # If "gt_boxes" does not exist, the proposals must be all negative and
            # should not be included in regression loss computation.
            # Here we just use proposal_boxes as an arbitrary placeholder because its
            # value won't be used in self.box_reg_loss().
            gt_boxes = cat(
                [(p.gt_boxes if p.has("gt_boxes") else p.proposal_boxes).tensor for p in proposals],
                dim=0,
            )
        else:
            proposal_boxes = gt_boxes = torch.empty((0, 4), device=proposal_deltas.device)

        if self.use_sigmoid_ce:
            loss_cls = self.sigmoid_cross_entropy_loss(scores, gt_classes)
        else:
            loss_cls = MY_CUSTOM_LOSS(scores, gt_classes, self.num_classes)

        losses = {
            "loss_cls": loss_cls,
            "loss_box_reg": self.box_reg_loss(
                proposal_boxes, gt_boxes, proposal_deltas, gt_classes
            ),
        }
        return {k: v * self.loss_weight.get(k, 1.0) for k, v in losses.items()}

Then, in my training script, I firstly register the new ROI Head module, I call it from in the cfg file using the reported function and after I try to run the training. (I have just reported here an extract of the training script, the other parts shouldn't be of interest).

def get_train_cfg(config_file_path, checkpoint_url, train_dataset_name, test_dataset_name, num_classes, device, output_dir):
    cfg = get_cfg() 

    cfg.merge_from_file(model_zoo.get_config_file(config_file_path))
    cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(checkpoint_url)
    cfg.DATASETS.TRAIN = (train_dataset_name,)
    cfg.DATASETS.TEST = (test_dataset_name,)

    cfg.DATALOADER.NUM_WORKERS = 2

    cfg.SOLVER.IMS_PER_BATCH = 2
    cfg.SOLVER.BASE_LR = 0.0002
    cfg.SOLVER.MAX_ITER = 2500
    cfg.SOLVER.STEPS = []   # for LR scheduling

    cfg.MODEL.ROI_HEADS.NUM_CLASSES = num_classes
    cfg.MODEL.DEVICE = device
    cfg.OUTPUT_DIR = output_dir
    cfg.MODEL.ROI_HEADS.NAME = "CustomROIHeads"

    return cfg
@ROI_HEADS_REGISTRY.register()
class CustomROIHeads(StandardROIHeads):
  def __init__(self, cfg):
    super().__init__(cfg, 
                     box_predictor=CustomRCNNOutput(cfg))
def main():
    cfg = get_train_cfg(config_file_path, checkpoint_url, train_dataset_name, test_dataset_name,
     num_classes, device, output_dir)

    with open(cfg_save_path, 'wb') as f:
        pickle.dump(cfg, f, protocol=pickle.HIGHEST_PROTOCOL)   # this will save the cfg
    
    os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)

    trainer = DefaultTrainer(cfg)   # call the default trainer engine and pass the custom cfg
    trainer.resume_or_load(resume=False)

    trainer.train() # to start the training

# to call the main method
if __name__ == "__main__":
    main()

Here is the error that I receive: `

Traceback (most recent call last):
  File "d:\Personale\train.py", line 107, in <module>
    main()
  File "d:\Personale\train.py", line 100, in main
    trainer = DefaultTrainer(cfg)   # call the default trainer engine and pass the custom cfg
  File "d:\personale\detectron2\detectron2\engine\defaults.py", line 376, in __init__
    model = self.build_model(cfg)
  File "d:\personale\detectron2\detectron2\engine\defaults.py", line 514, in build_model
    model = build_model(cfg)
  File "d:\personale\detectron2\detectron2\modeling\meta_arch\build.py", line 22, in build_model     
    model = META_ARCH_REGISTRY.get(meta_arch)(cfg)
  File "d:\personale\detectron2\detectron2\config\config.py", line 189, in wrapped
    explicit_args = _get_args_from_config(from_config_func, *args, **kwargs)
  File "d:\personale\detectron2\detectron2\config\config.py", line 245, in _get_args_from_config     
    ret = from_config_func(*args, **kwargs)
  File "d:\personale\detectron2\detectron2\modeling\meta_arch\rcnn.py", line 77, in from_config      
    "roi_heads": build_roi_heads(cfg, backbone.output_shape()),
  File "d:\personale\detectron2\detectron2\modeling\roi_heads\roi_heads.py", line 43, in build_roi_heads
    return ROI_HEADS_REGISTRY.get(name)(cfg, input_shape)
TypeError: CustomROIHeads.__init__() takes 2 positional arguments but 3 were given

`
Do you have any idea about how to solve my problem? It's for my Master Thesis project and currently what I did is to directly implement the new loss function on the source code and call the function in losses. It works, so it is not a problem of the algorithm, but just about how to implement it in detectron2 using a new module. This is important for me because once implementing also the new inference mode, it will be not flexible to implement it directly in the source code. Also for sharing, it would be impossible. I need to find the correct way to introduce a new module.
Thank you in advance!

eqqqjvef

eqqqjvef1#

根据链接的文档,您似乎缺少input_shape参数。您可以根据您正在使用的模型硬编码输入形状,例如,如下所示,使用input_shape=1024:图片来源:http://github.com/facebookresearch/检测器2/blob/主界面/配置界面/通用界面/模型界面/屏蔽界面。图片编号:~:文本=形状规格(通道%3D1024)
示例变更为:

@ROI_HEADS_REGISTRY.register()
class CustomROIHeads(StandardROIHeads):
  def __init__(self, cfg, input_shape):
    super().__init__(cfg, input_shape, 
                     box_predictor=CustomRCNNOutput(cfg,input_shape=1024))

相关问题