Skip to content

Register now for

# Reference for `ultralytics/models/utils/loss.py`

Note

This file is available at https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/utils/loss.py. If you spot a problem please help fix it by contributing a Pull Request 🛠️. Thank you 🙏!

## `` ultralytics.models.utils.loss.DETRLoss

    DETRLoss(
        nc: int = 80,
        loss_gain: dict[str, float] | None = None,
        aux_loss: bool = True,
        use_fl: bool = True,
        use_vfl: bool = False,
        use_uni_match: bool = False,
        uni_match_ind: int = 0,
        gamma: float = 1.5,
        alpha: float = 0.25,
    )

Bases: `Module`

DETR (DEtection TRansformer) Loss class for calculating various loss components.

This class computes classification loss, bounding box loss, GIoU loss, and optionally auxiliary losses for the DETR object detection model.

Attributes:

Name | Type | Description
--- | --- | ---
`nc` | `int` | Number of classes.
`loss_gain` | `dict[str, float]` | Coefficients for different loss components.
`aux_loss` | `bool` | Whether to compute auxiliary losses.
`use_fl` | `bool` | Whether to use FocalLoss.
`use_vfl` | `bool` | Whether to use VarifocalLoss.
`use_uni_match` | `bool` | Whether to use a fixed layer for auxiliary branch label assignment.
`uni_match_ind` | `int` | Index of fixed layer to use if use_uni_match is True.
`matcher` | `HungarianMatcher` | Object to compute matching cost and indices.
`fl` | `FocalLoss | None` | Focal Loss object if use_fl is True, otherwise None.
`vfl` | `VarifocalLoss | None` | Varifocal Loss object if use_vfl is True, otherwise None.
`device` | `device` | Device on which tensors are stored.

Uses default loss_gain if not provided. Initializes HungarianMatcher with preset cost gains. Supports auxiliary losses and various loss types.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`nc` | `int` | Number of classes. | `80`
`loss_gain` | `dict[str, float]` | Coefficients for different loss components. | `None`
`aux_loss` | `bool` | Whether to use auxiliary losses from each decoder layer. | `True`
`use_fl` | `bool` | Whether to use FocalLoss. | `True`
`use_vfl` | `bool` | Whether to use VarifocalLoss. | `False`
`use_uni_match` | `bool` | Whether to use fixed layer for auxiliary branch label assignment. | `False`
`uni_match_ind` | `int` | Index of fixed layer for uni_match. | `0`
`gamma` | `float` | The focusing parameter that controls how much the loss focuses on hard-to-classify examples. | `1.5`
`alpha` | `float` | The balancing factor used to address class imbalance. | `0.25`

Source code in `ultralytics/models/utils/loss.py`

    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    def __init__(
        self,
        nc: int = 80,
        loss_gain: dict[str, float] | None = None,
        aux_loss: bool = True,
        use_fl: bool = True,
        use_vfl: bool = False,
        use_uni_match: bool = False,
        uni_match_ind: int = 0,
        gamma: float = 1.5,
        alpha: float = 0.25,
    ):
        """
        Initialize DETR loss function with customizable components and gains.

        Uses default loss_gain if not provided. Initializes HungarianMatcher with preset cost gains. Supports auxiliary
        losses and various loss types.

        Args:
            nc (int): Number of classes.
            loss_gain (dict[str, float], optional): Coefficients for different loss components.
            aux_loss (bool): Whether to use auxiliary losses from each decoder layer.
            use_fl (bool): Whether to use FocalLoss.
            use_vfl (bool): Whether to use VarifocalLoss.
            use_uni_match (bool): Whether to use fixed layer for auxiliary branch label assignment.
            uni_match_ind (int): Index of fixed layer for uni_match.
            gamma (float): The focusing parameter that controls how much the loss focuses on hard-to-classify examples.
            alpha (float): The balancing factor used to address class imbalance.
        """
        super().__init__()

        if loss_gain is None:
            loss_gain = {"class": 1, "bbox": 5, "giou": 2, "no_object": 0.1, "mask": 1, "dice": 1}
        self.nc = nc
        self.matcher = HungarianMatcher(cost_gain={"class": 2, "bbox": 5, "giou": 2})
        self.loss_gain = loss_gain
        self.aux_loss = aux_loss
        self.fl = FocalLoss(gamma, alpha) if use_fl else None
        self.vfl = VarifocalLoss(gamma, alpha) if use_vfl else None

        self.use_uni_match = use_uni_match
        self.uni_match_ind = uni_match_ind
        self.device = None

### `` forward

    forward(
        pred_bboxes: Tensor,
        pred_scores: Tensor,
        batch: dict[str, Any],
        postfix: str = "",
        **kwargs: Any
    ) -> dict[str, torch.Tensor]

Calculate loss for predicted bounding boxes and scores.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`pred_bboxes` | `Tensor` | Predicted bounding boxes, shape (L, B, N, 4). | required
`pred_scores` | `Tensor` | Predicted class scores, shape (L, B, N, C). | required
`batch` | `dict[str, Any]` | Batch information containing cls, bboxes, and gt_groups. | required
`postfix` | `str` | Postfix for loss names. | `''`
`**kwargs` | `Any` | Additional arguments, may include 'match_indices'. | `{}`

Returns:

Type | Description
--- | ---
`dict[str, Tensor]` | Computed losses, including main and auxiliary (if enabled).

Notes

Uses last elements of pred_bboxes and pred_scores for main loss, and the rest for auxiliary losses if self.aux_loss is True.

Source code in `ultralytics/models/utils/loss.py`

    359
    360
    361
    362
    363
    364
    365
    366
    367
    368
    369
    370
    371
    372
    373
    374
    375
    376
    377
    378
    379
    380
    381
    382
    383
    384
    385
    386
    387
    388
    389
    390
    391
    392
    393
    394
    395
    396
    397
    398
    399
    def forward(
        self,
        pred_bboxes: torch.Tensor,
        pred_scores: torch.Tensor,
        batch: dict[str, Any],
        postfix: str = "",
        **kwargs: Any,
    ) -> dict[str, torch.Tensor]:
        """
        Calculate loss for predicted bounding boxes and scores.

        Args:
            pred_bboxes (torch.Tensor): Predicted bounding boxes, shape (L, B, N, 4).
            pred_scores (torch.Tensor): Predicted class scores, shape (L, B, N, C).
            batch (dict[str, Any]): Batch information containing cls, bboxes, and gt_groups.
            postfix (str, optional): Postfix for loss names.
            **kwargs (Any): Additional arguments, may include 'match_indices'.

        Returns:
            (dict[str, torch.Tensor]): Computed losses, including main and auxiliary (if enabled).

        Notes:
            Uses last elements of pred_bboxes and pred_scores for main loss, and the rest for auxiliary losses if
            self.aux_loss is True.
        """
        self.device = pred_bboxes.device
        match_indices = kwargs.get("match_indices", None)
        gt_cls, gt_bboxes, gt_groups = batch["cls"], batch["bboxes"], batch["gt_groups"]

        total_loss = self._get_loss(
            pred_bboxes[-1], pred_scores[-1], gt_bboxes, gt_cls, gt_groups, postfix=postfix, match_indices=match_indices
        )

        if self.aux_loss:
            total_loss.update(
                self._get_loss_aux(
                    pred_bboxes[:-1], pred_scores[:-1], gt_bboxes, gt_cls, gt_groups, match_indices, postfix
                )
            )

        return total_loss

## `` ultralytics.models.utils.loss.RTDETRDetectionLoss

    RTDETRDetectionLoss(
        nc: int = 80,
        loss_gain: dict[str, float] | None = None,
        aux_loss: bool = True,
        use_fl: bool = True,
        use_vfl: bool = False,
        use_uni_match: bool = False,
        uni_match_ind: int = 0,
        gamma: float = 1.5,
        alpha: float = 0.25,
    )

Bases: `DETRLoss`

Real-Time DeepTracker (RT-DETR) Detection Loss class that extends the DETRLoss.

This class computes the detection loss for the RT-DETR model, which includes the standard detection loss as well as an additional denoising training loss when provided with denoising metadata.

Source code in `ultralytics/models/utils/loss.py`

    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    def __init__(
        self,
        nc: int = 80,
        loss_gain: dict[str, float] | None = None,
        aux_loss: bool = True,
        use_fl: bool = True,
        use_vfl: bool = False,
        use_uni_match: bool = False,
        uni_match_ind: int = 0,
        gamma: float = 1.5,
        alpha: float = 0.25,
    ):
        """
        Initialize DETR loss function with customizable components and gains.

        Uses default loss_gain if not provided. Initializes HungarianMatcher with preset cost gains. Supports auxiliary
        losses and various loss types.

        Args:
            nc (int): Number of classes.
            loss_gain (dict[str, float], optional): Coefficients for different loss components.
            aux_loss (bool): Whether to use auxiliary losses from each decoder layer.
            use_fl (bool): Whether to use FocalLoss.
            use_vfl (bool): Whether to use VarifocalLoss.
            use_uni_match (bool): Whether to use fixed layer for auxiliary branch label assignment.
            uni_match_ind (int): Index of fixed layer for uni_match.
            gamma (float): The focusing parameter that controls how much the loss focuses on hard-to-classify examples.
            alpha (float): The balancing factor used to address class imbalance.
        """
        super().__init__()

        if loss_gain is None:
            loss_gain = {"class": 1, "bbox": 5, "giou": 2, "no_object": 0.1, "mask": 1, "dice": 1}
        self.nc = nc
        self.matcher = HungarianMatcher(cost_gain={"class": 2, "bbox": 5, "giou": 2})
        self.loss_gain = loss_gain
        self.aux_loss = aux_loss
        self.fl = FocalLoss(gamma, alpha) if use_fl else None
        self.vfl = VarifocalLoss(gamma, alpha) if use_vfl else None

        self.use_uni_match = use_uni_match
        self.uni_match_ind = uni_match_ind
        self.device = None

### `` forward

    forward(
        preds: tuple[Tensor, Tensor],
        batch: dict[str, Any],
        dn_bboxes: Tensor | None = None,
        dn_scores: Tensor | None = None,
        dn_meta: dict[str, Any] | None = None,
    ) -> dict[str, torch.Tensor]

Forward pass to compute detection loss with optional denoising loss.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`preds` | `tuple[Tensor, Tensor]` | Tuple containing predicted bounding boxes and scores. | required
`batch` | `dict[str, Any]` | Batch data containing ground truth information. | required
`dn_bboxes` | `Tensor` | Denoising bounding boxes. | `None`
`dn_scores` | `Tensor` | Denoising scores. | `None`
`dn_meta` | `dict[str, Any]` | Metadata for denoising. | `None`

Returns:

Type | Description
--- | ---
`dict[str, Tensor]` | Dictionary containing total loss and denoising loss if applicable.

Source code in `ultralytics/models/utils/loss.py`

    410
    411
    412
    413
    414
    415
    416
    417
    418
    419
    420
    421
    422
    423
    424
    425
    426
    427
    428
    429
    430
    431
    432
    433
    434
    435
    436
    437
    438
    439
    440
    441
    442
    443
    444
    445
    446
    447
    448
    449
    def forward(
        self,
        preds: tuple[torch.Tensor, torch.Tensor],
        batch: dict[str, Any],
        dn_bboxes: torch.Tensor | None = None,
        dn_scores: torch.Tensor | None = None,
        dn_meta: dict[str, Any] | None = None,
    ) -> dict[str, torch.Tensor]:
        """
        Forward pass to compute detection loss with optional denoising loss.

        Args:
            preds (tuple[torch.Tensor, torch.Tensor]): Tuple containing predicted bounding boxes and scores.
            batch (dict[str, Any]): Batch data containing ground truth information.
            dn_bboxes (torch.Tensor, optional): Denoising bounding boxes.
            dn_scores (torch.Tensor, optional): Denoising scores.
            dn_meta (dict[str, Any], optional): Metadata for denoising.

        Returns:
            (dict[str, torch.Tensor]): Dictionary containing total loss and denoising loss if applicable.
        """
        pred_bboxes, pred_scores = preds
        total_loss = super().forward(pred_bboxes, pred_scores, batch)

        # Check for denoising metadata to compute denoising training loss
        if dn_meta is not None:
            dn_pos_idx, dn_num_group = dn_meta["dn_pos_idx"], dn_meta["dn_num_group"]
            assert len(batch["gt_groups"]) == len(dn_pos_idx)

            # Get the match indices for denoising
            match_indices = self.get_dn_match_indices(dn_pos_idx, dn_num_group, batch["gt_groups"])

            # Compute the denoising training loss
            dn_loss = super().forward(dn_bboxes, dn_scores, batch, postfix="_dn", match_indices=match_indices)
            total_loss.update(dn_loss)
        else:
            # If no denoising metadata is provided, set denoising loss to zero
            total_loss.update({f"{k}_dn": torch.tensor(0.0, device=self.device) for k in total_loss.keys()})

        return total_loss

### `` get_dn_match_indices `staticmethod`

    get_dn_match_indices(
        dn_pos_idx: list[Tensor], dn_num_group: int, gt_groups: list[int]
    ) -> list[tuple[torch.Tensor, torch.Tensor]]

Get match indices for denoising.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`dn_pos_idx` | `list[Tensor]` | List of tensors containing positive indices for denoising. | required
`dn_num_group` | `int` | Number of denoising groups. | required
`gt_groups` | `list[int]` | List of integers representing number of ground truths per image. | required

Returns:

Type | Description
--- | ---
`list[tuple[Tensor, Tensor]]` | List of tuples containing matched indices for denoising.

Source code in `ultralytics/models/utils/loss.py`

    451
    452
    453
    454
    455
    456
    457
    458
    459
    460
    461
    462
    463
    464
    465
    466
    467
    468
    469
    470
    471
    472
    473
    474
    475
    476
    477
    478
    @staticmethod
    def get_dn_match_indices(
        dn_pos_idx: list[torch.Tensor], dn_num_group: int, gt_groups: list[int]
    ) -> list[tuple[torch.Tensor, torch.Tensor]]:
        """
        Get match indices for denoising.

        Args:
            dn_pos_idx (list[torch.Tensor]): List of tensors containing positive indices for denoising.
            dn_num_group (int): Number of denoising groups.
            gt_groups (list[int]): List of integers representing number of ground truths per image.

        Returns:
            (list[tuple[torch.Tensor, torch.Tensor]]): List of tuples containing matched indices for denoising.
        """
        dn_match_indices = []
        idx_groups = torch.as_tensor([0, *gt_groups[:-1]]).cumsum_(0)
        for i, num_gt in enumerate(gt_groups):
            if num_gt > 0:
                gt_idx = torch.arange(end=num_gt, dtype=torch.long) + idx_groups[i]
                gt_idx = gt_idx.repeat(dn_num_group)
                assert len(dn_pos_idx[i]) == len(gt_idx), (
                    f"Expected the same length, but got {len(dn_pos_idx[i])} and {len(gt_idx)} respectively."
                )
                dn_match_indices.append((dn_pos_idx[i], gt_idx))
            else:
                dn_match_indices.append((torch.zeros([0], dtype=torch.long), torch.zeros([0], dtype=torch.long)))
        return dn_match_indices

📅 Created 1 year ago ✏️ Updated 1 year ago