Inter-Class Relation matrix with (Left) and without (Right) Inter-Class Loss. Multiple minority classes are incorrectly predicted as car, a majority class, without Inter-Class Loss.

Abstract

Domain adaptive object detection adapts detection models to domains where annotated data is unavailable. Existing methods have been proposed to address the domain gap using the semi-supervised student-teacher framework. However, a fundamental issue arises from the class imbalance in the labelled training set, which can result in inaccurate pseudo-labels. The relationship between classes, especially where one class is a majority and the other minority, has a large impact on class bias. We propose Class-Aware Teacher (CAT) to address the class bias issue in the domain adaptation setting. In our work, we approximate the class relationships with our Inter-Class Relation module (ICRm) and exploit it to reduce the bias within the model. In this way, we are able to apply augmentations to highly related classes, both inter- and intra-domain, to boost the performance of minority classes while having minimal impact on majority classes. We further reduce the bias by implementing a class-relation weight to our classification loss. Experiments conducted on various datasets and ablation studies show that our method is able to address the class bias in the domain adaptation setting. On the Cityscapes $\to$ Foggy~Cityscapes dataset, we attained a 52.5 mAP, a substantial improvement over the 51.2 mAP achieved by the state-of-the-art method.

Framework

(a) Class-Aware Teacher (CAT) consists of: a student-teacher network; Inter-Class Relation module (ICRm), which estimates inter-class biases; Class-Relation Augmentation, which augments images to reduce the inter-class biases by mixing the cropped instances of related classes; and Inter-Class Loss, which emphasises the loss on highly misclassified minority classes. (b) Class-Relation Augmentation demonstrated on majority (Car) and minority (Bus) classes.

Results

Qualitative

We show the qualitative results of AT and CAT on the top and bottom, respectively.
CAT is able to address misclassification (col 1,2,4), false negatives (col 1,3), and false positives (col 1,3,4).
Box colour represents: Green > true positives, Blue > misclassified, Red > false negatives, Pink > false positives.

Cityscapes to Foggy Cityscapes

PASCAL VOC to Clipart1k

Citation