Remove arrange ops

docs/apis/data_cn.md

@@ -21,7 +21,7 @@
 |`num_workers`|`int` \| `str`|加载数据时使用的辅助进程数。若设置为`'auto'`，则按照如下规则确定使用进程数：当CPU核心数大于16时，使用8个数据读取辅助进程；否则，使用CPU核心数一半数量的辅助进程。|`'auto'`|
 |`shuffle`|`bool`|是否随机打乱数据集中的样本。|`False`|
 |`with_seg_labels`|`bool`|当数据集中包含每个时相的分割标签时，请指定此选项为`True`。|`False`|
-|`binarize_labels`|`bool`|若为`True`，则在除`Arrange`以外的所有数据变换算子处理完毕后对变化标签（和分割标签）进行二值化操作。例如，将取值为{0, 255}的标签二值化到{0, 1}。|`False`|
+|`binarize_labels`|`bool`|若为`True`，则在所有数据变换算子处理完毕后对变化标签（和分割标签）进行二值化操作。例如，将取值为{0, 255}的标签二值化到{0, 1}。|`False`|
 
 `CDDataset`对file list的要求如下：
 
@@ -179,32 +179,24 @@
 ```python
 # 使用Compose组合多种变换方式。Compose中包含的变换将按顺序串行执行
 train_transforms = T.Compose([
-    # 读取影像
-    T.DecodeImg(),
     # 将影像缩放到512x512大小
     T.Resize(target_size=512),
     # 以50%的概率实施随机水平翻转
     T.RandomHorizontalFlip(prob=0.5),
     # 将数据归一化到[-1,1]
     T.Normalize(
-        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
-    # 挑选并排布后续需要使用的信息
-    T.ArrangeSegmenter('train')
+        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
 ])
 ```
 
-一般来说，`Compose`对象接受的数据变换算子列表中，首个元素为`paddlers.transforms.DecodeImg`对象，用于读取影像数据；最后一个元素为[`Arrange`算子](https://github.com/PaddlePaddle/PaddleRS/blob/develop/paddlers/transforms/operators.py)，用于从`sample`字典中抽取信息并排列。
-
-对于图像分割任务和变化检测任务的验证集而言，可在`Arrange`算子之前插入`ReloadMask`算子以重新加载真值标签。示例如下：
+对于图像分割任务和变化检测任务的验证集而言，可使用`ReloadMask`算子重新加载真值标签。示例如下：
 
 ```python
 eval_transforms = T.Compose([
-    T.DecodeImg(),
     T.Resize(target_size=512),
     T.Normalize(
         mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
     # 重新加载标签
-    T.ReloadMask(),
-    T.ArrangeSegmenter('eval')
+    T.ReloadMask()
 ])
 ```

docs/apis/data_en.md

@@ -21,7 +21,7 @@ The initialization parameter list is as follows:
 |`num_workers`|`int` \| `str`|Number of auxiliary processes used when loading data. If it is set to `'auto'`, use the following rules to determine the number of processes to use: When the number of CPU cores is greater than 16, 8 data read auxiliary processes are used; otherwise, the number of auxiliary processes is set to half the counts of CPU cores.|`'auto'`|
 |`shuffle`|`bool`|Whether to randomly shuffle the samples in the dataset.|`False`|
 |`with_seg_labels`|`bool`|Specify this option as `True` when the dataset contains segmentation labels for each phase.|`False`|
-|`binarize_labels`|`bool`|If it is `True`, the change labels (and the segmentation label) are binarized after all data transformation operators except `Arrange` are applied. For example, binarize labels valued in {0, 255} to {0, 1}.|`False`|
+|`binarize_labels`|`bool`|If it is `True`, the change labels (and the segmentation label) are binarized after all data transformation operators are applied. For example, binarize labels valued in {0, 255} to {0, 1}.|`False`|
 
 The requirements of `CDDataset` for the file list are as follows:
 
@@ -180,32 +180,24 @@ Use `paddlers.transforms.Compose` to combine a set of data transformation operat
 # Compose a variety of transformations using Compose.
 # The transformations contained in Compose will be executed sequentially in sequence
 train_transforms = T.Compose([
-    # Read Image
-    T.DecodeImg(),
     # Scale the image to 512x512
     T.Resize(target_size=512),
     # Perform a random horizontal flip with a 50% probability
     T.RandomHorizontalFlip(prob=0.5),
     # Normalize data to [-1,1]
     T.Normalize(
-        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
-    # Select and organize the information that needs to be used later
-    T.ArrangeSegmenter('train')
+        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
 ])
 ```
 
-Generally, in the list of data transform operators accepted by a `Compose` object, the first element is `paddlers.Transforms.DecodeImg` object, used to read image data; the last element is [`Arrange` Operator](https://github.com/PaddlePaddle/PaddleRS/blob/develop/paddlers/transforms/operators.py, used to extract and arrange information from the `sample` dictionary.
-
-For the validation dataset of image segmentation task and change detection task, the `ReloadMask` operator can be inserted before the `Arrange` operator to reload the ground-truth label. The following is an example:
+For the validation dataset of image segmentation task and change detection task, a `ReloadMask` operator can be added to reload the ground-truth label. The following is an example:
 
 ```python
 eval_transforms = T.Compose([
-    T.DecodeImg(),
     T.Resize(target_size=512),
     T.Normalize(
         mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
     # Reload label
-    T.ReloadMask(),
-    T.ArrangeSegmenter('eval')
+    T.ReloadMask()
 ])
 ```

examples/c2fnet/train.py

@@ -24,25 +24,20 @@ NUM_BANDS = 3
 # 使用Compose组合多种变换方式。Compose中包含的变换将按顺序串行执行
 # API说明：https://github.com/PaddlePaddle/PaddleRS/blob/develop/docs/apis/data.md
 train_transforms = T.Compose([
-    # 读取影像
-    T.DecodeImg(),
     # 将影像缩放到512x512大小
     T.Resize(target_size=512),
     # 以50%的概率实施随机水平翻转
     T.RandomHorizontalFlip(prob=0.5),
     # 将数据归一化到[-1,1]
     T.Normalize(
-        mean=[0.5] * NUM_BANDS, std=[0.5] * NUM_BANDS),
-    T.ArrangeSegmenter('train')
+        mean=[0.5] * NUM_BANDS, std=[0.5] * NUM_BANDS)
 ])
 
 eval_transforms = T.Compose([
-    T.DecodeImg(),
     # 验证阶段与训练阶段的数据归一化方式必须相同
     T.Normalize(
         mean=[0.5] * NUM_BANDS, std=[0.5] * NUM_BANDS),
-    T.ReloadMask(),
-    T.ArrangeSegmenter('eval')
+    T.ReloadMask()
 ])
 
 # 分别构建训练和验证所用的数据集

examples/rs_research/configs/levircd/ablation/custom_model_c.yaml

@@ -3,6 +3,6 @@ _base_: ../levircd.yaml
 save_dir: ./exp/levircd/ablation/att_c/
 
 model: !Node
-    type: CustomTrainer
-    args:
-        att_types: c
+  type: CustomTrainer
+  args:
+    att_types: c

examples/rs_research/configs/levircd/ablation/custom_model_t.yaml

@@ -3,6 +3,6 @@ _base_: ../levircd.yaml
 save_dir: ./exp/levircd/ablation/att_t/
 
 model: !Node
-    type: CustomTrainer
-    args:
-        att_types: t
+  type: CustomTrainer
+  args:
+    att_types: t

examples/rs_research/configs/levircd/custom_model.yaml

@@ -3,4 +3,4 @@ _base_: ./levircd.yaml
 save_dir: ./exp/levircd/custom_model/
 
 model: !Node
-    type: CustomTrainer
+  type: CustomTrainer

examples/rs_research/configs/levircd/fc_ef.yaml

@@ -3,4 +3,4 @@ _base_: ./levircd.yaml
 save_dir: ./exp/levircd/fc_ef/
 
 model: !Node
-    type: FCEarlyFusion
+  type: FCEarlyFusion

examples/rs_research/configs/levircd/fc_siam_conc.yaml

@@ -3,4 +3,4 @@ _base_: ./levircd.yaml
 save_dir: ./exp/levircd/fc_siam_conc/
 
 model: !Node
-    type: FCSiamConc
+  type: FCSiamConc

examples/rs_research/configs/levircd/fc_siam_diff.yaml

@@ -3,4 +3,4 @@ _base_: ./levircd.yaml
 save_dir: ./exp/levircd/fc_siam_diff/
 
 model: !Node
-    type: FCSiamDiff
+  type: FCSiamDiff

examples/rs_research/configs/levircd/levircd.yaml

@@ -1,69 +1,63 @@
 # Basic configurations of LEVIR-CD dataset
 
 datasets:
-    train: !Node
-        type: CDDataset
-        args: 
-            data_dir: ./data/levircd/
-            file_list: ./data/levircd/train.txt
-            label_list: null
-            num_workers: 2
-            shuffle: True
-            with_seg_labels: False
-            binarize_labels: True
-    eval: !Node
-        type: CDDataset
-        args:
-            data_dir: ./data/levircd/
-            file_list: ./data/levircd/val.txt
-            label_list: null
-            num_workers: 0
-            shuffle: False
-            with_seg_labels: False
-            binarize_labels: True
+  train: !Node
+    type: CDDataset
+    args:
+      data_dir: ./data/levircd/
+      file_list: ./data/levircd/train.txt
+      label_list: null
+      num_workers: 2
+      shuffle: True
+      with_seg_labels: False
+      binarize_labels: True
+  eval: !Node
+    type: CDDataset
+    args:
+      data_dir: ./data/levircd/
+      file_list: ./data/levircd/val.txt
+      label_list: null
+      num_workers: 0
+      shuffle: False
+      with_seg_labels: False
+      binarize_labels: True
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomFlipOrRotate
-          args:
-            probs: [0.35, 0.35]
-            probsf: [0.5, 0.5, 0, 0, 0]
-            probsr: [0.33, 0.34, 0.33]
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomFlipOrRotate
+      args:
+        probs: [0.35, 0.35]
+        probsf: [0.5, 0.5, 0, 0, 0]
+        probsr: [0.33, 0.34, 0.33]
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
 download_on: False
 
 num_epochs: 50
 train_batch_size: 8
 optimizer: !Node
-    type: Adam
-    args:
-        learning_rate: !Node
-            type: StepDecay
-            module: paddle.optimizer.lr
-            args:
-                learning_rate: 0.002
-                step_size: 35000
-                gamma: 0.2
+  type: Adam
+  args:
+    learning_rate: !Node
+      type: StepDecay
+      module: paddle.optimizer.lr
+      args:
+        learning_rate: 0.002
+        step_size: 35000
+        gamma: 0.2
 save_interval_epochs: 5
 log_interval_steps: 50
 save_dir: ./exp/levircd/

examples/rs_research/train_cd.py

@@ -18,8 +18,6 @@ EXP_DIR = 'exp/levircd/custom_model/'
 # 使用Compose组合多种变换方式。Compose中包含的变换将按顺序串行执行
 # API说明：https://github.com/PaddlePaddle/PaddleRS/blob/develop/docs/apis/data.md
 train_transforms = T.Compose([
-    # 读取影像
-    T.DecodeImg(),
     # 随机翻转和旋转
     T.RandomFlipOrRotate(
         # 以0.35的概率执行随机翻转，0.35的概率执行随机旋转
@@ -30,16 +28,13 @@ train_transforms = T.Compose([
         probsr=[0.33, 0.34, 0.33]),
     # 将数据归一化到[-1,1]
     T.Normalize(
-        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
-    T.ArrangeChangeDetector('train')
+        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
 ])
 
 eval_transforms = T.Compose([
-    T.DecodeImg(),
     # 验证阶段与训练阶段的数据归一化方式必须相同
     T.Normalize(
-        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
-    T.ArrangeChangeDetector('eval')
+        mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
 ])
 
 # 分别构建训练、验证和测试所用的数据集

paddlers/datasets/base.py

@@ -12,16 +12,18 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from copy import deepcopy
+import copy
 
 from paddle.io import Dataset
 from paddle.fluid.dataloader.collate import default_collate_fn
 
 from paddlers.utils import get_num_workers
-from paddlers.transforms import construct_sample_from_dict, Compose, DecodeImg
+from paddlers.transforms import construct_sample_from_dict, Compose
 
 
 class BaseDataset(Dataset):
+    _KEYS_TO_KEEP = None
+    _KEYS_TO_DISCARD = None
     _collate_trans_info = False
 
     def __init__(self, data_dir, label_list, transforms, num_workers, shuffle):
@@ -29,8 +31,9 @@ class BaseDataset(Dataset):
 
         self.data_dir = data_dir
         self.label_list = label_list
-        self.transforms = deepcopy(transforms)
-        self.normal_transforms(self.transforms)
+        self.transforms = copy.deepcopy(transforms)
+        if isinstance(self.transforms, list):
+            self.transforms = Compose(self.transforms)
 
         self.num_workers = get_num_workers(num_workers)
         self.shuffle = shuffle
@@ -39,20 +42,25 @@ class BaseDataset(Dataset):
         sample = construct_sample_from_dict(self.file_list[idx])
         # `trans_info` will be used to store meta info about image shape
         sample['trans_info'] = []
-        outputs, trans_info = self.transforms(sample)
-        return outputs, trans_info
+        sample, trans_info = self.transforms(sample)
+        return sample, trans_info
 
     def collate_fn(self, batch):
+        if self._KEYS_TO_KEEP is not None:
+            new_batch = []
+            for sample, trans_info in batch:
+                new_sample = type(sample)()
+                for key in self._KEYS_TO_KEEP:
+                    if key in sample:
+                        new_sample[key] = sample[key]
+                new_batch.append((new_sample, trans_info))
+            batch = new_batch
+        if self._KEYS_TO_DISCARD:
+            for key in self._KEYS_TO_DISCARD:
+                for s, _ in batch:
+                    s.pop(key, None)
         if self._collate_trans_info:
             return default_collate_fn(
                 [s[0] for s in batch]), [s[1] for s in batch]
         else:
             return default_collate_fn([s[0] for s in batch])
-
-    def normal_transforms(self, trans):
-        # NOTE: add `DecodeImg` and convert to `Compose`
-        if isinstance(trans, list):
-            trans = Compose(trans)
-        if not isinstance(trans.transforms[0], DecodeImg):
-            trans.transforms.insert(0, DecodeImg())
-        self.transforms = trans

paddlers/datasets/cd_dataset.py

@@ -44,6 +44,7 @@ class CDDataset(BaseDataset):
             Defaults to False.
     """
 
+    _KEYS_TO_KEEP = ['image', 'image2', 'mask', 'aux_masks']
     _collate_trans_info = True
 
     def __init__(self,
@@ -132,7 +133,7 @@ class CDDataset(BaseDataset):
     def __getitem__(self, idx):
         sample = construct_sample_from_dict(self.file_list[idx])
         sample['trans_info'] = []
-        sample = self.transforms.apply_transforms(sample)
+        sample, trans_info = self.transforms(sample)
 
         if self.binarize_labels:
             # Requires 'mask' to exist
@@ -141,9 +142,7 @@ class CDDataset(BaseDataset):
                 sample['aux_masks'] = list(
                     map(self._binarize, sample['aux_masks']))
 
-        outputs = self.transforms.arrange_outputs(sample)
-
-        return outputs, sample['trans_info']
+        return sample, trans_info
 
     def __len__(self):
         return len(self.file_list)

paddlers/datasets/clas_dataset.py

@@ -34,6 +34,8 @@ class ClasDataset(BaseDataset):
         shuffle (bool, optional): Whether to shuffle the samples. Defaults to False.
     """
 
+    _KEYS_TO_KEEP = ['image', 'label']
+
     def __init__(self,
                  data_dir,
                  file_list,

paddlers/datasets/res_dataset.py

@@ -36,6 +36,7 @@ class ResDataset(BaseDataset):
             restoration tasks. Defaults to None.
     """
 
+    _KEYS_TO_KEEP = ['image', 'target']
     _collate_trans_info = True
 
     def __init__(self,

paddlers/datasets/seg_dataset.py

@@ -13,7 +13,6 @@
 # limitations under the License.
 
 import os.path as osp
-import copy
 
 from .base import BaseDataset
 from paddlers.utils import logging, get_encoding, norm_path, is_pic
@@ -35,6 +34,7 @@ class SegDataset(BaseDataset):
         shuffle (bool, optional): Whether to shuffle the samples. Defaults to False.
     """
 
+    _KEYS_TO_KEEP = ['image', 'mask']
     _collate_trans_info = True
 
     def __init__(self,

paddlers/datasets/voc.py

@@ -324,8 +324,8 @@ class VOCDetDataset(BaseDataset):
                 DecodeImg(to_rgb=False)(sample),
                 DecodeImg(to_rgb=False)(sample_mix)
             ])
-        sample = self.transforms(sample)
-        return sample
+        sample, trans_info = self.transforms(sample)
+        return sample, trans_info
 
     def __len__(self):
         return self.num_samples

paddlers/deploy/predictor.py

@@ -152,22 +152,6 @@ class Predictor(object):
     def preprocess(self, images, transforms):
         preprocessed_samples, batch_trans_info = self._model.preprocess(
             images, transforms, to_tensor=False)
-        if self.model_type == 'classifier':
-            preprocessed_samples = {'image': preprocessed_samples}
-        elif self.model_type == 'segmenter':
-            preprocessed_samples = {'image': preprocessed_samples[0]}
-        elif self.model_type == 'detector':
-            pass
-        elif self.model_type == 'change_detector':
-            preprocessed_samples = {
-                'image': preprocessed_samples[0],
-                'image2': preprocessed_samples[1]
-            }
-        elif self.model_type == 'restorer':
-            preprocessed_samples = {'image': preprocessed_samples[0]}
-        else:
-            logging.error(
-                "Invalid model type {}".format(self.model_type), exit=True)
         return preprocessed_samples, batch_trans_info
 
     def postprocess(self, net_outputs, batch_restore_list, topk=1):

paddlers/tasks/base.py

@@ -20,7 +20,6 @@ import math
 import json
 from functools import partial, wraps
 from inspect import signature
-from typing import Optional
 
 import yaml
 import paddle
@@ -30,7 +29,7 @@ from paddleslim.analysis import flops
 from paddleslim import L1NormFilterPruner, FPGMFilterPruner
 
 import paddlers
-from paddlers.transforms.operators import Compose, DecodeImg, Arrange
+from paddlers.transforms.operators import Compose
 import paddlers.utils.logging as logging
 from paddlers.utils import (
     seconds_to_hms, get_single_card_bs, dict2str, get_pretrain_weights,
@@ -63,7 +62,6 @@ class ModelMeta(type):
 
 
 class BaseModel(metaclass=ModelMeta):
-    _arrange: Optional[Arrange] = None
     find_unused_parameters = False
 
     def __init__(self, model_type):
@@ -97,12 +95,13 @@ class BaseModel(metaclass=ModelMeta):
                        resume_checkpoint=None,
                        is_backbone_weights=False,
                        load_optim_state=True):
+        # FIXME: Multi-process race?
         if pretrain_weights is not None and \
                 not osp.exists(pretrain_weights):
             if not osp.isdir(save_dir):
                 if osp.exists(save_dir):
                     os.remove(save_dir)
-                os.makedirs(save_dir)
+                os.makedirs(save_dir, exist_ok=True)
             # XXX: Hard-coding
             if self.model_type == 'classifier':
                 pretrain_weights = get_pretrain_weights(
@@ -206,11 +205,6 @@ class BaseModel(metaclass=ModelMeta):
                     else:
                         attr = op.__dict__
                     info['Transforms'].append({name: attr})
-                info['Transforms'].append({
-                    self._arrange.__name__: {
-                        'mode': 'test'
-                    }
-                })
         info['completed_epochs'] = self.completed_epochs
         return info
 
@@ -267,28 +261,11 @@ class BaseModel(metaclass=ModelMeta):
         open(osp.join(save_dir, '.success'), 'w').close()
         logging.info("Model saved in {}.".format(save_dir))
 
-    def _build_transforms(self, trans, mode):
-        if isinstance(trans, list):
-            trans = Compose(trans)
-        if not isinstance(trans.transforms[0], DecodeImg):
-            trans.transforms.insert(0, DecodeImg())
-        if self._arrange is Arrange or not issubclass(self._arrange, Arrange):
-            raise ValueError(
-                "`self._arrange` must be set to a concrete Arrange type.")
-        if trans.arrange is None:
-            # For backward compatibility, we only set `trans.arrange`
-            # when it is not set by user.
-            trans.arrange = self._arrange(mode)
-        return trans
-
     def build_data_loader(self,
                           dataset,
                           batch_size,
                           mode='train',
                           collate_fn=None):
-        # NOTE: Append `Arrange` to transforms
-        dataset.transforms = self._build_transforms(dataset.transforms, mode)
-
         if dataset.num_samples < batch_size:
             raise ValueError(
                 'The volume of dataset({}) must be larger than batch size({}).'
@@ -360,7 +337,7 @@ class BaseModel(metaclass=ModelMeta):
             log_writer = LogWriter(vdl_logdir)
 
         # task_id: refer to paddlers
-        task_id = getattr(paddlers, "task_id", "")
+        task_id = getattr(paddlers, 'task_id', '')
 
         thresh = .0001
         if early_stop:
@@ -368,7 +345,6 @@ class BaseModel(metaclass=ModelMeta):
 
         self.train_data_loader = self.build_data_loader(
             train_dataset, batch_size=train_batch_size, mode='train')
-        self._check_arrange(self.train_data_loader.dataset.transforms, 'train')
 
         if eval_dataset is not None:
             self.test_transforms = copy.deepcopy(eval_dataset.transforms)
@@ -543,7 +519,7 @@ class BaseModel(metaclass=ModelMeta):
                 saved. Otherwise, the pruned model will be saved at `save_dir`. 
                 Defaults to None.
         """
-        if self.status == "Pruned":
+        if self.status == 'Pruned':
             raise ValueError(
                 "A pruned model cannot be pruned for a second time!")
         pre_pruning_flops = flops(self.net, self.pruner.inputs)
@@ -636,8 +612,8 @@ class BaseModel(metaclass=ModelMeta):
                 "type": "Sink"
             }
         }]
-        pipeline_info["pipeline_nodes"] = nodes
-        pipeline_info["version"] = "1.0.0"
+        pipeline_info['pipeline_nodes'] = nodes
+        pipeline_info['version'] = '1.0.0'
         return pipeline_info
 
     def _build_inference_net(self):
@@ -705,15 +681,6 @@ class BaseModel(metaclass=ModelMeta):
             raise TypeError(
                 "`transforms` must be `paddlers.transforms.Compose`.")
 
-    def _check_arrange(self, transforms, mode):
-        arrange_obj = transforms.arrange
-        if not isinstance(arrange_obj, Arrange):
-            raise TypeError("`transforms.arrange` must be an Arrange object.")
-        if arrange_obj.mode != mode:
-            raise ValueError(
-                f"Incorrect arrange mode! Expected {repr(mode)} but got {repr(arrange_obj.mode)}."
-            )
-
     def run(self, net, inputs, mode):
         raise NotImplementedError
 

paddlers/tasks/change_detector.py

@@ -29,7 +29,6 @@ import paddlers.rs_models.cd as cmcd
 import paddlers.utils.logging as logging
 from paddlers.models import seg_losses
 from paddlers.transforms import Resize, decode_image, construct_sample
-from paddlers.transforms.operators import ArrangeChangeDetector
 from paddlers.utils import get_single_card_bs
 from paddlers.utils.checkpoint import cd_pretrain_weights_dict
 from .base import BaseModel
@@ -44,8 +43,6 @@ __all__ = [
 
 
 class BaseChangeDetector(BaseModel):
-    _arrange = ArrangeChangeDetector
-
     def __init__(self,
                  model_name,
                  num_classes=2,
@@ -114,7 +111,7 @@ class BaseChangeDetector(BaseModel):
 
     def run(self, net, inputs, mode):
         inputs, batch_restore_list = inputs
-        net_out = net(inputs[0], inputs[1])
+        net_out = net(inputs['image'], inputs['image2'])
         logit = net_out[0]
         outputs = OrderedDict()
         if mode == 'test':
@@ -142,12 +139,12 @@ class BaseChangeDetector(BaseModel):
                 pred = paddle.unsqueeze(net_out[0], axis=1)  # NCHW
             else:
                 pred = paddle.argmax(logit, axis=1, keepdim=True, dtype='int32')
-            label = inputs[2]
+            label = inputs['mask']
             if label.ndim == 3:
                 paddle.unsqueeze_(label, axis=1)
             if label.ndim != 4:
-                raise ValueError("Expected label.ndim == 4 but got {}".format(
-                    label.ndim))
+                raise ValueError(
+                    "Expected `label.ndim` == 4 but got {}.".format(label.ndim))
             pred = self.postprocess(pred, batch_restore_list)[0]  # NCHW
             intersect_area, pred_area, label_area = ppseg.utils.metrics.calculate_area(
                 pred, label, self.num_classes)
@@ -160,12 +157,10 @@ class BaseChangeDetector(BaseModel):
             if hasattr(net, 'USE_MULTITASK_DECODER') and \
                 net.USE_MULTITASK_DECODER is True:
                 # CD+Seg
-                if len(inputs) != 5:
-                    raise ValueError(
-                        "Cannot perform loss computation with {} inputs.".
-                        format(len(inputs)))
+                if 'aux_masks' not in inputs:
+                    raise ValueError("Auxiliary masks not found.")
                 labels_list = [
-                    inputs[2 + idx]
+                    inputs['aux_masks'][idx]
                     for idx in map(attrgetter('value'), net.OUT_TYPES)
                 ]
                 loss_list = metrics.multitask_loss_computation(
@@ -174,7 +169,9 @@ class BaseChangeDetector(BaseModel):
                     losses=self.losses)
             else:
                 loss_list = metrics.loss_computation(
-                    logits_list=net_out, labels=inputs[2], losses=self.losses)
+                    logits_list=net_out,
+                    labels=inputs['mask'],
+                    losses=self.losses)
             loss = sum(loss_list)
             outputs['loss'] = loss
         return outputs
@@ -451,7 +448,6 @@ class BaseChangeDetector(BaseModel):
             )
         self.eval_data_loader = self.build_data_loader(
             eval_dataset, batch_size=batch_size, mode='eval')
-        self._check_arrange(self.eval_data_loader.dataset.transforms, 'eval')
 
         intersect_area_all = 0
         pred_area_all = 0
@@ -535,7 +531,7 @@ class BaseChangeDetector(BaseModel):
             img_file (list[tuple] | tuple[str|np.ndarray]): Tuple of image paths or 
                 decoded image data for bi-temporal images, which also could constitute
                 a list, meaning all image pairs to be predicted as a mini-batch.
-            transforms (paddlers.transforms.Compose|list|None, optional): Transforms for 
+            transforms (paddlers.transforms.Compose|None, optional): Transforms for 
                 inputs. If None, the transforms for evaluation process will be used. 
                 Defaults to None.
 
@@ -561,8 +557,6 @@ class BaseChangeDetector(BaseModel):
             images = [img_file]
         else:
             images = img_file
-        transforms = self._build_transforms(transforms, "test")
-        self._check_arrange(transforms, "test")
         data = self.preprocess(images, transforms, self.model_type)
         self.net.eval()
         outputs = self.run(self.net, data, 'test')
@@ -634,7 +628,7 @@ class BaseChangeDetector(BaseModel):
             # XXX: sample do not contain 'image_t1' and 'image_t2'.
             sample = construct_sample(image=im1, image2=im2)
             data = transforms(sample)
-            im1, im2 = data[0][:2]
+            im1, im2 = data[0]['image'], data[0]['image2']
             trans_info = data[1]
             batch_im1.append(im1)
             batch_im2.append(im2)
@@ -646,7 +640,7 @@ class BaseChangeDetector(BaseModel):
             batch_im1 = np.asarray(batch_im1)
             batch_im2 = np.asarray(batch_im2)
 
-        return (batch_im1, batch_im2), batch_trans_info
+        return {'image': batch_im1, 'image2': batch_im2}, batch_trans_info
 
     def postprocess(self, batch_pred, batch_restore_list):
         if isinstance(batch_pred, (tuple, list)) and self.status == 'Infer':
@@ -719,13 +713,6 @@ class BaseChangeDetector(BaseModel):
             score_maps.append(score_map.squeeze())
         return label_maps, score_maps
 
-    def _check_arrange(self, transforms, mode):
-        super()._check_arrange(transforms, mode)
-        if not isinstance(transforms.arrange, ArrangeChangeDetector):
-            raise TypeError(
-                "`transforms.arrange` must be an `ArrangeChangeDetector` object."
-            )
-
     def set_losses(self, losses, weights=None):
         if weights is None:
             weights = [1. for _ in range(len(losses))]

paddlers/tasks/classifier.py

@@ -29,7 +29,6 @@ from paddlers.models import clas_losses
 from paddlers.models.ppcls.data.postprocess import build_postprocess
 from paddlers.utils.checkpoint import cls_pretrain_weights_dict
 from paddlers.transforms import Resize, decode_image, construct_sample
-from paddlers.transforms.operators import ArrangeClassifier
 
 from .base import BaseModel
 
@@ -37,8 +36,6 @@ __all__ = ["ResNet50_vd", "MobileNetV3", "HRNet", "CondenseNetV2"]
 
 
 class BaseClassifier(BaseModel):
-    _arrange = ArrangeClassifier
-
     def __init__(self,
                  model_name,
                  in_channels=3,
@@ -122,13 +119,13 @@ class BaseClassifier(BaseModel):
         return input_spec
 
     def run(self, net, inputs, mode):
-        net_out = net(inputs[0])
+        net_out = net(inputs['image'])
 
         if mode == 'test':
             return self.postprocess(net_out)
 
         outputs = OrderedDict()
-        label = paddle.to_tensor(inputs[1], dtype="int64")
+        label = paddle.to_tensor(inputs['label'], dtype="int64")
 
         if mode == 'eval':
             label = paddle.unsqueeze(label, axis=-1)
@@ -404,8 +401,6 @@ class BaseClassifier(BaseModel):
         if nranks < 2 or local_rank == 0:
             self.eval_data_loader = self.build_data_loader(
                 eval_dataset, batch_size=batch_size, mode='eval')
-            self._check_arrange(self.eval_data_loader.dataset.transforms,
-                                'eval')
             logging.info(
                 "Start to evaluate(total_samples={}, total_steps={})...".format(
                     eval_dataset.num_samples, eval_dataset.num_samples))
@@ -414,7 +409,6 @@ class BaseClassifier(BaseModel):
             top5s = []
             with paddle.no_grad():
                 for step, data in enumerate(self.eval_data_loader):
-                    data.append(eval_dataset.transforms.transforms)
                     outputs = self.run(self.net, data, 'eval')
                     top1s.append(outputs["top1"])
                     top5s.append(outputs["top5"])
@@ -461,8 +455,6 @@ class BaseClassifier(BaseModel):
             images = [img_file]
         else:
             images = img_file
-        transforms = self._build_transforms(transforms, "test")
-        self._check_arrange(transforms, "test")
         data, _ = self.preprocess(images, transforms, self.model_type)
         self.net.eval()
 
@@ -495,20 +487,14 @@ class BaseClassifier(BaseModel):
                 im = decode_image(im, read_raw=True)
             sample = construct_sample(image=im)
             data = transforms(sample)
-            im = data[0][0]
+            im = data[0]['image']
             batch_im.append(im)
         if to_tensor:
             batch_im = paddle.to_tensor(batch_im)
         else:
             batch_im = np.asarray(batch_im)
 
-        return batch_im, None
-
-    def _check_arrange(self, transforms, mode):
-        super()._check_arrange(transforms, mode)
-        if not isinstance(transforms.arrange, ArrangeClassifier):
-            raise TypeError(
-                "`transforms.arrange` must be an `ArrangeClassifier` object.")
+        return {'image': batch_im}, None
 
     def build_data_loader(self,
                           dataset,

paddlers/tasks/object_detector.py

@@ -26,7 +26,6 @@ import paddlers.models.ppdet as ppdet
 from paddlers.models.ppdet.modeling.proposal_generator.target_layer import BBoxAssigner, MaskAssigner
 from paddlers.transforms import decode_image, construct_sample
 from paddlers.transforms.operators import _NormalizeBox, _PadBox, _BboxXYXY2XYWH, Resize, Pad
-from paddlers.transforms.operators import ArrangeDetector
 from paddlers.transforms.batch_operators import BatchCompose, BatchRandomResize, BatchRandomResizeByShort, \
     _BatchPad, _Gt2YoloTarget
 from paddlers.models.ppdet.optimizer import ModelEMA
@@ -43,8 +42,6 @@ __all__ = [
 
 
 class BaseDetector(BaseModel):
-    _arrange = ArrangeDetector
-
     def __init__(self, model_name, num_classes=80, **params):
         self.init_params.update(locals())
         if 'with_net' in self.init_params:
@@ -109,6 +106,8 @@ class BaseDetector(BaseModel):
         return backbone
 
     def run(self, net, inputs, mode):
+        if mode == 'eval' and 'gt_poly' in inputs:
+            del inputs['gt_poly']
         net_out = net(inputs)
         if mode in ['train', 'eval']:
             outputs = net_out
@@ -574,7 +573,6 @@ class BaseDetector(BaseModel):
                     classwise=False)
             scores = collections.OrderedDict()
 
-            self._check_arrange(eval_dataset.transforms, 'eval')
             logging.info(
                 "Start to evaluate(total_samples={}, total_steps={})...".format(
                     eval_dataset.num_samples, eval_dataset.num_samples))
@@ -627,9 +625,6 @@ class BaseDetector(BaseModel):
             images = [img_file]
         else:
             images = img_file
-
-        transforms = self._build_transforms(transforms, "test")
-        self._check_arrange(transforms, "test")
         batch_samples, _ = self.preprocess(images, transforms)
         self.net.eval()
         outputs = self.run(self.net, batch_samples, 'test')
@@ -646,9 +641,8 @@ class BaseDetector(BaseModel):
             if isinstance(im, str):
                 im = decode_image(im, read_raw=True)
             sample = construct_sample(image=im)
-            sample = transforms(sample)
-            data = sample[0]
-            batch_samples.append(data)
+            data = transforms(sample)
+            batch_samples.append(data[0])
         batch_transforms = self._compose_batch_transform(transforms, 'test')
         batch_samples = batch_transforms(batch_samples)
         if to_tensor:
@@ -725,12 +719,6 @@ class BaseDetector(BaseModel):
 
         return results
 
-    def _check_arrange(self, transforms, mode):
-        super()._check_arrange(transforms, mode)
-        if not isinstance(transforms.arrange, ArrangeDetector):
-            raise TypeError(
-                "`transforms.arrange` must be an `ArrangeDetector` object.")
-
     def get_pruning_info(self):
         info = super().get_pruning_info()
         info['pruner_inputs'] = {

paddlers/tasks/restorer.py

@@ -29,7 +29,6 @@ import paddlers.utils.logging as logging
 from paddlers.models import res_losses
 from paddlers.models.ppgan.modules.init import init_weights
 from paddlers.transforms import Resize, decode_image, construct_sample
-from paddlers.transforms.operators import ArrangeRestorer
 from paddlers.transforms.functions import calc_hr_shape
 from paddlers.utils.checkpoint import res_pretrain_weights_dict
 from .base import BaseModel
@@ -40,7 +39,6 @@ __all__ = ["DRN", "LESRCNN", "ESRGAN"]
 
 
 class BaseRestorer(BaseModel):
-    _arrange = ArrangeRestorer
     MIN_MAX = (0., 1.)
     TEST_OUT_KEY = None
 
@@ -122,13 +120,13 @@ class BaseRestorer(BaseModel):
 
         if mode == 'test':
             if self.status == 'Infer':
-                net_out = net(inputs[0])
+                net_out = net(inputs['image'])
                 res_map_list = self.postprocess(net_out, batch_restore_list)
             else:
                 if isinstance(net, GANAdapter):
-                    net_out = net.generator(inputs[0])
+                    net_out = net.generator(inputs['image'])
                 else:
-                    net_out = net(inputs[0])
+                    net_out = net(inputs['image'])
                 if self.TEST_OUT_KEY is not None:
                     net_out = net_out[self.TEST_OUT_KEY]
                 pred = self.postprocess(net_out, batch_restore_list)
@@ -140,12 +138,12 @@ class BaseRestorer(BaseModel):
 
         if mode == 'eval':
             if isinstance(net, GANAdapter):
-                net_out = net.generator(inputs[0])
+                net_out = net.generator(inputs['image'])
             else:
-                net_out = net(inputs[0])
+                net_out = net(inputs['image'])
             if self.TEST_OUT_KEY is not None:
                 net_out = net_out[self.TEST_OUT_KEY]
-            tar = inputs[1]
+            tar = inputs['target']
             pred = self.postprocess(net_out, batch_restore_list)[0]  # NCHW
             pred = self._tensor_to_images(pred)
             outputs['pred'] = pred
@@ -155,8 +153,8 @@ class BaseRestorer(BaseModel):
         if mode == 'train':
             # This is used by non-GAN models.
             # For GAN models, self.run_gan() should be used.
-            net_out = net(inputs[0])
-            loss = self.losses(net_out, inputs[1])
+            net_out = net(inputs['image'])
+            loss = self.losses(net_out, inputs['target'])
             outputs['loss'] = loss
         return outputs
 
@@ -413,7 +411,6 @@ class BaseRestorer(BaseModel):
         if nranks < 2 or local_rank == 0:
             self.eval_data_loader = self.build_data_loader(
                 eval_dataset, batch_size=batch_size, mode='eval')
-            self._check_arrange(eval_dataset.transforms, 'eval')
             # XXX: Hard-code crop_border and test_y_channel
             psnr = metrics.PSNR(crop_border=4, test_y_channel=True)
             ssim = metrics.SSIM(crop_border=4, test_y_channel=True)
@@ -469,8 +466,6 @@ class BaseRestorer(BaseModel):
             images = [img_file]
         else:
             images = img_file
-        transforms = self._build_transforms(transforms, "test")
-        self._check_arrange(transforms, "test")
         data = self.preprocess(images, transforms, self.model_type)
         self.net.eval()
         outputs = self.run(self.net, data, 'test')
@@ -490,7 +485,7 @@ class BaseRestorer(BaseModel):
                 im = decode_image(im, read_raw=True)
             sample = construct_sample(image=im)
             data = transforms(sample)
-            im = data[0][0]
+            im = data[0]['image']
             trans_info = data[1]
             batch_im.append(im)
             batch_trans_info.append(trans_info)
@@ -499,7 +494,7 @@ class BaseRestorer(BaseModel):
         else:
             batch_im = np.asarray(batch_im)
 
-        return (batch_im, ), batch_trans_info
+        return {'image': batch_im}, batch_trans_info
 
     def postprocess(self, batch_pred, batch_restore_list):
         if self.status == 'Infer':
@@ -564,12 +559,6 @@ class BaseRestorer(BaseModel):
             res_maps.append(res_map.squeeze())
         return res_maps
 
-    def _check_arrange(self, transforms, mode):
-        super()._check_arrange(transforms, mode)
-        if not isinstance(transforms.arrange, ArrangeRestorer):
-            raise TypeError(
-                "`transforms.arrange` must be an `ArrangeRestorer` object.")
-
     def build_data_loader(self,
                           dataset,
                           batch_size,
@@ -712,7 +701,9 @@ class DRN(BaseRestorer):
 
     def train_step(self, step, data, net):
         outputs = self.run_gan(
-            net, data[0], mode='train', gan_mode='forward_primary')
+            net, (data[0]['image'], data[0]['target']),
+            mode='train',
+            gan_mode='forward_primary')
         outputs.update(
             self.run_gan(
                 net, (outputs['sr'], outputs['lr']),
@@ -872,14 +863,16 @@ class ESRGAN(BaseRestorer):
             optim_g, optim_d = self.optimizer
 
             outputs = self.run_gan(
-                net, data[0], mode='train', gan_mode='forward_g')
+                net, (data[0]['image'], data[0]['target']),
+                mode='train',
+                gan_mode='forward_g')
             optim_g.clear_grad()
             (outputs['loss_g_pps'] + outputs['loss_g_gan']).backward()
             optim_g.step()
 
             outputs.update(
                 self.run_gan(
-                    net, (outputs['g_pred'], data[0][1]),
+                    net, (outputs['g_pred'], data[0]['target']),
                     mode='train',
                     gan_mode='forward_d'))
             optim_d.clear_grad()

paddlers/tasks/segmenter.py

@@ -28,7 +28,6 @@ import paddlers.rs_models.seg as cmseg
 import paddlers.utils.logging as logging
 from paddlers.models import seg_losses
 from paddlers.transforms import Resize, decode_image, construct_sample
-from paddlers.transforms.operators import ArrangeSegmenter
 from paddlers.utils import get_single_card_bs, DisablePrint
 from paddlers.utils.checkpoint import seg_pretrain_weights_dict
 from .base import BaseModel
@@ -43,8 +42,6 @@ __all__ = [
 
 
 class BaseSegmenter(BaseModel):
-    _arrange = ArrangeSegmenter
-
     def __init__(self,
                  model_name,
                  num_classes=2,
@@ -117,7 +114,7 @@ class BaseSegmenter(BaseModel):
 
     def run(self, net, inputs, mode):
         inputs, batch_restore_list = inputs
-        net_out = net(inputs[0])
+        net_out = net(inputs['image'])
         logit = net_out[0]
         outputs = OrderedDict()
         if mode == 'test':
@@ -145,7 +142,7 @@ class BaseSegmenter(BaseModel):
                 pred = paddle.unsqueeze(net_out[0], axis=1)  # NCHW
             else:
                 pred = paddle.argmax(logit, axis=1, keepdim=True, dtype='int32')
-            label = inputs[1]
+            label = inputs['mask']
             if label.ndim == 3:
                 paddle.unsqueeze_(label, axis=1)
             if label.ndim != 4:
@@ -161,7 +158,7 @@ class BaseSegmenter(BaseModel):
                                                            self.num_classes)
         if mode == 'train':
             loss_list = metrics.loss_computation(
-                logits_list=net_out, labels=inputs[1], losses=self.losses)
+                logits_list=net_out, labels=inputs['mask'], losses=self.losses)
             loss = sum(loss_list)
             outputs['loss'] = loss
         return outputs
@@ -429,7 +426,6 @@ class BaseSegmenter(BaseModel):
                 "is forcibly set to {}.".format(batch_size))
         self.eval_data_loader = self.build_data_loader(
             eval_dataset, batch_size=batch_size, mode='eval')
-        self._check_arrange(eval_dataset.transforms, 'eval')
 
         intersect_area_all = 0
         pred_area_all = 0
@@ -528,8 +524,6 @@ class BaseSegmenter(BaseModel):
             images = [img_file]
         else:
             images = img_file
-        transforms = self._build_transforms(transforms, "test")
-        self._check_arrange(transforms, "test")
         data = self.preprocess(images, transforms, self.model_type)
         self.net.eval()
         outputs = self.run(self.net, data, 'test')
@@ -599,7 +593,7 @@ class BaseSegmenter(BaseModel):
                 im = decode_image(im, read_raw=True)
             sample = construct_sample(image=im)
             data = transforms(sample)
-            im = data[0][0]
+            im = data[0]['image']
             trans_info = data[1]
             batch_im.append(im)
             batch_trans_info.append(trans_info)
@@ -608,7 +602,7 @@ class BaseSegmenter(BaseModel):
         else:
             batch_im = np.asarray(batch_im)
 
-        return (batch_im, ), batch_trans_info
+        return {'image': batch_im}, batch_trans_info
 
     def postprocess(self, batch_pred, batch_restore_list):
         if isinstance(batch_pred, (tuple, list)) and self.status == 'Infer':
@@ -681,12 +675,6 @@ class BaseSegmenter(BaseModel):
             score_maps.append(score_map.squeeze())
         return label_maps, score_maps
 
-    def _check_arrange(self, transforms, mode):
-        super()._check_arrange(transforms, mode)
-        if not isinstance(transforms.arrange, ArrangeSegmenter):
-            raise TypeError(
-                "`transforms.arrange` must be an `ArrangeSegmenter` object.")
-
     def set_losses(self, losses, weights=None):
         if weights is None:
             weights = [1. for _ in range(len(losses))]
@@ -924,15 +912,14 @@ class C2FNet(BaseSegmenter):
     def run(self, net, inputs, mode):
         inputs, batch_restore_list = inputs
         with paddle.no_grad():
-            pre_coarse = self.coarse_model(inputs[0])
+            pre_coarse = self.coarse_model(inputs['image'])
             pre_coarse = pre_coarse[0]
             heatmaps = pre_coarse
 
         if mode == 'test':
-            net_out = net(inputs[0], heatmaps)
+            net_out = net(inputs['image'], heatmaps)
             logit = net_out[0]
             outputs = OrderedDict()
-            origin_shape = inputs[1]
             if self.status == 'Infer':
                 label_map_list, score_map_list = self.postprocess(
                     net_out, batch_restore_list)
@@ -953,19 +940,19 @@ class C2FNet(BaseSegmenter):
             outputs['score_map'] = score_map_list
 
         if mode == 'eval':
-            net_out = net(inputs[0], heatmaps)
+            net_out = net(inputs['image'], heatmaps)
             logit = net_out[0]
             outputs = OrderedDict()
             if self.status == 'Infer':
                 pred = paddle.unsqueeze(net_out[0], axis=1)  # NCHW
             else:
                 pred = paddle.argmax(logit, axis=1, keepdim=True, dtype='int32')
-            label = inputs[1]
+            label = inputs['mask']
             if label.ndim == 3:
                 paddle.unsqueeze_(label, axis=1)
             if label.ndim != 4:
-                raise ValueError("Expected label.ndim == 4 but got {}".format(
-                    label.ndim))
+                raise ValueError(
+                    "Expected `label.ndim` == 4 but got {}.".format(label.ndim))
             pred = self.postprocess(pred, batch_restore_list)[0]  # NCHW
             intersect_area, pred_area, label_area = ppseg.utils.metrics.calculate_area(
                 pred, label, self.num_classes)
@@ -975,7 +962,7 @@ class C2FNet(BaseSegmenter):
             outputs['conf_mat'] = metrics.confusion_matrix(pred, label,
                                                            self.num_classes)
         if mode == 'train':
-            net_out = net(inputs[0], heatmaps, inputs[1])
+            net_out = net(inputs['image'], heatmaps, inputs['mask'])
             logit = [net_out[0], ]
             labels = net_out[1]
             outputs = OrderedDict()

paddlers/transforms/batch_operators.py

@@ -14,10 +14,6 @@
 
 import traceback
 import random
-try:
-    from collections.abc import Sequence
-except Exception:
-    from collections import Sequence
 
 import numpy as np
 from paddle.fluid.dataloader.collate import default_collate_fn
@@ -27,6 +23,15 @@ from .box_utils import jaccard_overlap
 from paddlers.utils import logging
 
 
+class _IPermute(Transform):
+    def __init__(self):
+        super(_IPermute, self).__init__()
+
+    def apply_im(self, image):
+        image = np.moveaxis(image, 0, 2)
+        return image
+
+
 class BatchTransform(Transform):
     is_batch_transform = True
 
@@ -38,6 +43,17 @@ class BatchCompose(BatchTransform):
         self.collate_batch = collate_batch
 
     def __call__(self, samples):
+        iperm_op = _IPermute()
+        perm_status_list = []
+        for i, sample in enumerate(samples):
+            permuted = sample.pop('permuted', False)
+            if permuted:
+                # If a sample is permuted, we apply the inverse-permute
+                # operator, such that it is possible to reuse non-batched data 
+                # transformation operators later.
+                samples[i] = iperm_op(sample)
+            perm_status_list.append(permuted)
+
         if self.batch_transforms is not None:
             for op in self.batch_transforms:
                 try:
@@ -49,7 +65,11 @@ class BatchCompose(BatchTransform):
                                         op, e, str(stack_info)))
                     raise e
 
-        samples = _Permute()(samples)
+        # Recover permutation status
+        perm_op = _Permute()
+        for i, permuted in enumerate(perm_status_list):
+            if permuted:
+                samples[i] = perm_op(samples[i])
 
         extra_key = ['h', 'w', 'flipped', 'trans_info']
         for k in extra_key:

paddlers/transforms/functions.py

@@ -37,11 +37,9 @@ def normalize(im, mean, std, min_value=[0, 0, 0], max_value=[255, 255, 255]):
     return im
 
 
-def permute(im, to_bgr=False):
+def permute(im):
     im = np.swapaxes(im, 1, 2)
     im = np.swapaxes(im, 1, 0)
-    if to_bgr:
-        im = im[[2, 1, 0], :, :]
     return im
 
 

paddlers/transforms/operators.py

@@ -27,10 +27,10 @@ import imghdr
 from PIL import Image
 from joblib import load
 
-import paddlers
 import paddlers.transforms.functions as F
 import paddlers.transforms.indices as indices
 import paddlers.transforms.satellites as satellites
+from paddlers.utils import logging
 
 __all__ = [
     "construct_sample",
@@ -92,43 +92,67 @@ def construct_sample_from_dict(dict_like_obj):
 
 class Compose(object):
     """
-    Apply a series of data augmentation strategies to the input.
+    Apply a series of data transformation operators to the input.
     All input images should be in Height-Width-Channel ([H, W, C]) format.
 
     Args:
-        transforms (list[paddlers.transforms.Transform]): List of data preprocess or
-            augmentation operators.
+        transforms (list[paddlers.transforms.Transform]): List of data preprocessing 
+            or augmentation operators.
+        auto_decode (bool, optional): Whether to automatically decode image(s) before
+            any data transformation operator is applied. If an image decoding operator 
+            is detected in `transforms`, no auto-decoding operation will be done. 
+            Defaults to True.
+        auto_permute(bool, optional): Whether to automatically permute the processed
+            image(s). Defaults to True.
 
     Raises:
-        TypeError: Invalid type of transforms.
-        ValueError: Invalid length of transforms.
+        TypeError: `transforms` has an unsupported type.
+        ValueError: `transforms` has an illegal length.
     """
 
-    def __init__(self, transforms):
+    def __init__(self, transforms, auto_decode=True, auto_permute=True):
         super(Compose, self).__init__()
         if not isinstance(transforms, list):
             raise TypeError(
-                "Type of transforms is invalid. Must be a list, but received is {}."
+                "`transforms` has an unsupported type. Must be a list, but received is {}."
                 .format(type(transforms)))
         if len(transforms) < 1:
             raise ValueError(
-                "Length of transforms must not be less than 1, but received is {}."
+                "Length of `transforms` must not be less than 1, but received is {}."
                 .format(len(transforms)))
-        transforms = copy.deepcopy(transforms)
-        # We will have to do a late binding of `self.arrange`
-        self.arrange = None
         self.transforms = transforms
+        # Deprecation warning
+        for op in self.transforms:
+            if isinstance(op, Arrange):
+                logging.warning(
+                    "Including an `Arrange` object in the transformation operator list is deprecated and will not take effect."
+                )
+                # We only trigger this warning once for each `Compose` object
+                break
+
+        self.auto_decode = auto_decode
+        self.auto_permute = auto_permute
 
     def __call__(self, sample):
-        """
-        This is equivalent to sequentially calling compose_obj.apply_transforms()
-            and compose_obj.arrange_outputs().
-        """
         if 'trans_info' not in sample:
             sample['trans_info'] = []
+
+        if self.auto_decode:
+            for op in self.transforms:
+                if isinstance(op, DecodeImg):
+                    break
+            else:
+                # Initialize the decoder with default parameters
+                decoder = DecodeImg()
+                sample = decoder(sample)
+
         sample = self.apply_transforms(sample)
+
+        if self.auto_permute:
+            permute_op = _Permute()
+            sample = permute_op(sample)
+
         trans_info = sample['trans_info']
-        sample = self.arrange_outputs(sample)
         return sample, trans_info
 
     def apply_transforms(self, sample):
@@ -136,14 +160,14 @@ class Compose(object):
             # Skip batch transforms
             if getattr(op, 'is_batch_transform', False):
                 continue
+            # For backward compatibility, we bypass `Arrange` objects
+            # here and leave the arrangement operation to the 
+            # training/evaluation APIs.
+            if isinstance(op, Arrange):
+                continue
             sample = op(sample)
         return sample
 
-    def arrange_outputs(self, sample):
-        if self.arrange is not None:
-            sample = self.arrange(sample)
-        return sample
-
 
 class Transform(object):
     """
@@ -1928,11 +1952,12 @@ class _Permute(Transform):
         super(_Permute, self).__init__()
 
     def apply(self, sample):
-        sample['image'] = F.permute(sample['image'], False)
+        sample['image'] = F.permute(sample['image'])
         if 'image2' in sample:
-            sample['image2'] = F.permute(sample['image2'], False)
+            sample['image2'] = F.permute(sample['image2'])
         if 'target' in sample:
-            sample['target'] = F.permute(sample['target'], False)
+            sample['target'] = F.permute(sample['target'])
+        sample['permuted'] = True
         return sample
 
 

test_tipc/configs/cd/_base_/airchange.yaml

@@ -3,59 +3,53 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: CDDataset
-        args: 
-            data_dir: ./test_tipc/data/airchange/
-            file_list: ./test_tipc/data/airchange/train.txt
-            label_list: null
-            num_workers: 0
-            shuffle: True
-            with_seg_labels: False
-            binarize_labels: True
-    eval: !Node
-        type: CDDataset
-        args:
-            data_dir: ./test_tipc/data/airchange/
-            file_list: ./test_tipc/data/airchange/eval.txt
-            label_list: null
-            num_workers: 0
-            shuffle: False
-            with_seg_labels: False
-            binarize_labels: True
+  train: !Node
+    type: CDDataset
+    args:
+      data_dir: ./test_tipc/data/airchange/
+      file_list: ./test_tipc/data/airchange/train.txt
+      label_list: null
+      num_workers: 0
+      shuffle: True
+      with_seg_labels: False
+      binarize_labels: True
+  eval: !Node
+    type: CDDataset
+    args:
+      data_dir: ./test_tipc/data/airchange/
+      file_list: ./test_tipc/data/airchange/eval.txt
+      label_list: null
+      num_workers: 0
+      shuffle: False
+      with_seg_labels: False
+      binarize_labels: True
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomCrop
-          args:
-            crop_size: 256
-            aspect_ratio: [0.5, 2.0]
-            scaling: [0.2, 1.0]
-        - !Node
-          type: RandomHorizontalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomCrop
+      args:
+        crop_size: 256
+        aspect_ratio: [0.5, 2.0]
+        scaling: [0.2, 1.0]
+    - !Node
+      type: RandomHorizontalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
 download_on: False
 download_url: https://paddlers.bj.bcebos.com/datasets/airchange.zip
 download_path: ./test_tipc/data/

test_tipc/configs/cd/_base_/levircd.yaml

@@ -3,53 +3,47 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: CDDataset
-        args: 
-            data_dir: ./test_tipc/data/levircd/
-            file_list: ./test_tipc/data/levircd/train.txt
-            label_list: null
-            num_workers: 0
-            shuffle: True
-            with_seg_labels: False
-            binarize_labels: True
-    eval: !Node
-        type: CDDataset
-        args:
-            data_dir: ./test_tipc/data/levircd/
-            file_list: ./test_tipc/data/levircd/val.txt
-            label_list: null
-            num_workers: 0
-            shuffle: False
-            with_seg_labels: False
-            binarize_labels: True
+  train: !Node
+    type: CDDataset
+    args:
+      data_dir: ./test_tipc/data/levircd/
+      file_list: ./test_tipc/data/levircd/train.txt
+      label_list: null
+      num_workers: 0
+      shuffle: True
+      with_seg_labels: False
+      binarize_labels: True
+  eval: !Node
+    type: CDDataset
+    args:
+      data_dir: ./test_tipc/data/levircd/
+      file_list: ./test_tipc/data/levircd/val.txt
+      label_list: null
+      num_workers: 0
+      shuffle: False
+      with_seg_labels: False
+      binarize_labels: True
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomHorizontalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeChangeDetector
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomHorizontalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
 download_on: False
 
 num_epochs: 10

test_tipc/configs/cd/bit/bit_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/bit/
 
 model: !Node
-    type: BIT
+  type: BIT

test_tipc/configs/cd/bit/bit_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/bit/
 
 model: !Node
-    type: BIT
+  type: BIT

test_tipc/configs/cd/cdnet/cdnet_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/cdnet/
 
 model: !Node
-    type: CDNet
+  type: CDNet

test_tipc/configs/cd/cdnet/cdnet_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/cdnet/
 
 model: !Node
-    type: CDNet
+  type: CDNet

test_tipc/configs/cd/changeformer/changeformer_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/changeformer/
 
 model: !Node
-    type: ChangeFormer
+  type: ChangeFormer

test_tipc/configs/cd/changeformer/changeformer_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/changeformer/
 
 model: !Node
-    type: ChangeFormer
+  type: ChangeFormer

test_tipc/configs/cd/dsamnet/dsamnet_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/dsamnet/
 
 model: !Node
-    type: DSAMNet
+  type: DSAMNet

test_tipc/configs/cd/dsamnet/dsamnet_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/dsamnet/
 
 model: !Node
-    type: DSAMNet
+  type: DSAMNet

test_tipc/configs/cd/dsifn/dsifn_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/dsifn/
 
 model: !Node
-    type: DSIFN
+  type: DSIFN

test_tipc/configs/cd/dsifn/dsifn_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/dsifn/
 
 model: !Node
-    type: DSIFN
+  type: DSIFN

test_tipc/configs/cd/fc_ef/fc_ef_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/fc_ef/
 
 model: !Node
-    type: FCEarlyFusion
+  type: FCEarlyFusion

test_tipc/configs/cd/fc_ef/fc_ef_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/fc_ef/
 
 model: !Node
-    type: FCEarlyFusion
+  type: FCEarlyFusion

test_tipc/configs/cd/fc_siam_conc/fc_siam_conc_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/fc_siam_conc/
 
 model: !Node
-    type: FCSiamConc
+  type: FCSiamConc

test_tipc/configs/cd/fc_siam_conc/fc_siam_conc_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/fc_siam_conc/
 
 model: !Node
-    type: FCSiamConc
+  type: FCSiamConc

test_tipc/configs/cd/fc_siam_diff/fc_siam_diff_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/fc_siam_diff/
 
 model: !Node
-    type: FCSiamDiff
+  type: FCSiamDiff

test_tipc/configs/cd/fc_siam_diff/fc_siam_diff_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/fc_siam_diff/
 
 model: !Node
-    type: FCSiamDiff
+  type: FCSiamDiff

test_tipc/configs/cd/fccdn/fccdn_airchange.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/fccdn/
 
 model: !Node
-    type: FCCDN
+  type: FCCDN
 
 learning_rate: 0.07
 lr_decay_power: 0.6

test_tipc/configs/cd/fccdn/fccdn_levircd.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/fccdn/
 
 model: !Node
-    type: FCCDN
+  type: FCCDN
 
 learning_rate: 0.001

test_tipc/configs/cd/p2v/p2v_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/p2v/
 
 model: !Node
-    type: P2V
+  type: P2V

test_tipc/configs/cd/p2v/p2v_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/p2v/
 
 model: !Node
-    type: P2V
+  type: P2V

test_tipc/configs/cd/snunet/snunet_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/snunet/
 
 model: !Node
-    type: SNUNet
+  type: SNUNet

test_tipc/configs/cd/snunet/snunet_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/snunet/
 
 model: !Node
-    type: SNUNet
+  type: SNUNet

test_tipc/configs/cd/stanet/stanet_airchange.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/airchange.yaml
 save_dir: ./test_tipc/output/cd/stanet/
 
 model: !Node
-    type: STANet
+  type: STANet

test_tipc/configs/cd/stanet/stanet_levircd.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/levircd.yaml
 save_dir: ./test_tipc/output/cd/stanet/
 
 model: !Node
-    type: STANet
+  type: STANet

test_tipc/configs/clas/_base_/ucmerced.yaml

@@ -3,61 +3,55 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: ClasDataset
-        args: 
-            data_dir: ./test_tipc/data/ucmerced/
-            file_list: ./test_tipc/data/ucmerced/train.txt
-            label_list: ./test_tipc/data/ucmerced/labels.txt
-            num_workers: 0
-            shuffle: True
-    eval: !Node
-        type: ClasDataset
-        args:
-            data_dir: ./test_tipc/data/ucmerced/
-            file_list: ./test_tipc/data/ucmerced/val.txt
-            label_list: ./test_tipc/data/ucmerced/labels.txt
-            num_workers: 0
-            shuffle: False
+  train: !Node
+    type: ClasDataset
+    args:
+      data_dir: ./test_tipc/data/ucmerced/
+      file_list: ./test_tipc/data/ucmerced/train.txt
+      label_list: ./test_tipc/data/ucmerced/labels.txt
+      num_workers: 0
+      shuffle: True
+  eval: !Node
+    type: ClasDataset
+    args:
+      data_dir: ./test_tipc/data/ucmerced/
+      file_list: ./test_tipc/data/ucmerced/val.txt
+      label_list: ./test_tipc/data/ucmerced/labels.txt
+      num_workers: 0
+      shuffle: False
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 256
-        - !Node
-          type: RandomHorizontalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: RandomVerticalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeClassifier
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 256
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeClassifier
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 256
+    - !Node
+      type: RandomHorizontalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: RandomVerticalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 256
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
 download_on: False
 download_url: https://paddlers.bj.bcebos.com/datasets/ucmerced.zip
 download_path: ./test_tipc/data/

test_tipc/configs/clas/condensenetv2/condensenetv2_ucmerced.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/ucmerced.yaml
 save_dir: ./test_tipc/output/clas/condensenetv2/
 
 model: !Node
-    type: CondenseNetV2
-    args:
-        num_classes: 21
+  type: CondenseNetV2
+  args:
+    num_classes: 21

test_tipc/configs/clas/hrnet/hrnet_ucmerced.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/ucmerced.yaml
 save_dir: ./test_tipc/output/clas/hrnet/
 
 model: !Node
-    type: HRNet
-    args:
-        num_classes: 21
+  type: HRNet
+  args:
+    num_classes: 21

test_tipc/configs/clas/mobilenetv3/mobilenetv3_ucmerced.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/ucmerced.yaml
 save_dir: ./test_tipc/output/clas/mobilenetv3/
 
 model: !Node
-    type: MobileNetV3
-    args:
-        num_classes: 21
+  type: MobileNetV3
+  args:
+    num_classes: 21

test_tipc/configs/clas/resnet50_vd/resnet50_vd_ucmerced.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/ucmerced.yaml
 save_dir: ./test_tipc/output/clas/resnet50_vd/
 
 model: !Node
-    type: ResNet50_vd
-    args:
-        num_classes: 21
+  type: ResNet50_vd
+  args:
+    num_classes: 21

test_tipc/configs/det/_base_/rsod.yaml

@@ -3,61 +3,55 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: VOCDetDataset
-        args: 
-            data_dir: ./test_tipc/data/rsod/
-            file_list: ./test_tipc/data/rsod/train.txt
-            label_list: ./test_tipc/data/rsod/labels.txt
-            shuffle: True
-    eval: !Node
-        type: VOCDetDataset
-        args:
-            data_dir: ./test_tipc/data/rsod/
-            file_list: ./test_tipc/data/rsod/val.txt
-            label_list: ./test_tipc/data/rsod/labels.txt
-            shuffle: False
+  train: !Node
+    type: VOCDetDataset
+    args:
+      data_dir: ./test_tipc/data/rsod/
+      file_list: ./test_tipc/data/rsod/train.txt
+      label_list: ./test_tipc/data/rsod/labels.txt
+      shuffle: True
+  eval: !Node
+    type: VOCDetDataset
+    args:
+      data_dir: ./test_tipc/data/rsod/
+      file_list: ./test_tipc/data/rsod/val.txt
+      label_list: ./test_tipc/data/rsod/labels.txt
+      shuffle: False
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomDistort
-        - !Node
-          type: RandomExpand
-        - !Node
-          type: RandomCrop
-        - !Node
-          type: RandomHorizontalFlip
-        - !Node
-          type: BatchRandomResize
-          args:
-            target_sizes: [320, 352, 384, 416, 448, 480, 512, 544, 576, 608]
-            interp: RANDOM
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.485, 0.456, 0.406]
-            std: [0.229, 0.224, 0.225]
-        - !Node
-          type: ArrangeDetector
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 608
-            interp: CUBIC
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.485, 0.456, 0.406]
-            std: [0.229, 0.224, 0.225]
-        - !Node
-          type: ArrangeDetector
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomDistort
+    - !Node
+      type: RandomExpand
+    - !Node
+      type: RandomCrop
+    - !Node
+      type: RandomHorizontalFlip
+    - !Node
+      type: BatchRandomResize
+      args:
+        target_sizes: [320, 352, 384, 416, 448, 480, 512, 544, 576, 608]
+        interp: RANDOM
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.485, 0.456, 0.406]
+        std: [0.229, 0.224, 0.225]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 608
+        interp: CUBIC
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.485, 0.456, 0.406]
+        std: [0.229, 0.224, 0.225]
 download_on: False
 
 num_epochs: 10
@@ -69,6 +63,6 @@ learning_rate: 0.0001
 use_vdl: False
 resume_checkpoint: ''
 train:
-    pretrain_weights: COCO
-    warmup_steps: 0
-    warmup_start_lr: 0.0
+  pretrain_weights: COCO
+  warmup_steps: 0
+  warmup_start_lr: 0.0

test_tipc/configs/det/_base_/sarship.yaml

@@ -3,61 +3,55 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: VOCDetDataset
-        args: 
-            data_dir: ./test_tipc/data/sarship/
-            file_list: ./test_tipc/data/sarship/train.txt
-            label_list: ./test_tipc/data/sarship/labels.txt
-            shuffle: True
-    eval: !Node
-        type: VOCDetDataset
-        args:
-            data_dir: ./test_tipc/data/sarship/
-            file_list: ./test_tipc/data/sarship/eval.txt
-            label_list: ./test_tipc/data/sarship/labels.txt
-            shuffle: False
+  train: !Node
+    type: VOCDetDataset
+    args:
+      data_dir: ./test_tipc/data/sarship/
+      file_list: ./test_tipc/data/sarship/train.txt
+      label_list: ./test_tipc/data/sarship/labels.txt
+      shuffle: True
+  eval: !Node
+    type: VOCDetDataset
+    args:
+      data_dir: ./test_tipc/data/sarship/
+      file_list: ./test_tipc/data/sarship/eval.txt
+      label_list: ./test_tipc/data/sarship/labels.txt
+      shuffle: False
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomDistort
-        - !Node
-          type: RandomExpand
-        - !Node
-          type: RandomCrop
-        - !Node
-          type: RandomHorizontalFlip
-        - !Node
-          type: BatchRandomResize
-          args:
-            target_sizes: [320, 352, 384, 416, 448, 480, 512, 544, 576, 608]
-            interp: RANDOM
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.485, 0.456, 0.406]
-            std: [0.229, 0.224, 0.225]
-        - !Node
-          type: ArrangeDetector
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 608
-            interp: CUBIC
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.485, 0.456, 0.406]
-            std: [0.229, 0.224, 0.225]
-        - !Node
-          type: ArrangeDetector
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomDistort
+    - !Node
+      type: RandomExpand
+    - !Node
+      type: RandomCrop
+    - !Node
+      type: RandomHorizontalFlip
+    - !Node
+      type: BatchRandomResize
+      args:
+        target_sizes: [320, 352, 384, 416, 448, 480, 512, 544, 576, 608]
+        interp: RANDOM
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.485, 0.456, 0.406]
+        std: [0.229, 0.224, 0.225]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 608
+        interp: CUBIC
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.485, 0.456, 0.406]
+        std: [0.229, 0.224, 0.225]
 download_on: False
 download_url: https://paddlers.bj.bcebos.com/datasets/sarship.zip
 download_path: ./test_tipc/data/
@@ -71,6 +65,6 @@ learning_rate: 0.0001
 use_vdl: False
 resume_checkpoint: ''
 train:
-    pretrain_weights: COCO
-    warmup_steps: 0
-    warmup_start_lr: 0.0
+  pretrain_weights: COCO
+  warmup_steps: 0
+  warmup_start_lr: 0.0

test_tipc/configs/det/faster_rcnn/faster_rcnn_rsod.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/rsod.yaml
 save_dir: ./test_tipc/output/det/faster_rcnn/
 
 model: !Node
-    type: FasterRCNN
-    args:
-        num_classes: 4
+  type: FasterRCNN
+  args:
+    num_classes: 4

test_tipc/configs/det/faster_rcnn/faster_rcnn_sarship.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/sarship.yaml
 save_dir: ./test_tipc/output/det/faster_rcnn/
 
 model: !Node
-    type: FasterRCNN
-    args:
-        num_classes: 1
+  type: FasterRCNN
+  args:
+    num_classes: 1

test_tipc/configs/det/ppyolo/ppyolo_rsod.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/rsod.yaml
 save_dir: ./test_tipc/output/det/ppyolo/
 
 model: !Node
-    type: PPYOLO
-    args:
-        num_classes: 4
+  type: PPYOLO
+  args:
+    num_classes: 4

test_tipc/configs/det/ppyolo/ppyolo_sarship.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/sarship.yaml
 save_dir: ./test_tipc/output/det/ppyolo/
 
 model: !Node
-    type: PPYOLO
-    args:
-        num_classes: 1
+  type: PPYOLO
+  args:
+    num_classes: 1

test_tipc/configs/det/ppyolo_tiny/ppyolo_tiny_rsod.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/rsod.yaml
 save_dir: ./test_tipc/output/det/ppyolo_tiny/
 
 model: !Node
-    type: PPYOLOTiny
-    args:
-        num_classes: 4
+  type: PPYOLOTiny
+  args:
+    num_classes: 4

test_tipc/configs/det/ppyolo_tiny/ppyolo_tiny_sarship.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/sarship.yaml
 save_dir: ./test_tipc/output/det/ppyolo_tiny/
 
 model: !Node
-    type: PPYOLOTiny
-    args:
-        num_classes: 1
+  type: PPYOLOTiny
+  args:
+    num_classes: 1

test_tipc/configs/det/ppyolov2/ppyolov2_rsod.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/rsod.yaml
 save_dir: ./test_tipc/output/det/ppyolov2/
 
 model: !Node
-    type: PPYOLOv2
-    args:
-        num_classes: 4
+  type: PPYOLOv2
+  args:
+    num_classes: 4

test_tipc/configs/det/ppyolov2/ppyolov2_sarship.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/sarship.yaml
 save_dir: ./test_tipc/output/det/ppyolov2/
 
 model: !Node
-    type: PPYOLOv2
-    args:
-        num_classes: 1
+  type: PPYOLOv2
+  args:
+    num_classes: 1

test_tipc/configs/det/yolov3/yolov3_rsod.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/rsod.yaml
 save_dir: ./test_tipc/output/det/yolov3/
 
 model: !Node
-    type: YOLOv3
-    args:
-        num_classes: 4
+  type: YOLOv3
+  args:
+    num_classes: 4

test_tipc/configs/det/yolov3/yolov3_sarship.yaml

@@ -5,6 +5,6 @@ _base_: ../_base_/sarship.yaml
 save_dir: ./test_tipc/output/det/yolov3/
 
 model: !Node
-    type: YOLOv3
-    args:
-        num_classes: 1
+  type: YOLOv3
+  args:
+    num_classes: 1

test_tipc/configs/res/_base_/rssr.yaml

@@ -3,61 +3,55 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: ResDataset
-        args: 
-            data_dir: ./test_tipc/data/rssr/
-            file_list: ./test_tipc/data/rssr/train.txt
-            num_workers: 0
-            shuffle: True
-            sr_factor: 4
-    eval: !Node
-        type: ResDataset
-        args:
-            data_dir: ./test_tipc/data/rssr/
-            file_list: ./test_tipc/data/rssr/val.txt
-            num_workers: 0
-            shuffle: False
-            sr_factor: 4
+  train: !Node
+    type: ResDataset
+    args:
+      data_dir: ./test_tipc/data/rssr/
+      file_list: ./test_tipc/data/rssr/train.txt
+      num_workers: 0
+      shuffle: True
+      sr_factor: 4
+  eval: !Node
+    type: ResDataset
+    args:
+      data_dir: ./test_tipc/data/rssr/
+      file_list: ./test_tipc/data/rssr/val.txt
+      num_workers: 0
+      shuffle: False
+      sr_factor: 4
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: RandomCrop
-          args:
-            crop_size: 32
-        - !Node
-          type: RandomHorizontalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: RandomVerticalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.0, 0.0, 0.0]
-            std: [1.0, 1.0, 1.0]
-        - !Node
-          type: ArrangeRestorer
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 256
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.0, 0.0, 0.0]
-            std: [1.0, 1.0, 1.0]
-        - !Node
-          type: ArrangeRestorer
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: RandomCrop
+      args:
+        crop_size: 32
+    - !Node
+      type: RandomHorizontalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: RandomVerticalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.0, 0.0, 0.0]
+        std: [1.0, 1.0, 1.0]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 256
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.0, 0.0, 0.0]
+        std: [1.0, 1.0, 1.0]
 download_on: False
 download_url: https://paddlers.bj.bcebos.com/datasets/rssr.zip
 download_path: ./test_tipc/data/

test_tipc/configs/res/drn/drn_rssr.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/rssr.yaml
 save_dir: ./test_tipc/output/res/drn/
 
 model: !Node
-    type: DRN
+  type: DRN

test_tipc/configs/res/esrgan/esrgan_rssr.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/rssr.yaml
 save_dir: ./test_tipc/output/res/esrgan/
 
 model: !Node
-    type: ESRGAN
+  type: ESRGAN

test_tipc/configs/res/lesrcnn/lesrcnn_rssr.yaml

@@ -5,4 +5,4 @@ _base_: ../_base_/rssr.yaml
 save_dir: ./test_tipc/output/res/lesrcnn/
 
 model: !Node
-    type: LESRCNN
+  type: LESRCNN

test_tipc/configs/seg/_base_/rsseg.yaml

@@ -3,59 +3,53 @@
 seed: 1024
 
 datasets:
-    train: !Node
-        type: SegDataset
-        args: 
-            data_dir: ./test_tipc/data/rsseg/
-            file_list: ./test_tipc/data/rsseg/train.txt
-            label_list: ./test_tipc/data/rsseg/labels.txt
-            num_workers: 0
-            shuffle: True
-    eval: !Node
-        type: SegDataset
-        args:
-            data_dir: ./test_tipc/data/rsseg/
-            file_list: ./test_tipc/data/rsseg/val.txt
-            label_list: ./test_tipc/data/rsseg/labels.txt
-            num_workers: 0
-            shuffle: False
+  train: !Node
+    type: SegDataset
+    args:
+      data_dir: ./test_tipc/data/rsseg/
+      file_list: ./test_tipc/data/rsseg/train.txt
+      label_list: ./test_tipc/data/rsseg/labels.txt
+      num_workers: 0
+      shuffle: True
+  eval: !Node
+    type: SegDataset
+    args:
+      data_dir: ./test_tipc/data/rsseg/
+      file_list: ./test_tipc/data/rsseg/val.txt
+      label_list: ./test_tipc/data/rsseg/labels.txt
+      num_workers: 0
+      shuffle: False
 transforms:
-    train:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 512
-        - !Node
-          type: RandomHorizontalFlip
-          args:
-            prob: 0.5
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ArrangeSegmenter
-          args: ['train']
-    eval:
-        - !Node
-          type: DecodeImg
-        - !Node
-          type: Resize
-          args:
-            target_size: 512
-        - !Node
-          type: Normalize
-          args:
-            mean: [0.5, 0.5, 0.5]
-            std: [0.5, 0.5, 0.5]
-        - !Node
-          type: ReloadMask
-        - !Node
-          type: ArrangeSegmenter
-          args: ['eval']
+  train:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 512
+    - !Node
+      type: RandomHorizontalFlip
+      args:
+        prob: 0.5
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+  eval:
+    - !Node
+      type: DecodeImg
+    - !Node
+      type: Resize
+      args:
+        target_size: 512
+    - !Node
+      type: Normalize
+      args:
+        mean: [0.5, 0.5, 0.5]
+        std: [0.5, 0.5, 0.5]
+    - !Node
+      type: ReloadMask
 download_on: False
 download_url: https://paddlers.bj.bcebos.com/datasets/rsseg.zip
 download_path: ./test_tipc/data/

test_tipc/configs/seg/bisenetv2/bisenetv2_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/bisenetv2/
 
 model: !Node
-    type: BiSeNetV2
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: BiSeNetV2
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/deeplabv3p/deeplabv3p_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/deeplabv3p/
 
 model: !Node
-    type: DeepLabV3P
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: DeepLabV3P
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/factseg/factseg_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/factseg/
 
 model: !Node
-    type: FactSeg
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: FactSeg
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/farseg/farseg_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/farseg/
 
 model: !Node
-    type: FarSeg
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: FarSeg
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/fast_scnn/fast_scnn_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/fast_scnn/
 
 model: !Node
-    type: FastSCNN
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: FastSCNN
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/hrnet/hrnet_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/hrnet/
 
 model: !Node
-    type: HRNet
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: HRNet
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/configs/seg/unet/unet_rsseg.yaml

@@ -5,7 +5,7 @@ _base_: ../_base_/rsseg.yaml
 save_dir: ./test_tipc/output/seg/unet/
 
 model: !Node
-    type: UNet
-    args:
-        in_channels: 3
-        num_classes: 5
+  type: UNet
+  args:
+    in_channels: 3
+    num_classes: 5

test_tipc/infer.py

@@ -145,22 +145,6 @@ class TIPCPredictor(object):
     def preprocess(self, images, transforms):
         preprocessed_samples, batch_trans_info = self._model.preprocess(
             images, transforms, to_tensor=False)
-        if self._model.model_type == 'classifier':
-            preprocessed_samples = {'image': preprocessed_samples}
-        elif self._model.model_type == 'segmenter':
-            preprocessed_samples = {'image': preprocessed_samples[0]}
-        elif self._model.model_type == 'detector':
-            pass
-        elif self._model.model_type == 'change_detector':
-            preprocessed_samples = {
-                'image': preprocessed_samples[0],
-                'image2': preprocessed_samples[1]
-            }
-        elif self._model.model_type == 'restorer':
-            preprocessed_samples = {'image': preprocessed_samples[0]}
-        else:
-            logging.error(
-                "Invalid model type {}".format(self.model_type), exit=True)
         return preprocessed_samples, batch_trans_info
 
     def postprocess(self, net_outputs, batch_restore_list, topk=1):

tests/tasks/test_slider_predict.py

@@ -232,7 +232,7 @@ class _TestSliderPredictNamespace:
 class TestSegSliderPredict(_TestSliderPredictNamespace.TestSliderPredict):
     def setUp(self):
         self.model = pdrs.tasks.seg.UNet(in_channels=10)
-        self.transforms = [T.Normalize([0.5] * 10, [0.5] * 10)]
+        self.transforms = T.Compose([T.Normalize([0.5] * 10, [0.5] * 10)])
         self.image_path = "data/ssst/multispectral.tif"
         self.ref_path = self.image_path
         self.basename = osp.basename(self.ref_path)
@@ -241,7 +241,7 @@ class TestSegSliderPredict(_TestSliderPredictNamespace.TestSliderPredict):
 class TestCDSliderPredict(_TestSliderPredictNamespace.TestSliderPredict):
     def setUp(self):
         self.model = pdrs.tasks.cd.BIT(in_channels=10)
-        self.transforms = [T.Normalize([0.5] * 10, [0.5] * 10)]
+        self.transforms = T.Compose([T.Normalize([0.5] * 10, [0.5] * 10)])
         self.image_path = ("data/ssmt/multispectral_t1.tif",
                            "data/ssmt/multispectral_t2.tif")
         self.ref_path = self.image_path[0]