segformer

mindnlp.transformers.models.segformer.configuration_segformer

SegFormer model configuration

mindnlp.transformers.models.segformer.configuration_segformer.SegformerConfig

Bases: PretrainedConfig

This is the configuration class to store the configuration of a [SegformerModel]. It is used to instantiate an SegFormer model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the SegFormer nvidia/segformer-b0-finetuned-ade-512-512 architecture.

Configuration objects inherit from [PretrainedConfig] and can be used to control the model outputs. Read the documentation from [PretrainedConfig] for more information.

PARAMETER	DESCRIPTION
num_channels	The number of input channels. TYPE: `int`, *optional*, defaults to 3 DEFAULT: 3
num_encoder_blocks	The number of encoder blocks (i.e. stages in the Mix Transformer encoder). TYPE: `int`, *optional*, defaults to 4 DEFAULT: 4
depths	The number of layers in each encoder block. TYPE: `List[int]`, *optional*, defaults to `[2, 2, 2, 2]` DEFAULT: [2, 2, 2, 2]
sr_ratios	Sequence reduction ratios in each encoder block. TYPE: `List[int]`, *optional*, defaults to `[8, 4, 2, 1]` DEFAULT: [8, 4, 2, 1]
hidden_sizes	Dimension of each of the encoder blocks. TYPE: `List[int]`, *optional*, defaults to `[32, 64, 160, 256]` DEFAULT: [32, 64, 160, 256]
patch_sizes	Patch size before each encoder block. TYPE: `List[int]`, *optional*, defaults to `[7, 3, 3, 3]` DEFAULT: [7, 3, 3, 3]
strides	Stride before each encoder block. TYPE: `List[int]`, *optional*, defaults to `[4, 2, 2, 2]` DEFAULT: [4, 2, 2, 2]
num_attention_heads	Number of attention heads for each attention layer in each block of the Transformer encoder. TYPE: `List[int]`, *optional*, defaults to `[1, 2, 5, 8]` DEFAULT: [1, 2, 5, 8]
mlp_ratios	Ratio of the size of the hidden layer compared to the size of the input layer of the Mix FFNs in the encoder blocks. TYPE: `List[int]`, *optional*, defaults to `[4, 4, 4, 4]` DEFAULT: [4, 4, 4, 4]
hidden_act	The non-linear activation function (function or string) in the encoder and pooler. If string, "gelu", "relu", "selu" and "gelu_new" are supported. TYPE: `str` or `function`, *optional*, defaults to `"gelu"` DEFAULT: 'gelu'
hidden_dropout_prob	The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. TYPE: `float`, *optional*, defaults to 0.0 DEFAULT: 0.0
attention_probs_dropout_prob	The dropout ratio for the attention probabilities. TYPE: `float`, *optional*, defaults to 0.0 DEFAULT: 0.0
classifier_dropout_prob	The dropout probability before the classification head. TYPE: `float`, *optional*, defaults to 0.1 DEFAULT: 0.1
initializer_range	The standard deviation of the truncated_normal_initializer for initializing all weight matrices. TYPE: `float`, *optional*, defaults to 0.02 DEFAULT: 0.02
drop_path_rate	The dropout probability for stochastic depth, used in the blocks of the Transformer encoder. TYPE: `float`, *optional*, defaults to 0.1 DEFAULT: 0.1
layer_norm_eps	The epsilon used by the layer normalization layers. TYPE: `float`, *optional*, defaults to 1e-06 DEFAULT: 1e-06
decoder_hidden_size	The dimension of the all-MLP decode head. TYPE: `int`, *optional*, defaults to 256 DEFAULT: 256
semantic_loss_ignore_index	The index that is ignored by the loss function of the semantic segmentation model. TYPE: `int`, *optional*, defaults to 255 DEFAULT: 255

Example

>>> from transformers import SegformerModel, SegformerConfig
...
>>> # Initializing a SegFormer nvidia/segformer-b0-finetuned-ade-512-512 style configuration
>>> configuration = SegformerConfig()
...
>>> # Initializing a model from the nvidia/segformer-b0-finetuned-ade-512-512 style configuration
>>> model = SegformerModel(configuration)
...
>>> # Accessing the model configuration
>>> configuration = model.config

Source code in mindnlp/transformers/models/segformer/configuration_segformer.py

class SegformerConfig(PretrainedConfig):
    r"""
    This is the configuration class to store the configuration of a [`SegformerModel`]. It is used to instantiate an
    SegFormer model according to the specified arguments, defining the model architecture. Instantiating a
    configuration with the defaults will yield a similar configuration to that of the SegFormer
    [nvidia/segformer-b0-finetuned-ade-512-512](https://huggingface.co/nvidia/segformer-b0-finetuned-ade-512-512)
    architecture.

    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
    documentation from [`PretrainedConfig`] for more information.

    Args:
        num_channels (`int`, *optional*, defaults to 3):
            The number of input channels.
        num_encoder_blocks (`int`, *optional*, defaults to 4):
            The number of encoder blocks (i.e. stages in the Mix Transformer encoder).
        depths (`List[int]`, *optional*, defaults to `[2, 2, 2, 2]`):
            The number of layers in each encoder block.
        sr_ratios (`List[int]`, *optional*, defaults to `[8, 4, 2, 1]`):
            Sequence reduction ratios in each encoder block.
        hidden_sizes (`List[int]`, *optional*, defaults to `[32, 64, 160, 256]`):
            Dimension of each of the encoder blocks.
        patch_sizes (`List[int]`, *optional*, defaults to `[7, 3, 3, 3]`):
            Patch size before each encoder block.
        strides (`List[int]`, *optional*, defaults to `[4, 2, 2, 2]`):
            Stride before each encoder block.
        num_attention_heads (`List[int]`, *optional*, defaults to `[1, 2, 5, 8]`):
            Number of attention heads for each attention layer in each block of the Transformer encoder.
        mlp_ratios (`List[int]`, *optional*, defaults to `[4, 4, 4, 4]`):
            Ratio of the size of the hidden layer compared to the size of the input layer of the Mix FFNs in the
            encoder blocks.
        hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`):
            The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
            `"relu"`, `"selu"` and `"gelu_new"` are supported.
        hidden_dropout_prob (`float`, *optional*, defaults to 0.0):
            The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
        attention_probs_dropout_prob (`float`, *optional*, defaults to 0.0):
            The dropout ratio for the attention probabilities.
        classifier_dropout_prob (`float`, *optional*, defaults to 0.1):
            The dropout probability before the classification head.
        initializer_range (`float`, *optional*, defaults to 0.02):
            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
        drop_path_rate (`float`, *optional*, defaults to 0.1):
            The dropout probability for stochastic depth, used in the blocks of the Transformer encoder.
        layer_norm_eps (`float`, *optional*, defaults to 1e-06):
            The epsilon used by the layer normalization layers.
        decoder_hidden_size (`int`, *optional*, defaults to 256):
            The dimension of the all-MLP decode head.
        semantic_loss_ignore_index (`int`, *optional*, defaults to 255):
            The index that is ignored by the loss function of the semantic segmentation model.

    Example:
        ```python
        >>> from transformers import SegformerModel, SegformerConfig
        ...
        >>> # Initializing a SegFormer nvidia/segformer-b0-finetuned-ade-512-512 style configuration
        >>> configuration = SegformerConfig()
        ...
        >>> # Initializing a model from the nvidia/segformer-b0-finetuned-ade-512-512 style configuration
        >>> model = SegformerModel(configuration)
        ...
        >>> # Accessing the model configuration
        >>> configuration = model.config
        ```
    """
    model_type = "segformer"

    def __init__(
        self,
        num_channels=3,
        num_encoder_blocks=4,
        depths=[2, 2, 2, 2],
        sr_ratios=[8, 4, 2, 1],
        hidden_sizes=[32, 64, 160, 256],
        patch_sizes=[7, 3, 3, 3],
        strides=[4, 2, 2, 2],
        num_attention_heads=[1, 2, 5, 8],
        mlp_ratios=[4, 4, 4, 4],
        hidden_act="gelu",
        hidden_dropout_prob=0.0,
        attention_probs_dropout_prob=0.0,
        classifier_dropout_prob=0.1,
        initializer_range=0.02,
        drop_path_rate=0.1,
        layer_norm_eps=1e-6,
        decoder_hidden_size=256,
        semantic_loss_ignore_index=255,
        **kwargs,
    ):
        '''
        Initializes a new instance of SegformerConfig.

        Args:
            self: The instance of the SegformerConfig class.
            num_channels (int, optional): The number of input channels. Defaults to 3.
            num_encoder_blocks (int, optional): The number of encoder blocks. Defaults to 4.
            depths (list of int): The depths of each stage in the encoder block.
            sr_ratios (list of int): The spatial reduction ratios for each stage in the encoder block.
            hidden_sizes (list of int): The hidden sizes for each stage in the encoder block.
            patch_sizes (list of int): The patch sizes for each stage in the encoder block.
            strides (list of int): The strides for each stage in the encoder block.
            num_attention_heads (list of int): The number of attention heads for each stage in the encoder block.
            mlp_ratios (list of int): The ratio of mlp hidden size to the input size for each stage in the encoder block.
            hidden_act (str): The activation function for the hidden layer. Defaults to 'gelu'.
            hidden_dropout_prob (float): The dropout probability for the hidden layers. Defaults to 0.0.
            attention_probs_dropout_prob (float): The dropout probability for the attention probabilities. Defaults to 0.0.
            classifier_dropout_prob (float): The dropout probability for the classifier. Defaults to 0.1.
            initializer_range (float): The standard deviation of the truncated_normal_initializer for initializing all
                weight matrices. Defaults to 0.02.
            drop_path_rate (float): The dropout rate for stochastic depth. Defaults to 0.1.
            layer_norm_eps (float): The epsilon value for layer normalization. Defaults to 1e-06.
            decoder_hidden_size (int): The hidden size of the decoder.
            semantic_loss_ignore_index (int): The index to ignore in the semantic loss calculation.
            **kwargs: Additional keyword arguments.

        Returns:
            None.

        Raises:
            FutureWarning: If 'reshape_last_stage' is set to False in the kwargs, a warning about the deprecation of
                this argument will be raised.
        '''
        super().__init__(**kwargs)

        if "reshape_last_stage" in kwargs and kwargs["reshape_last_stage"] is False:
            warnings.warn(
                "Reshape_last_stage is set to False in this config. This argument is deprecated and will soon be"
                " removed, as the behaviour will default to that of reshape_last_stage = True.",
                FutureWarning,
            )

        self.num_channels = num_channels
        self.num_encoder_blocks = num_encoder_blocks
        self.depths = depths
        self.sr_ratios = sr_ratios
        self.hidden_sizes = hidden_sizes
        self.patch_sizes = patch_sizes
        self.strides = strides
        self.mlp_ratios = mlp_ratios
        self.num_attention_heads = num_attention_heads
        self.hidden_act = hidden_act
        self.hidden_dropout_prob = hidden_dropout_prob
        self.attention_probs_dropout_prob = attention_probs_dropout_prob
        self.classifier_dropout_prob = classifier_dropout_prob
        self.initializer_range = initializer_range
        self.drop_path_rate = drop_path_rate
        self.layer_norm_eps = layer_norm_eps
        self.decoder_hidden_size = decoder_hidden_size
        self.reshape_last_stage = kwargs.get("reshape_last_stage", True)
        self.semantic_loss_ignore_index = semantic_loss_ignore_index

mindnlp.transformers.models.segformer.configuration_segformer.SegformerConfig.__init__(num_channels=3, num_encoder_blocks=4, depths=[2, 2, 2, 2], sr_ratios=[8, 4, 2, 1], hidden_sizes=[32, 64, 160, 256], patch_sizes=[7, 3, 3, 3], strides=[4, 2, 2, 2], num_attention_heads=[1, 2, 5, 8], mlp_ratios=[4, 4, 4, 4], hidden_act='gelu', hidden_dropout_prob=0.0, attention_probs_dropout_prob=0.0, classifier_dropout_prob=0.1, initializer_range=0.02, drop_path_rate=0.1, layer_norm_eps=1e-06, decoder_hidden_size=256, semantic_loss_ignore_index=255, **kwargs)

Initializes a new instance of SegformerConfig.

PARAMETER	DESCRIPTION
self	The instance of the SegformerConfig class.
num_channels	The number of input channels. Defaults to 3. TYPE: int DEFAULT: 3
num_encoder_blocks	The number of encoder blocks. Defaults to 4. TYPE: int DEFAULT: 4
depths	The depths of each stage in the encoder block. TYPE: list of int DEFAULT: [2, 2, 2, 2]
sr_ratios	The spatial reduction ratios for each stage in the encoder block. TYPE: list of int DEFAULT: [8, 4, 2, 1]
hidden_sizes	The hidden sizes for each stage in the encoder block. TYPE: list of int DEFAULT: [32, 64, 160, 256]
patch_sizes	The patch sizes for each stage in the encoder block. TYPE: list of int DEFAULT: [7, 3, 3, 3]
strides	The strides for each stage in the encoder block. TYPE: list of int DEFAULT: [4, 2, 2, 2]
num_attention_heads	The number of attention heads for each stage in the encoder block. TYPE: list of int DEFAULT: [1, 2, 5, 8]
mlp_ratios	The ratio of mlp hidden size to the input size for each stage in the encoder block. TYPE: list of int DEFAULT: [4, 4, 4, 4]
hidden_act	The activation function for the hidden layer. Defaults to 'gelu'. TYPE: str DEFAULT: 'gelu'
hidden_dropout_prob	The dropout probability for the hidden layers. Defaults to 0.0. TYPE: float DEFAULT: 0.0
attention_probs_dropout_prob	The dropout probability for the attention probabilities. Defaults to 0.0. TYPE: float DEFAULT: 0.0
classifier_dropout_prob	The dropout probability for the classifier. Defaults to 0.1. TYPE: float DEFAULT: 0.1
initializer_range	The standard deviation of the truncated_normal_initializer for initializing all weight matrices. Defaults to 0.02. TYPE: float DEFAULT: 0.02
drop_path_rate	The dropout rate for stochastic depth. Defaults to 0.1. TYPE: float DEFAULT: 0.1
layer_norm_eps	The epsilon value for layer normalization. Defaults to 1e-06. TYPE: float DEFAULT: 1e-06
decoder_hidden_size	The hidden size of the decoder. TYPE: int DEFAULT: 256
semantic_loss_ignore_index	The index to ignore in the semantic loss calculation. TYPE: int DEFAULT: 255
**kwargs	Additional keyword arguments. DEFAULT: {}

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
FutureWarning	If 'reshape_last_stage' is set to False in the kwargs, a warning about the deprecation of this argument will be raised.

Source code in mindnlp/transformers/models/segformer/configuration_segformer.py

def __init__(
    self,
    num_channels=3,
    num_encoder_blocks=4,
    depths=[2, 2, 2, 2],
    sr_ratios=[8, 4, 2, 1],
    hidden_sizes=[32, 64, 160, 256],
    patch_sizes=[7, 3, 3, 3],
    strides=[4, 2, 2, 2],
    num_attention_heads=[1, 2, 5, 8],
    mlp_ratios=[4, 4, 4, 4],
    hidden_act="gelu",
    hidden_dropout_prob=0.0,
    attention_probs_dropout_prob=0.0,
    classifier_dropout_prob=0.1,
    initializer_range=0.02,
    drop_path_rate=0.1,
    layer_norm_eps=1e-6,
    decoder_hidden_size=256,
    semantic_loss_ignore_index=255,
    **kwargs,
):
    '''
    Initializes a new instance of SegformerConfig.

    Args:
        self: The instance of the SegformerConfig class.
        num_channels (int, optional): The number of input channels. Defaults to 3.
        num_encoder_blocks (int, optional): The number of encoder blocks. Defaults to 4.
        depths (list of int): The depths of each stage in the encoder block.
        sr_ratios (list of int): The spatial reduction ratios for each stage in the encoder block.
        hidden_sizes (list of int): The hidden sizes for each stage in the encoder block.
        patch_sizes (list of int): The patch sizes for each stage in the encoder block.
        strides (list of int): The strides for each stage in the encoder block.
        num_attention_heads (list of int): The number of attention heads for each stage in the encoder block.
        mlp_ratios (list of int): The ratio of mlp hidden size to the input size for each stage in the encoder block.
        hidden_act (str): The activation function for the hidden layer. Defaults to 'gelu'.
        hidden_dropout_prob (float): The dropout probability for the hidden layers. Defaults to 0.0.
        attention_probs_dropout_prob (float): The dropout probability for the attention probabilities. Defaults to 0.0.
        classifier_dropout_prob (float): The dropout probability for the classifier. Defaults to 0.1.
        initializer_range (float): The standard deviation of the truncated_normal_initializer for initializing all
            weight matrices. Defaults to 0.02.
        drop_path_rate (float): The dropout rate for stochastic depth. Defaults to 0.1.
        layer_norm_eps (float): The epsilon value for layer normalization. Defaults to 1e-06.
        decoder_hidden_size (int): The hidden size of the decoder.
        semantic_loss_ignore_index (int): The index to ignore in the semantic loss calculation.
        **kwargs: Additional keyword arguments.

    Returns:
        None.

    Raises:
        FutureWarning: If 'reshape_last_stage' is set to False in the kwargs, a warning about the deprecation of
            this argument will be raised.
    '''
    super().__init__(**kwargs)

    if "reshape_last_stage" in kwargs and kwargs["reshape_last_stage"] is False:
        warnings.warn(
            "Reshape_last_stage is set to False in this config. This argument is deprecated and will soon be"
            " removed, as the behaviour will default to that of reshape_last_stage = True.",
            FutureWarning,
        )

    self.num_channels = num_channels
    self.num_encoder_blocks = num_encoder_blocks
    self.depths = depths
    self.sr_ratios = sr_ratios
    self.hidden_sizes = hidden_sizes
    self.patch_sizes = patch_sizes
    self.strides = strides
    self.mlp_ratios = mlp_ratios
    self.num_attention_heads = num_attention_heads
    self.hidden_act = hidden_act
    self.hidden_dropout_prob = hidden_dropout_prob
    self.attention_probs_dropout_prob = attention_probs_dropout_prob
    self.classifier_dropout_prob = classifier_dropout_prob
    self.initializer_range = initializer_range
    self.drop_path_rate = drop_path_rate
    self.layer_norm_eps = layer_norm_eps
    self.decoder_hidden_size = decoder_hidden_size
    self.reshape_last_stage = kwargs.get("reshape_last_stage", True)
    self.semantic_loss_ignore_index = semantic_loss_ignore_index

mindnlp.transformers.models.segformer.image_processing_segformer

Image processor class for Segformer.

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor

Bases: BaseImageProcessor

Constructs a Segformer image processor.

PARAMETER	DESCRIPTION
do_resize	Whether to resize the image's (height, width) dimensions to the specified (size["height"], size["width"]). Can be overridden by the do_resize parameter in the preprocess method. TYPE: `bool`, *optional*, defaults to `True` DEFAULT: True
size	512, "width": 512}): Size of the output image after resizing. Can be overridden by thesizeparameter in thepreprocess` method. TYPE: `Dict[str, int]` *optional*, defaults to `{"height" DEFAULT: None
resample	Resampling filter to use if resizing the image. Can be overridden by the resample parameter in the preprocess method. TYPE: `PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR` DEFAULT: BILINEAR
do_rescale	Whether to rescale the image by the specified scale rescale_factor. Can be overridden by the do_rescale parameter in the preprocess method. TYPE: `bool`, *optional*, defaults to `True` DEFAULT: True
rescale_factor	Whether to normalize the image. Can be overridden by the do_normalize parameter in the preprocess method. TYPE: `int` or `float`, *optional*, defaults to `1/255` DEFAULT: 1 / 255
do_normalize	Whether to normalize the image. Can be overridden by the do_normalize parameter in the preprocess method. TYPE: `bool`, *optional*, defaults to `True` DEFAULT: True
image_mean	Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the image_mean parameter in the preprocess method. TYPE: `float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_MEAN` DEFAULT: None
image_std	Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the image_std parameter in the preprocess method. TYPE: `float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_STD` DEFAULT: None
do_reduce_labels	Whether or not to reduce all label values of segmentation maps by 1. Usually used for datasets where 0 is used for background, and background itself is not included in all classes of a dataset (e.g. ADE20k). The background label will be replaced by 255. Can be overridden by the do_reduce_labels parameter in the preprocess method. TYPE: `bool`, *optional*, defaults to `False` DEFAULT: False

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

class SegformerImageProcessor(BaseImageProcessor):
    r"""
    Constructs a Segformer image processor.

    Args:
        do_resize (`bool`, *optional*, defaults to `True`):
            Whether to resize the image's (height, width) dimensions to the specified `(size["height"],
            size["width"])`. Can be overridden by the `do_resize` parameter in the `preprocess` method.
        size (`Dict[str, int]` *optional*, defaults to `{"height": 512, "width": 512}`):
            Size of the output image after resizing. Can be overridden by the `size` parameter in the `preprocess`
            method.
        resample (`PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR`):
            Resampling filter to use if resizing the image. Can be overridden by the `resample` parameter in the
            `preprocess` method.
        do_rescale (`bool`, *optional*, defaults to `True`):
            Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by the `do_rescale`
            parameter in the `preprocess` method.
        rescale_factor (`int` or `float`, *optional*, defaults to `1/255`):
            Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess`
            method.
        do_normalize (`bool`, *optional*, defaults to `True`):
            Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess`
            method.
        image_mean (`float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`):
            Mean to use if normalizing the image. This is a float or list of floats the length of the number of
            channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method.
        image_std (`float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_STD`):
            Standard deviation to use if normalizing the image. This is a float or list of floats the length of the
            number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method.
        do_reduce_labels (`bool`, *optional*, defaults to `False`):
            Whether or not to reduce all label values of segmentation maps by 1. Usually used for datasets where 0 is
            used for background, and background itself is not included in all classes of a dataset (e.g. ADE20k). The
            background label will be replaced by 255. Can be overridden by the `do_reduce_labels` parameter in the
            `preprocess` method.
    """
    model_input_names = ["pixel_values"]

    def __init__(
        self,
        do_resize: bool = True,
        size: Dict[str, int] = None,
        resample: PILImageResampling = PILImageResampling.BILINEAR,
        do_rescale: bool = True,
        rescale_factor: Union[int, float] = 1 / 255,
        do_normalize: bool = True,
        image_mean: Optional[Union[float, List[float]]] = None,
        image_std: Optional[Union[float, List[float]]] = None,
        do_reduce_labels: bool = False,
        **kwargs,
    ) -> None:
        """Initialize the SegformerImageProcessor.

        This method initializes the SegformerImageProcessor object with the provided parameters.

        Args:
            self: The SegformerImageProcessor object.
            do_resize (bool, optional): Whether to resize the image. Defaults to True.
            size (Dict[str, int], optional): The desired height and width of the image.
                Defaults to {'height': 512, 'width': 512}.
            resample (PILImageResampling, optional): The resampling method to use during resizing.
                Defaults to PILImageResampling.BILINEAR.
            do_rescale (bool, optional): Whether to rescale the image. Defaults to True.
            rescale_factor (Union[int, float], optional): The rescale factor to apply to the image. Defaults to 1 / 255.
            do_normalize (bool, optional): Whether to normalize the image. Defaults to True.
            image_mean (Optional[Union[float, List[float]]], optional): The mean values used for image normalization.
                Defaults to None, which uses IMAGENET_DEFAULT_MEAN.
            image_std (Optional[Union[float, List[float]]], optional):
                The standard deviation values used for image normalization. Defaults to None, which uses
                IMAGENET_DEFAULT_STD.
            do_reduce_labels (bool, optional): Whether to reduce the number of labels. Defaults to False.

        Returns:
            None

        Raises:
            FutureWarning: If the 'reduce_labels' parameter is used. This parameter is deprecated and will be removed
                in a future version. Please use 'do_reduce_labels' instead.
        """
        if "reduce_labels" in kwargs:
            warnings.warn(
                "The `reduce_labels` parameter is deprecated and will be removed in a future version. Please use "
                "`do_reduce_labels` instead.",
                FutureWarning,
            )
            do_reduce_labels = kwargs.pop("reduce_labels")

        super().__init__(**kwargs)
        size = size if size is not None else {"height": 512, "width": 512}
        size = get_size_dict(size)
        self.do_resize = do_resize
        self.size = size
        self.resample = resample
        self.do_rescale = do_rescale
        self.rescale_factor = rescale_factor
        self.do_normalize = do_normalize
        self.image_mean = image_mean if image_mean is not None else IMAGENET_DEFAULT_MEAN
        self.image_std = image_std if image_std is not None else IMAGENET_DEFAULT_STD
        self.do_reduce_labels = do_reduce_labels
        self._valid_processor_keys = [
            "images",
            "segmentation_maps",
            "do_resize",
            "size",
            "resample",
            "do_rescale",
            "rescale_factor",
            "do_normalize",
            "image_mean",
            "image_std",
            "do_reduce_labels",
            "return_tensors",
            "data_format",
            "input_data_format",
        ]

    @classmethod
    def from_dict(cls, image_processor_dict: Dict[str, Any], **kwargs):
        """
        Overrides the `from_dict` method from the base class to make sure `do_reduce_labels` is updated if image
        processor is created using from_dict and kwargs e.g. `SegformerImageProcessor.from_pretrained(checkpoint,
        reduce_labels=True)`
        """
        image_processor_dict = image_processor_dict.copy()
        if "reduce_labels" in kwargs:
            image_processor_dict["reduce_labels"] = kwargs.pop("reduce_labels")
        return super().from_dict(image_processor_dict, **kwargs)

    # Copied from transformers.models.vit.image_processing_vit.ViTImageProcessor.resize
    def resize(
        self,
        image: np.ndarray,
        size: Dict[str, int],
        resample: PILImageResampling = PILImageResampling.BILINEAR,
        data_format: Optional[Union[str, ChannelDimension]] = None,
        input_data_format: Optional[Union[str, ChannelDimension]] = None,
        **kwargs,
    ) -> np.ndarray:
        """
        Resize an image to `(size["height"], size["width"])`.

        Args:
            image (`np.ndarray`):
                Image to resize.
            size (`Dict[str, int]`):
                Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image.
            resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`):
                `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`.
            data_format (`ChannelDimension` or `str`, *optional*):
                The channel dimension format for the output image. If unset, the channel dimension format of the input
                image is used. Can be one of:

                - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
                - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
                - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.
            input_data_format (`ChannelDimension` or `str`, *optional*):
                The channel dimension format for the input image. If unset, the channel dimension format is inferred
                from the input image. Can be one of:

                - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
                - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
                - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.

        Returns:
            `np.ndarray`: The resized image.
        """
        size = get_size_dict(size)
        if "height" not in size or "width" not in size:
            raise ValueError(f"The `size` dictionary must contain the keys `height` and `width`. Got {size.keys()}")
        output_size = (size["height"], size["width"])
        return resize(
            image,
            size=output_size,
            resample=resample,
            data_format=data_format,
            input_data_format=input_data_format,
            **kwargs,
        )

    # Copied from transformers.models.beit.image_processing_beit.BeitImageProcessor.reduce_label
    def reduce_label(self, label: ImageInput) -> np.ndarray:
        """
        Reduces the label values in the input image for Segformer image processing.

        Args:
            self: Instance of the SegformerImageProcessor class.
            label (ImageInput): Input label image to be processed. It should be in a compatible format for processing.

        Returns:
            np.ndarray: A NumPy array representing the processed label image with reduced values.

        Raises:
            None.
        """
        label = to_numpy_array(label)
        # Avoid using underflow conversion
        label[label == 0] = 255
        label = label - 1
        label[label == 254] = 255
        return label

    def _preprocess(
        self,
        image: ImageInput,
        do_reduce_labels: bool,
        do_resize: bool,
        do_rescale: bool,
        do_normalize: bool,
        size: Optional[Dict[str, int]] = None,
        resample: PILImageResampling = None,
        rescale_factor: Optional[float] = None,
        image_mean: Optional[Union[float, List[float]]] = None,
        image_std: Optional[Union[float, List[float]]] = None,
        input_data_format: Optional[Union[str, ChannelDimension]] = None,
    ):
        """
        Preprocesses an input image according to specified options.

        Args:
            self: The instance of the SegformerImageProcessor class.
            image (ImageInput): The input image to be preprocessed.
            do_reduce_labels (bool): Whether to reduce the labels.
            do_resize (bool): Whether to resize the image.
            do_rescale (bool): Whether to rescale the image.
            do_normalize (bool): Whether to normalize the image.
            size (Optional[Dict[str, int]]): Optional dictionary specifying the target size for resizing the image.
                Default is None.
            resample (PILImageResampling): The resampling method to be used during resizing. Default is None.
            rescale_factor (Optional[float]): Optional scaling factor for rescaling the image. Default is None.
            image_mean (Optional[Union[float, List[float]]]): Optional mean value(s) to be used for normalization.
                Default is None.
            image_std (Optional[Union[float, List[float]]]): Optional standard deviation value(s) to be used for
                normalization. Default is None.
            input_data_format (Optional[Union[str, ChannelDimension]]): Optional data format of the input image.
                Default is None.

        Returns:
            None: This method modifies the input image in-place.

        Raises:
            None.

        """
        if do_reduce_labels:
            image = self.reduce_label(image)

        if do_resize:
            image = self.resize(image=image, size=size, resample=resample, input_data_format=input_data_format)

        if do_rescale:
            image = self.rescale(image=image, scale=rescale_factor, input_data_format=input_data_format)

        if do_normalize:
            image = self.normalize(image=image, mean=image_mean, std=image_std, input_data_format=input_data_format)

        return image

    def _preprocess_image(
        self,
        image: ImageInput,
        do_resize: bool = None,
        size: Dict[str, int] = None,
        resample: PILImageResampling = None,
        do_rescale: bool = None,
        rescale_factor: float = None,
        do_normalize: bool = None,
        image_mean: Optional[Union[float, List[float]]] = None,
        image_std: Optional[Union[float, List[float]]] = None,
        data_format: Optional[Union[str, ChannelDimension]] = None,
        input_data_format: Optional[Union[str, ChannelDimension]] = None,
    ) -> np.ndarray:
        """Preprocesses a single image."""
        # All transformations expect numpy arrays.
        image = to_numpy_array(image)
        if is_scaled_image(image) and do_rescale:
            logger.warning_once(
                "It looks like you are trying to rescale already rescaled images. If the input"
                " images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again."
            )
        if input_data_format is None:
            input_data_format = infer_channel_dimension_format(image)
        image = self._preprocess(
            image=image,
            do_reduce_labels=False,
            do_resize=do_resize,
            size=size,
            resample=resample,
            do_rescale=do_rescale,
            rescale_factor=rescale_factor,
            do_normalize=do_normalize,
            image_mean=image_mean,
            image_std=image_std,
            input_data_format=input_data_format,
        )
        if data_format is not None:
            image = to_channel_dimension_format(image, data_format, input_channel_dim=input_data_format)
        return image

    def _preprocess_mask(
        self,
        segmentation_map: ImageInput,
        do_reduce_labels: bool = None,
        do_resize: bool = None,
        size: Dict[str, int] = None,
        input_data_format: Optional[Union[str, ChannelDimension]] = None,
    ) -> np.ndarray:
        """Preprocesses a single mask."""
        segmentation_map = to_numpy_array(segmentation_map)
        # Add channel dimension if missing - needed for certain transformations
        if segmentation_map.ndim == 2:
            added_channel_dim = True
            segmentation_map = segmentation_map[None, ...]
            input_data_format = ChannelDimension.FIRST
        else:
            added_channel_dim = False
            if input_data_format is None:
                input_data_format = infer_channel_dimension_format(segmentation_map, num_channels=1)
        # reduce zero label if needed
        segmentation_map = self._preprocess(
            image=segmentation_map,
            do_reduce_labels=do_reduce_labels,
            do_resize=do_resize,
            resample=PILImageResampling.NEAREST,
            size=size,
            do_rescale=False,
            do_normalize=False,
            input_data_format=input_data_format,
        )
        # Remove extra channel dimension if added for processing
        if added_channel_dim:
            segmentation_map = segmentation_map.squeeze(0)
        segmentation_map = segmentation_map.astype(np.int64)
        return segmentation_map

    def __call__(self, images, segmentation_maps=None, **kwargs):
        """
        Preprocesses a batch of images and optionally segmentation maps.

        Overrides the `__call__` method of the `Preprocessor` class so that both images and segmentation maps can be
        passed in as positional arguments.
        """
        return super().__call__(images, segmentation_maps=segmentation_maps, **kwargs)

    def preprocess(
        self,
        images: ImageInput,
        segmentation_maps: Optional[ImageInput] = None,
        do_resize: Optional[bool] = None,
        size: Optional[Dict[str, int]] = None,
        resample: PILImageResampling = None,
        do_rescale: Optional[bool] = None,
        rescale_factor: Optional[float] = None,
        do_normalize: Optional[bool] = None,
        image_mean: Optional[Union[float, List[float]]] = None,
        image_std: Optional[Union[float, List[float]]] = None,
        do_reduce_labels: Optional[bool] = None,
        return_tensors: Optional[Union[str, TensorType]] = None,
        data_format: ChannelDimension = ChannelDimension.FIRST,
        input_data_format: Optional[Union[str, ChannelDimension]] = None,
        **kwargs,
    ) -> PIL.Image.Image:
        """
        Preprocess an image or batch of images.

        Args:
            images (`ImageInput`):
                Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If
                passing in images with pixel values between 0 and 1, set `do_rescale=False`.
            segmentation_maps (`ImageInput`, *optional*):
                Segmentation map to preprocess.
            do_resize (`bool`, *optional*, defaults to `self.do_resize`):
                Whether to resize the image.
            size (`Dict[str, int]`, *optional*, defaults to `self.size`):
                Size of the image after `resize` is applied.
            resample (`int`, *optional*, defaults to `self.resample`):
                Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`, Only
                has an effect if `do_resize` is set to `True`.
            do_rescale (`bool`, *optional*, defaults to `self.do_rescale`):
                Whether to rescale the image values between [0 - 1].
            rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`):
                Rescale factor to rescale the image by if `do_rescale` is set to `True`.
            do_normalize (`bool`, *optional*, defaults to `self.do_normalize`):
                Whether to normalize the image.
            image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`):
                Image mean.
            image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`):
                Image standard deviation.
            do_reduce_labels (`bool`, *optional*, defaults to `self.do_reduce_labels`):
                Whether or not to reduce all label values of segmentation maps by 1. Usually used for datasets where 0
                is used for background, and background itself is not included in all classes of a dataset (e.g.
                ADE20k). The background label will be replaced by 255.
            return_tensors (`str` or `TensorType`, *optional*):
                The type of tensors to return. Can be one of:

                - Unset: Return a list of `np.ndarray`.
                - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`.
                - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `mindspore.Tensor`.
                - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`.
                - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`.
            data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`):
                The channel dimension format for the output image. Can be one of:

                - `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
                - `ChannelDimension.LAST`: image in (height, width, num_channels) format.
            input_data_format (`ChannelDimension` or `str`, *optional*):
                The channel dimension format for the input image. If unset, the channel dimension format is inferred
                from the input image. Can be one of:

                - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
                - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
                - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.
        """
        do_resize = do_resize if do_resize is not None else self.do_resize
        do_rescale = do_rescale if do_rescale is not None else self.do_rescale
        do_normalize = do_normalize if do_normalize is not None else self.do_normalize
        do_reduce_labels = do_reduce_labels if do_reduce_labels is not None else self.do_reduce_labels
        resample = resample if resample is not None else self.resample
        size = size if size is not None else self.size
        rescale_factor = rescale_factor if rescale_factor is not None else self.rescale_factor
        image_mean = image_mean if image_mean is not None else self.image_mean
        image_std = image_std if image_std is not None else self.image_std

        images = make_list_of_images(images)

        validate_kwargs(captured_kwargs=kwargs.keys(), valid_processor_keys=self._valid_processor_keys)

        if segmentation_maps is not None:
            segmentation_maps = make_list_of_images(segmentation_maps, expected_ndims=2)

        if not valid_images(images):
            raise ValueError(
                "Invalid image type. Must be of type PIL.Image.Image, numpy.ndarray, "
                "mindspore.Tensor, tf.Tensor or jax.ndarray."
            )
        validate_preprocess_arguments(
            do_rescale=do_rescale,
            rescale_factor=rescale_factor,
            do_normalize=do_normalize,
            image_mean=image_mean,
            image_std=image_std,
            do_resize=do_resize,
            size=size,
            resample=resample,
        )

        images = [
            self._preprocess_image(
                image=img,
                do_resize=do_resize,
                resample=resample,
                size=size,
                do_rescale=do_rescale,
                rescale_factor=rescale_factor,
                do_normalize=do_normalize,
                image_mean=image_mean,
                image_std=image_std,
                data_format=data_format,
                input_data_format=input_data_format,
            )
            for img in images
        ]

        data = {"pixel_values": images}

        if segmentation_maps is not None:
            segmentation_maps = [
                self._preprocess_mask(
                    segmentation_map=segmentation_map,
                    do_reduce_labels=do_reduce_labels,
                    do_resize=do_resize,
                    size=size,
                    input_data_format=input_data_format,
                )
                for segmentation_map in segmentation_maps
            ]
            data["labels"] = segmentation_maps

        return BatchFeature(data=data, tensor_type=return_tensors)

    # Copied from transformers.models.beit.image_processing_beit.BeitImageProcessor.post_process_semantic_segmentation with Beit->Segformer
    def post_process_semantic_segmentation(self, outputs, target_sizes: List[Tuple] = None):
        """
        Converts the output of [`SegformerForSemanticSegmentation`] into semantic segmentation maps.
        Only supports PyTorch.

        Args:
            outputs ([`SegformerForSemanticSegmentation`]):
                Raw outputs of the model.
            target_sizes (`List[Tuple]` of length `batch_size`, *optional*):
                List of tuples corresponding to the requested final size (height, width) of each prediction. If unset,
                predictions will not be resized.

        Returns:
            semantic_segmentation: `List[mindspore.Tensor]` of length `batch_size`, where each item is a semantic
                segmentation map of shape (height, width) corresponding to the target_sizes entry (if `target_sizes` is
                specified). Each entry of each `mindspore.Tensor` correspond to a semantic class id.
        """
        # TODO: add support for other frameworks
        logits = outputs.logits

        # Resize logits and compute semantic segmentation maps
        if target_sizes is not None:
            if len(logits) != len(target_sizes):
                raise ValueError(
                    "Make sure that you pass in as many target sizes as the batch dimension of the logits"
                )

            if is_mindspore_tensor(target_sizes):
                target_sizes = target_sizes.numpy()

            semantic_segmentation = []

            for idx in range(len(logits)):
                resized_logits = ops.interpolate(
                    logits[idx].unsqueeze(dim=0), size=target_sizes[idx], mode="bilinear", align_corners=False
                )
                semantic_map = resized_logits[0].argmax(axis=0)
                semantic_segmentation.append(semantic_map)
        else:
            semantic_segmentation = logits.argmax(axis=1)
            semantic_segmentation = [semantic_segmentation[i] for i in range(semantic_segmentation.shape[0])]

        return semantic_segmentation

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.__call__(images, segmentation_maps=None, **kwargs)

Preprocesses a batch of images and optionally segmentation maps.

Overrides the __call__ method of the Preprocessor class so that both images and segmentation maps can be passed in as positional arguments.

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def __call__(self, images, segmentation_maps=None, **kwargs):
    """
    Preprocesses a batch of images and optionally segmentation maps.

    Overrides the `__call__` method of the `Preprocessor` class so that both images and segmentation maps can be
    passed in as positional arguments.
    """
    return super().__call__(images, segmentation_maps=segmentation_maps, **kwargs)

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.__init__(do_resize=True, size=None, resample=PILImageResampling.BILINEAR, do_rescale=True, rescale_factor=1 / 255, do_normalize=True, image_mean=None, image_std=None, do_reduce_labels=False, **kwargs)

Initialize the SegformerImageProcessor.

This method initializes the SegformerImageProcessor object with the provided parameters.

PARAMETER	DESCRIPTION
self	The SegformerImageProcessor object.
do_resize	Whether to resize the image. Defaults to True. TYPE: bool DEFAULT: True
size	The desired height and width of the image. Defaults to {'height': 512, 'width': 512}. TYPE: Dict[str, int] DEFAULT: None
resample	The resampling method to use during resizing. Defaults to PILImageResampling.BILINEAR. TYPE: PILImageResampling DEFAULT: BILINEAR
do_rescale	Whether to rescale the image. Defaults to True. TYPE: bool DEFAULT: True
rescale_factor	The rescale factor to apply to the image. Defaults to 1 / 255. TYPE: Union[int, float] DEFAULT: 1 / 255
do_normalize	Whether to normalize the image. Defaults to True. TYPE: bool DEFAULT: True
image_mean	The mean values used for image normalization. Defaults to None, which uses IMAGENET_DEFAULT_MEAN. TYPE: Optional[Union[float, List[float]]] DEFAULT: None
image_std	The standard deviation values used for image normalization. Defaults to None, which uses IMAGENET_DEFAULT_STD. TYPE: Optional[Union[float, List[float]]] DEFAULT: None
do_reduce_labels	Whether to reduce the number of labels. Defaults to False. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
None	None

RAISES	DESCRIPTION
FutureWarning	If the 'reduce_labels' parameter is used. This parameter is deprecated and will be removed in a future version. Please use 'do_reduce_labels' instead.

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def __init__(
    self,
    do_resize: bool = True,
    size: Dict[str, int] = None,
    resample: PILImageResampling = PILImageResampling.BILINEAR,
    do_rescale: bool = True,
    rescale_factor: Union[int, float] = 1 / 255,
    do_normalize: bool = True,
    image_mean: Optional[Union[float, List[float]]] = None,
    image_std: Optional[Union[float, List[float]]] = None,
    do_reduce_labels: bool = False,
    **kwargs,
) -> None:
    """Initialize the SegformerImageProcessor.

    This method initializes the SegformerImageProcessor object with the provided parameters.

    Args:
        self: The SegformerImageProcessor object.
        do_resize (bool, optional): Whether to resize the image. Defaults to True.
        size (Dict[str, int], optional): The desired height and width of the image.
            Defaults to {'height': 512, 'width': 512}.
        resample (PILImageResampling, optional): The resampling method to use during resizing.
            Defaults to PILImageResampling.BILINEAR.
        do_rescale (bool, optional): Whether to rescale the image. Defaults to True.
        rescale_factor (Union[int, float], optional): The rescale factor to apply to the image. Defaults to 1 / 255.
        do_normalize (bool, optional): Whether to normalize the image. Defaults to True.
        image_mean (Optional[Union[float, List[float]]], optional): The mean values used for image normalization.
            Defaults to None, which uses IMAGENET_DEFAULT_MEAN.
        image_std (Optional[Union[float, List[float]]], optional):
            The standard deviation values used for image normalization. Defaults to None, which uses
            IMAGENET_DEFAULT_STD.
        do_reduce_labels (bool, optional): Whether to reduce the number of labels. Defaults to False.

    Returns:
        None

    Raises:
        FutureWarning: If the 'reduce_labels' parameter is used. This parameter is deprecated and will be removed
            in a future version. Please use 'do_reduce_labels' instead.
    """
    if "reduce_labels" in kwargs:
        warnings.warn(
            "The `reduce_labels` parameter is deprecated and will be removed in a future version. Please use "
            "`do_reduce_labels` instead.",
            FutureWarning,
        )
        do_reduce_labels = kwargs.pop("reduce_labels")

    super().__init__(**kwargs)
    size = size if size is not None else {"height": 512, "width": 512}
    size = get_size_dict(size)
    self.do_resize = do_resize
    self.size = size
    self.resample = resample
    self.do_rescale = do_rescale
    self.rescale_factor = rescale_factor
    self.do_normalize = do_normalize
    self.image_mean = image_mean if image_mean is not None else IMAGENET_DEFAULT_MEAN
    self.image_std = image_std if image_std is not None else IMAGENET_DEFAULT_STD
    self.do_reduce_labels = do_reduce_labels
    self._valid_processor_keys = [
        "images",
        "segmentation_maps",
        "do_resize",
        "size",
        "resample",
        "do_rescale",
        "rescale_factor",
        "do_normalize",
        "image_mean",
        "image_std",
        "do_reduce_labels",
        "return_tensors",
        "data_format",
        "input_data_format",
    ]

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.from_dict(image_processor_dict, **kwargs) classmethod

Overrides the from_dict method from the base class to make sure do_reduce_labels is updated if image processor is created using from_dict and kwargs e.g. SegformerImageProcessor.from_pretrained(checkpoint, reduce_labels=True)

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

@classmethod
def from_dict(cls, image_processor_dict: Dict[str, Any], **kwargs):
    """
    Overrides the `from_dict` method from the base class to make sure `do_reduce_labels` is updated if image
    processor is created using from_dict and kwargs e.g. `SegformerImageProcessor.from_pretrained(checkpoint,
    reduce_labels=True)`
    """
    image_processor_dict = image_processor_dict.copy()
    if "reduce_labels" in kwargs:
        image_processor_dict["reduce_labels"] = kwargs.pop("reduce_labels")
    return super().from_dict(image_processor_dict, **kwargs)

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.post_process_semantic_segmentation(outputs, target_sizes=None)

Converts the output of [SegformerForSemanticSegmentation] into semantic segmentation maps. Only supports PyTorch.

PARAMETER	DESCRIPTION
outputs	Raw outputs of the model. TYPE: [`SegformerForSemanticSegmentation`]
target_sizes	List of tuples corresponding to the requested final size (height, width) of each prediction. If unset, predictions will not be resized. TYPE: `List[Tuple]` of length `batch_size`, *optional* DEFAULT: None

RETURNS	DESCRIPTION
semantic_segmentation	List[mindspore.Tensor] of length batch_size, where each item is a semantic segmentation map of shape (height, width) corresponding to the target_sizes entry (if target_sizes is specified). Each entry of each mindspore.Tensor correspond to a semantic class id.

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def post_process_semantic_segmentation(self, outputs, target_sizes: List[Tuple] = None):
    """
    Converts the output of [`SegformerForSemanticSegmentation`] into semantic segmentation maps.
    Only supports PyTorch.

    Args:
        outputs ([`SegformerForSemanticSegmentation`]):
            Raw outputs of the model.
        target_sizes (`List[Tuple]` of length `batch_size`, *optional*):
            List of tuples corresponding to the requested final size (height, width) of each prediction. If unset,
            predictions will not be resized.

    Returns:
        semantic_segmentation: `List[mindspore.Tensor]` of length `batch_size`, where each item is a semantic
            segmentation map of shape (height, width) corresponding to the target_sizes entry (if `target_sizes` is
            specified). Each entry of each `mindspore.Tensor` correspond to a semantic class id.
    """
    # TODO: add support for other frameworks
    logits = outputs.logits

    # Resize logits and compute semantic segmentation maps
    if target_sizes is not None:
        if len(logits) != len(target_sizes):
            raise ValueError(
                "Make sure that you pass in as many target sizes as the batch dimension of the logits"
            )

        if is_mindspore_tensor(target_sizes):
            target_sizes = target_sizes.numpy()

        semantic_segmentation = []

        for idx in range(len(logits)):
            resized_logits = ops.interpolate(
                logits[idx].unsqueeze(dim=0), size=target_sizes[idx], mode="bilinear", align_corners=False
            )
            semantic_map = resized_logits[0].argmax(axis=0)
            semantic_segmentation.append(semantic_map)
    else:
        semantic_segmentation = logits.argmax(axis=1)
        semantic_segmentation = [semantic_segmentation[i] for i in range(semantic_segmentation.shape[0])]

    return semantic_segmentation

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.preprocess(images, segmentation_maps=None, do_resize=None, size=None, resample=None, do_rescale=None, rescale_factor=None, do_normalize=None, image_mean=None, image_std=None, do_reduce_labels=None, return_tensors=None, data_format=ChannelDimension.FIRST, input_data_format=None, **kwargs)

Preprocess an image or batch of images.

PARAMETER	DESCRIPTION
images	Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set do_rescale=False. TYPE: `ImageInput`
segmentation_maps	Segmentation map to preprocess. TYPE: `ImageInput`, *optional* DEFAULT: None
do_resize	Whether to resize the image. TYPE: `bool`, *optional*, defaults to `self.do_resize` DEFAULT: None
size	Size of the image after resize is applied. TYPE: `Dict[str, int]`, *optional*, defaults to `self.size` DEFAULT: None
resample	Resampling filter to use if resizing the image. This can be one of the enum PILImageResampling, Only has an effect if do_resize is set to True. TYPE: `int`, *optional*, defaults to `self.resample` DEFAULT: None
do_rescale	Whether to rescale the image values between [0 - 1]. TYPE: `bool`, *optional*, defaults to `self.do_rescale` DEFAULT: None
rescale_factor	Rescale factor to rescale the image by if do_rescale is set to True. TYPE: `float`, *optional*, defaults to `self.rescale_factor` DEFAULT: None
do_normalize	Whether to normalize the image. TYPE: `bool`, *optional*, defaults to `self.do_normalize` DEFAULT: None
image_mean	Image mean. TYPE: `float` or `List[float]`, *optional*, defaults to `self.image_mean` DEFAULT: None
image_std	Image standard deviation. TYPE: `float` or `List[float]`, *optional*, defaults to `self.image_std` DEFAULT: None
do_reduce_labels	Whether or not to reduce all label values of segmentation maps by 1. Usually used for datasets where 0 is used for background, and background itself is not included in all classes of a dataset (e.g. ADE20k). The background label will be replaced by 255. TYPE: `bool`, *optional*, defaults to `self.do_reduce_labels` DEFAULT: None
return_tensors	The type of tensors to return. Can be one of: Unset: Return a list of np.ndarray. TensorType.TENSORFLOW or 'tf': Return a batch of type tf.Tensor. TensorType.PYTORCH or 'pt': Return a batch of type mindspore.Tensor. TensorType.NUMPY or 'np': Return a batch of type np.ndarray. TensorType.JAX or 'jax': Return a batch of type jax.numpy.ndarray. TYPE: `str` or `TensorType`, *optional* DEFAULT: None
data_format	The channel dimension format for the output image. Can be one of: ChannelDimension.FIRST: image in (num_channels, height, width) format. ChannelDimension.LAST: image in (height, width, num_channels) format. TYPE: `ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST` DEFAULT: FIRST
input_data_format	The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: "channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format. "channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format. "none" or ChannelDimension.NONE: image in (height, width) format. TYPE: `ChannelDimension` or `str`, *optional* DEFAULT: None

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def preprocess(
    self,
    images: ImageInput,
    segmentation_maps: Optional[ImageInput] = None,
    do_resize: Optional[bool] = None,
    size: Optional[Dict[str, int]] = None,
    resample: PILImageResampling = None,
    do_rescale: Optional[bool] = None,
    rescale_factor: Optional[float] = None,
    do_normalize: Optional[bool] = None,
    image_mean: Optional[Union[float, List[float]]] = None,
    image_std: Optional[Union[float, List[float]]] = None,
    do_reduce_labels: Optional[bool] = None,
    return_tensors: Optional[Union[str, TensorType]] = None,
    data_format: ChannelDimension = ChannelDimension.FIRST,
    input_data_format: Optional[Union[str, ChannelDimension]] = None,
    **kwargs,
) -> PIL.Image.Image:
    """
    Preprocess an image or batch of images.

    Args:
        images (`ImageInput`):
            Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If
            passing in images with pixel values between 0 and 1, set `do_rescale=False`.
        segmentation_maps (`ImageInput`, *optional*):
            Segmentation map to preprocess.
        do_resize (`bool`, *optional*, defaults to `self.do_resize`):
            Whether to resize the image.
        size (`Dict[str, int]`, *optional*, defaults to `self.size`):
            Size of the image after `resize` is applied.
        resample (`int`, *optional*, defaults to `self.resample`):
            Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`, Only
            has an effect if `do_resize` is set to `True`.
        do_rescale (`bool`, *optional*, defaults to `self.do_rescale`):
            Whether to rescale the image values between [0 - 1].
        rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`):
            Rescale factor to rescale the image by if `do_rescale` is set to `True`.
        do_normalize (`bool`, *optional*, defaults to `self.do_normalize`):
            Whether to normalize the image.
        image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`):
            Image mean.
        image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`):
            Image standard deviation.
        do_reduce_labels (`bool`, *optional*, defaults to `self.do_reduce_labels`):
            Whether or not to reduce all label values of segmentation maps by 1. Usually used for datasets where 0
            is used for background, and background itself is not included in all classes of a dataset (e.g.
            ADE20k). The background label will be replaced by 255.
        return_tensors (`str` or `TensorType`, *optional*):
            The type of tensors to return. Can be one of:

            - Unset: Return a list of `np.ndarray`.
            - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`.
            - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `mindspore.Tensor`.
            - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`.
            - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`.
        data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`):
            The channel dimension format for the output image. Can be one of:

            - `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
            - `ChannelDimension.LAST`: image in (height, width, num_channels) format.
        input_data_format (`ChannelDimension` or `str`, *optional*):
            The channel dimension format for the input image. If unset, the channel dimension format is inferred
            from the input image. Can be one of:

            - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
            - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
            - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.
    """
    do_resize = do_resize if do_resize is not None else self.do_resize
    do_rescale = do_rescale if do_rescale is not None else self.do_rescale
    do_normalize = do_normalize if do_normalize is not None else self.do_normalize
    do_reduce_labels = do_reduce_labels if do_reduce_labels is not None else self.do_reduce_labels
    resample = resample if resample is not None else self.resample
    size = size if size is not None else self.size
    rescale_factor = rescale_factor if rescale_factor is not None else self.rescale_factor
    image_mean = image_mean if image_mean is not None else self.image_mean
    image_std = image_std if image_std is not None else self.image_std

    images = make_list_of_images(images)

    validate_kwargs(captured_kwargs=kwargs.keys(), valid_processor_keys=self._valid_processor_keys)

    if segmentation_maps is not None:
        segmentation_maps = make_list_of_images(segmentation_maps, expected_ndims=2)

    if not valid_images(images):
        raise ValueError(
            "Invalid image type. Must be of type PIL.Image.Image, numpy.ndarray, "
            "mindspore.Tensor, tf.Tensor or jax.ndarray."
        )
    validate_preprocess_arguments(
        do_rescale=do_rescale,
        rescale_factor=rescale_factor,
        do_normalize=do_normalize,
        image_mean=image_mean,
        image_std=image_std,
        do_resize=do_resize,
        size=size,
        resample=resample,
    )

    images = [
        self._preprocess_image(
            image=img,
            do_resize=do_resize,
            resample=resample,
            size=size,
            do_rescale=do_rescale,
            rescale_factor=rescale_factor,
            do_normalize=do_normalize,
            image_mean=image_mean,
            image_std=image_std,
            data_format=data_format,
            input_data_format=input_data_format,
        )
        for img in images
    ]

    data = {"pixel_values": images}

    if segmentation_maps is not None:
        segmentation_maps = [
            self._preprocess_mask(
                segmentation_map=segmentation_map,
                do_reduce_labels=do_reduce_labels,
                do_resize=do_resize,
                size=size,
                input_data_format=input_data_format,
            )
            for segmentation_map in segmentation_maps
        ]
        data["labels"] = segmentation_maps

    return BatchFeature(data=data, tensor_type=return_tensors)

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.reduce_label(label)

Reduces the label values in the input image for Segformer image processing.

PARAMETER	DESCRIPTION
self	Instance of the SegformerImageProcessor class.
label	Input label image to be processed. It should be in a compatible format for processing. TYPE: ImageInput

RETURNS	DESCRIPTION
ndarray	np.ndarray: A NumPy array representing the processed label image with reduced values.

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def reduce_label(self, label: ImageInput) -> np.ndarray:
    """
    Reduces the label values in the input image for Segformer image processing.

    Args:
        self: Instance of the SegformerImageProcessor class.
        label (ImageInput): Input label image to be processed. It should be in a compatible format for processing.

    Returns:
        np.ndarray: A NumPy array representing the processed label image with reduced values.

    Raises:
        None.
    """
    label = to_numpy_array(label)
    # Avoid using underflow conversion
    label[label == 0] = 255
    label = label - 1
    label[label == 254] = 255
    return label

mindnlp.transformers.models.segformer.image_processing_segformer.SegformerImageProcessor.resize(image, size, resample=PILImageResampling.BILINEAR, data_format=None, input_data_format=None, **kwargs)

Resize an image to (size["height"], size["width"]).

PARAMETER	DESCRIPTION
image	Image to resize. TYPE: `np.ndarray`
size	Dictionary in the format {"height": int, "width": int} specifying the size of the output image. TYPE: `Dict[str, int]`
resample	PILImageResampling filter to use when resizing the image e.g. PILImageResampling.BILINEAR. TYPE: `PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR` DEFAULT: BILINEAR
data_format	The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: "channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format. "channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format. "none" or ChannelDimension.NONE: image in (height, width) format. TYPE: `ChannelDimension` or `str`, *optional* DEFAULT: None
input_data_format	The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: "channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format. "channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format. "none" or ChannelDimension.NONE: image in (height, width) format. TYPE: `ChannelDimension` or `str`, *optional* DEFAULT: None

RETURNS	DESCRIPTION
ndarray	np.ndarray: The resized image.

Source code in mindnlp/transformers/models/segformer/image_processing_segformer.py

def resize(
    self,
    image: np.ndarray,
    size: Dict[str, int],
    resample: PILImageResampling = PILImageResampling.BILINEAR,
    data_format: Optional[Union[str, ChannelDimension]] = None,
    input_data_format: Optional[Union[str, ChannelDimension]] = None,
    **kwargs,
) -> np.ndarray:
    """
    Resize an image to `(size["height"], size["width"])`.

    Args:
        image (`np.ndarray`):
            Image to resize.
        size (`Dict[str, int]`):
            Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image.
        resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`):
            `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`.
        data_format (`ChannelDimension` or `str`, *optional*):
            The channel dimension format for the output image. If unset, the channel dimension format of the input
            image is used. Can be one of:

            - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
            - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
            - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.
        input_data_format (`ChannelDimension` or `str`, *optional*):
            The channel dimension format for the input image. If unset, the channel dimension format is inferred
            from the input image. Can be one of:

            - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
            - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
            - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.

    Returns:
        `np.ndarray`: The resized image.
    """
    size = get_size_dict(size)
    if "height" not in size or "width" not in size:
        raise ValueError(f"The `size` dictionary must contain the keys `height` and `width`. Got {size.keys()}")
    output_size = (size["height"], size["width"])
    return resize(
        image,
        size=output_size,
        resample=resample,
        data_format=data_format,
        input_data_format=input_data_format,
        **kwargs,
    )

mindnlp.transformers.models.segformer.modeling_segformer

MindSpore SegFormer model.

mindnlp.transformers.models.segformer.modeling_segformer.SegFormerImageClassifierOutput dataclass

Bases: ImageClassifierOutput

Base class for outputs of image classification models.

PARAMETER	DESCRIPTION
loss	Classification (or regression if config.num_labels==1) loss. TYPE: `mindspore.Tensor` of shape `(1,)`, *optional*, returned when `labels` is provided DEFAULT: None
logits	Classification (or regression if config.num_labels==1) scores (before SoftMax). TYPE: `mindspore.Tensor` of shape `(batch_size, config.num_labels)` DEFAULT: None

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegFormerImageClassifierOutput(ImageClassifierOutput):
    """
    Base class for outputs of image classification models.

    Args:
        loss (`mindspore.Tensor` of shape `(1,)`, *optional*, returned when `labels` is provided):
            Classification (or regression if config.num_labels==1) loss.
        logits (`mindspore.Tensor` of shape `(batch_size, config.num_labels)`):
            Classification (or regression if config.num_labels==1) scores (before SoftMax).
        hidden_states (`tuple(mindspore.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed
            or when `config.output_hidden_states=True`):
            Tuple of `mindspore.Tensor` (one for the output of the embeddings, if the model has an embedding layer, +
            one for the output of each stage) of shape `(batch_size, num_channels, height, width)`. Hidden-states (also
            called feature maps) of the model at the output of each stage.
        attentions (`tuple(mindspore.Tensor)`, *optional*, returned when `output_attentions=True` is passed
            or when `config.output_attentions=True`):
            Tuple of `mindspore.Tensor` (one for each layer) of shape `(batch_size, num_heads, patch_size,
            sequence_length)`.

            Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
            heads.
    """
    loss: Optional[mindspore.Tensor] = None
    logits: mindspore.Tensor = None
    hidden_states: Optional[Tuple[mindspore.Tensor]] = None
    attentions: Optional[Tuple[mindspore.Tensor]] = None

mindnlp.transformers.models.segformer.modeling_segformer.SegformerAttention

Bases: Module

This class represents the attention mechanism used in the Segformer model. It inherits from the nn.Module class.

ATTRIBUTE	DESCRIPTION
self	Instance of the SegformerEfficientSelfAttention class that handles self-attention computations. TYPE: SegformerEfficientSelfAttention
output	Instance of the SegformerSelfOutput class that computes the final attention output. TYPE: SegformerSelfOutput
pruned_heads	A set that stores the indices of pruned attention heads. TYPE: set

METHOD	DESCRIPTION
__init__	Initializes the SegformerAttention class. Args: config (object): The configuration object. hidden_size (int): The size of the hidden layers. num_attention_heads (int): The number of attention heads. sequence_reduction_ratio (float): The sequence reduction ratio.
prune_heads	Prunes the specified attention heads from the model. Args: heads (list): A list of attention heads to be pruned.
forward	Constructs the attention mechanism. Args: hidden_states (object): The input hidden states. height (int): The height of the input. width (int): The width of the input. output_attentions (bool, optional): Whether to output the attention weights. Defaults to False. Returns: tuple: A tuple containing the attention output and any additional outputs.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerAttention(nn.Module):

    """
    This class represents the attention mechanism used in the Segformer model. It inherits from the `nn.Module` class.

    Attributes:
        self (SegformerEfficientSelfAttention): Instance of the SegformerEfficientSelfAttention class that handles
            self-attention computations.
        output (SegformerSelfOutput): Instance of the SegformerSelfOutput class that computes the final attention output.
        pruned_heads (set): A set that stores the indices of pruned attention heads.

    Methods:
        __init__(self, config, hidden_size, num_attention_heads, sequence_reduction_ratio):
            Initializes the SegformerAttention class.

            Args:

            - config (object): The configuration object.
            - hidden_size (int): The size of the hidden layers.
            - num_attention_heads (int): The number of attention heads.
            - sequence_reduction_ratio (float): The sequence reduction ratio.

        prune_heads(self, heads):
            Prunes the specified attention heads from the model.

            Args:

            - heads (list): A list of attention heads to be pruned.

        forward(self, hidden_states, height, width, output_attentions=False): Constructs the attention mechanism.

            Args:

            - hidden_states (object): The input hidden states.
            - height (int): The height of the input.
            - width (int): The width of the input.
            - output_attentions (bool, optional): Whether to output the attention weights. Defaults to False.

            Returns:

            - tuple: A tuple containing the attention output and any additional outputs.
    """
    def __init__(self, config, hidden_size, num_attention_heads, sequence_reduction_ratio):
        """
        Initializes the SegformerAttention class.

        Args:
            self: The instance of the class.
            config: A configuration object containing various parameters for the attention mechanism.
                - Type: object
                - Purpose: It provides the configuration settings for the attention mechanism.
                - Restrictions: Must be a valid configuration object.
            hidden_size: The size of the hidden layers in the attention mechanism.
                - Type: int
                - Purpose: It defines the dimensionality of the hidden layers.
                - Restrictions: Must be a positive integer.
            num_attention_heads: The number of attention heads to be used in the attention mechanism.
                - Type: int
                - Purpose: It determines the parallel attention computations.
                - Restrictions: Must be a positive integer.
            sequence_reduction_ratio: The ratio by which the input sequence length is reduced in the attention mechanism.
                - Type: int
                - Purpose: It controls the reduction of the input sequence length.
                - Restrictions: Must be a positive integer.

        Returns:
            None.

        Raises:
            None
        """
        super().__init__()
        self.self = SegformerEfficientSelfAttention(
            config=config,
            hidden_size=hidden_size,
            num_attention_heads=num_attention_heads,
            sequence_reduction_ratio=sequence_reduction_ratio,
        )
        self.output = SegformerSelfOutput(config, hidden_size=hidden_size)
        self.pruned_heads = set()

    def prune_heads(self, heads):
        """
        This method 'prune_heads' is defined in the class 'SegformerAttention' and is used to prune the attention heads
        and corresponding linear layers based on the provided 'heads' input.

        Args:
            self (object): The instance of the 'SegformerAttention' class.
            heads (list): A list containing the indices of attention heads to be pruned.
                The indices should be within the valid range of attention heads for the model.

        Returns:
            None.

        Raises:
            ValueError: If the length of the 'heads' list is 0, indicating no heads to be pruned.
            TypeError: If the 'heads' parameter is not provided as a list.
            IndexError: If the indices in the 'heads' list are out of range for the attention heads in the model.
        """
        if len(heads) == 0:
            return
        heads, index = find_pruneable_heads_and_indices(
            heads, self.self.num_attention_heads, self.self.attention_head_size, self.pruned_heads
        )

        # Prune linear layers
        self.self.query = prune_linear_layer(self.self.query, index)
        self.self.key = prune_linear_layer(self.self.key, index)
        self.self.value = prune_linear_layer(self.self.value, index)
        self.output.dense = prune_linear_layer(self.output.dense, index, dim=1)

        # Update hyper params and store pruned heads
        self.self.num_attention_heads = self.self.num_attention_heads - len(heads)
        self.self.all_head_size = self.self.attention_head_size * self.self.num_attention_heads
        self.pruned_heads = self.pruned_heads.union(heads)

    def forward(self, hidden_states, height, width, output_attentions=False):
        """
        Construct the attention output of the SegformerAttention module.

        Args:
            self (SegformerAttention): The instance of the SegformerAttention class.
            hidden_states (tensor): The input hidden states tensor of shape
                (batch_size, sequence_length, hidden_size).
            height (int): The height of the attention output.
            width (int): The width of the attention output.
            output_attentions (bool, optional): Whether to output attentions. Defaults to False.

        Returns:
            tuple: A tuple containing the attention output tensor of shape
                (batch_size, sequence_length, hidden_size),  and any additional outputs as returned by the
                self.self() method.

        Raises:
            None

        """
        self_outputs = self.self(hidden_states, height, width, output_attentions)

        attention_output = self.output(self_outputs[0], hidden_states)
        outputs = (attention_output,) + self_outputs[1:]  # add attentions if we output them
        return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerAttention.__init__(config, hidden_size, num_attention_heads, sequence_reduction_ratio)

Initializes the SegformerAttention class.

PARAMETER	DESCRIPTION
self	The instance of the class.
config	A configuration object containing various parameters for the attention mechanism. - Type: object - Purpose: It provides the configuration settings for the attention mechanism. - Restrictions: Must be a valid configuration object.
hidden_size	The size of the hidden layers in the attention mechanism. - Type: int - Purpose: It defines the dimensionality of the hidden layers. - Restrictions: Must be a positive integer.
num_attention_heads	The number of attention heads to be used in the attention mechanism. - Type: int - Purpose: It determines the parallel attention computations. - Restrictions: Must be a positive integer.
sequence_reduction_ratio	The ratio by which the input sequence length is reduced in the attention mechanism. - Type: int - Purpose: It controls the reduction of the input sequence length. - Restrictions: Must be a positive integer.

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config, hidden_size, num_attention_heads, sequence_reduction_ratio):
    """
    Initializes the SegformerAttention class.

    Args:
        self: The instance of the class.
        config: A configuration object containing various parameters for the attention mechanism.
            - Type: object
            - Purpose: It provides the configuration settings for the attention mechanism.
            - Restrictions: Must be a valid configuration object.
        hidden_size: The size of the hidden layers in the attention mechanism.
            - Type: int
            - Purpose: It defines the dimensionality of the hidden layers.
            - Restrictions: Must be a positive integer.
        num_attention_heads: The number of attention heads to be used in the attention mechanism.
            - Type: int
            - Purpose: It determines the parallel attention computations.
            - Restrictions: Must be a positive integer.
        sequence_reduction_ratio: The ratio by which the input sequence length is reduced in the attention mechanism.
            - Type: int
            - Purpose: It controls the reduction of the input sequence length.
            - Restrictions: Must be a positive integer.

    Returns:
        None.

    Raises:
        None
    """
    super().__init__()
    self.self = SegformerEfficientSelfAttention(
        config=config,
        hidden_size=hidden_size,
        num_attention_heads=num_attention_heads,
        sequence_reduction_ratio=sequence_reduction_ratio,
    )
    self.output = SegformerSelfOutput(config, hidden_size=hidden_size)
    self.pruned_heads = set()

mindnlp.transformers.models.segformer.modeling_segformer.SegformerAttention.forward(hidden_states, height, width, output_attentions=False)

Construct the attention output of the SegformerAttention module.

PARAMETER	DESCRIPTION
self	The instance of the SegformerAttention class. TYPE: SegformerAttention
hidden_states	The input hidden states tensor of shape (batch_size, sequence_length, hidden_size). TYPE: tensor
height	The height of the attention output. TYPE: int
width	The width of the attention output. TYPE: int
output_attentions	Whether to output attentions. Defaults to False. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
tuple	A tuple containing the attention output tensor of shape (batch_size, sequence_length, hidden_size), and any additional outputs as returned by the self.self() method.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states, height, width, output_attentions=False):
    """
    Construct the attention output of the SegformerAttention module.

    Args:
        self (SegformerAttention): The instance of the SegformerAttention class.
        hidden_states (tensor): The input hidden states tensor of shape
            (batch_size, sequence_length, hidden_size).
        height (int): The height of the attention output.
        width (int): The width of the attention output.
        output_attentions (bool, optional): Whether to output attentions. Defaults to False.

    Returns:
        tuple: A tuple containing the attention output tensor of shape
            (batch_size, sequence_length, hidden_size),  and any additional outputs as returned by the
            self.self() method.

    Raises:
        None

    """
    self_outputs = self.self(hidden_states, height, width, output_attentions)

    attention_output = self.output(self_outputs[0], hidden_states)
    outputs = (attention_output,) + self_outputs[1:]  # add attentions if we output them
    return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerAttention.prune_heads(heads)

This method 'prune_heads' is defined in the class 'SegformerAttention' and is used to prune the attention heads and corresponding linear layers based on the provided 'heads' input.

PARAMETER	DESCRIPTION
self	The instance of the 'SegformerAttention' class. TYPE: object
heads	A list containing the indices of attention heads to be pruned. The indices should be within the valid range of attention heads for the model. TYPE: list

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
ValueError	If the length of the 'heads' list is 0, indicating no heads to be pruned.
TypeError	If the 'heads' parameter is not provided as a list.
IndexError	If the indices in the 'heads' list are out of range for the attention heads in the model.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def prune_heads(self, heads):
    """
    This method 'prune_heads' is defined in the class 'SegformerAttention' and is used to prune the attention heads
    and corresponding linear layers based on the provided 'heads' input.

    Args:
        self (object): The instance of the 'SegformerAttention' class.
        heads (list): A list containing the indices of attention heads to be pruned.
            The indices should be within the valid range of attention heads for the model.

    Returns:
        None.

    Raises:
        ValueError: If the length of the 'heads' list is 0, indicating no heads to be pruned.
        TypeError: If the 'heads' parameter is not provided as a list.
        IndexError: If the indices in the 'heads' list are out of range for the attention heads in the model.
    """
    if len(heads) == 0:
        return
    heads, index = find_pruneable_heads_and_indices(
        heads, self.self.num_attention_heads, self.self.attention_head_size, self.pruned_heads
    )

    # Prune linear layers
    self.self.query = prune_linear_layer(self.self.query, index)
    self.self.key = prune_linear_layer(self.self.key, index)
    self.self.value = prune_linear_layer(self.self.value, index)
    self.output.dense = prune_linear_layer(self.output.dense, index, dim=1)

    # Update hyper params and store pruned heads
    self.self.num_attention_heads = self.self.num_attention_heads - len(heads)
    self.self.all_head_size = self.self.attention_head_size * self.self.num_attention_heads
    self.pruned_heads = self.pruned_heads.union(heads)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDWConv

Bases: Module

The SegformerDWConv class represents a depthwise separable convolutional layer for segmentation tasks. This class inherits from the nn.Module module.

ATTRIBUTE	DESCRIPTION
dim	The dimensionality of the input and output channels for the depthwise separable convolution. TYPE: int

METHOD	DESCRIPTION
__init__	Initializes the SegformerDWConv object with a specified dimensionality for input and output channels.
forward	Applies the depthwise separable convolution to the input hidden_states and returns the processed output.

Example

>>> # Create a SegformerDWConv object with default dimensionality
>>> seg_dwconv = SegformerDWConv()
...
>>> # Apply the depthwise separable convolution to a set of hidden states
>>> output = seg_dwconv.forward(hidden_states, height, width)

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerDWConv(nn.Module):

    """
    The SegformerDWConv class represents a depthwise separable convolutional layer for segmentation tasks.
    This class inherits from the nn.Module module.

    Attributes:
        dim (int): The dimensionality of the input and output channels for the depthwise separable convolution.

    Methods:
        __init__: Initializes the SegformerDWConv object with a specified dimensionality for input and output channels.
        forward: Applies the depthwise separable convolution to the input hidden_states and returns the processed output.

    Example:
        ```python
        >>> # Create a SegformerDWConv object with default dimensionality
        >>> seg_dwconv = SegformerDWConv()
        ...
        >>> # Apply the depthwise separable convolution to a set of hidden states
        >>> output = seg_dwconv.forward(hidden_states, height, width)
        ```
    """
    def __init__(self, dim=768):
        """
        Initializes a SegformerDWConv instance.

        Args:
            self: The instance of the SegformerDWConv class.
            dim (int): The dimension of the input and output channels. Defaults to 768.

        Returns:
            None.

        Raises:
            ValueError: If the provided dimension is not a positive integer.
            TypeError: If the provided dimension is not an integer.
            RuntimeError: If an error occurs during the initialization process.
        """
        super().__init__()
        self.dwconv = nn.Conv2d(dim, dim, 3, 1, pad_mode='pad', padding=1, bias=True, group=dim)

    def forward(self, hidden_states, height, width):
        """
        Constructs the SegformerDWConv.

        Args:
            self (SegformerDWConv): An instance of the SegformerDWConv class.
            hidden_states (torch.Tensor): A tensor of shape (batch_size, seq_len, num_channels)
                representing the hidden states.
            height (int): The desired height of the hidden states after transformation.
            width (int): The desired width of the hidden states after transformation.

        Returns:
            None.

        Raises:
            None.
        """
        batch_size, seq_len, num_channels = hidden_states.shape
        hidden_states = hidden_states.swapaxes(1, 2).view(batch_size, num_channels, height, width)
        hidden_states = self.dwconv(hidden_states)
        hidden_states = hidden_states.flatten(start_dim=2).swapaxes(1, 2)

        return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDWConv.__init__(dim=768)

Initializes a SegformerDWConv instance.

PARAMETER	DESCRIPTION
self	The instance of the SegformerDWConv class.
dim	The dimension of the input and output channels. Defaults to 768. TYPE: int DEFAULT: 768

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
ValueError	If the provided dimension is not a positive integer.
TypeError	If the provided dimension is not an integer.
RuntimeError	If an error occurs during the initialization process.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, dim=768):
    """
    Initializes a SegformerDWConv instance.

    Args:
        self: The instance of the SegformerDWConv class.
        dim (int): The dimension of the input and output channels. Defaults to 768.

    Returns:
        None.

    Raises:
        ValueError: If the provided dimension is not a positive integer.
        TypeError: If the provided dimension is not an integer.
        RuntimeError: If an error occurs during the initialization process.
    """
    super().__init__()
    self.dwconv = nn.Conv2d(dim, dim, 3, 1, pad_mode='pad', padding=1, bias=True, group=dim)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDWConv.forward(hidden_states, height, width)

Constructs the SegformerDWConv.

PARAMETER	DESCRIPTION
self	An instance of the SegformerDWConv class. TYPE: SegformerDWConv
hidden_states	A tensor of shape (batch_size, seq_len, num_channels) representing the hidden states. TYPE: Tensor
height	The desired height of the hidden states after transformation. TYPE: int
width	The desired width of the hidden states after transformation. TYPE: int

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states, height, width):
    """
    Constructs the SegformerDWConv.

    Args:
        self (SegformerDWConv): An instance of the SegformerDWConv class.
        hidden_states (torch.Tensor): A tensor of shape (batch_size, seq_len, num_channels)
            representing the hidden states.
        height (int): The desired height of the hidden states after transformation.
        width (int): The desired width of the hidden states after transformation.

    Returns:
        None.

    Raises:
        None.
    """
    batch_size, seq_len, num_channels = hidden_states.shape
    hidden_states = hidden_states.swapaxes(1, 2).view(batch_size, num_channels, height, width)
    hidden_states = self.dwconv(hidden_states)
    hidden_states = hidden_states.flatten(start_dim=2).swapaxes(1, 2)

    return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDecodeHead

Bases: SegformerPreTrainedModel

The SegformerDecodeHead class is a subclass of SegformerPreTrainedModel and represents the decoding head component of the Segformer model.

This class contains methods for forwarding the decoding head and generating logits for semantic segmentation.

ATTRIBUTE	DESCRIPTION
linear_c	A list of MLP (Multi-Layer Perceptron) modules for each encoder block. TYPE: ModuleList
linear_fuse	A convolutional layer used for fusing the hidden states of all encoder blocks. TYPE: Conv2d
batch_norm	A batch normalization layer applied to the fused hidden states. TYPE: BatchNorm2d
activation	An activation function applied to the hidden states. TYPE: ReLU
dropout	A dropout layer applied to the hidden states. TYPE: Dropout
classifier	A convolutional layer for generating the final logits. TYPE: Conv2d
config	The configuration object containing hyperparameters and settings for the SegformerDecodeHead.

METHOD	DESCRIPTION
forward	mindspore.Tensor) -> mindspore.Tensor: Constructs the decoding head and generates logits for semantic segmentation based on the given encoder hidden states. Args: encoder_hidden_states (mindspore.Tensor): A tensor containing the hidden states of the encoder blocks. Returns: mindspore.Tensor: The logits for semantic segmentation.

Note

• The SegformerDecodeHead class requires an instance of SegformerPreTrainedModel as its parent class.
• The decoding head consists of multiple MLP modules, a fusion layer, batch normalization, activation, dropout, and a final classifier.
• The forward method takes the encoder hidden states as input and performs the necessary computations to generate the logits.
• The SegformerDecodeHead class is designed to be used in conjunction with the Segformer model for semantic segmentation tasks.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerDecodeHead(SegformerPreTrainedModel):
    """
    The `SegformerDecodeHead` class is a subclass of `SegformerPreTrainedModel` and represents the decoding head
    component of the Segformer model.

    This class contains methods for forwarding the decoding head and generating logits for semantic segmentation.

    Attributes:
        linear_c (nn.ModuleList): A list of MLP (Multi-Layer Perceptron) modules for each encoder block.
        linear_fuse (nn.Conv2d): A convolutional layer used for fusing the hidden states of all encoder blocks.
        batch_norm (nn.BatchNorm2d): A batch normalization layer applied to the fused hidden states.
        activation (nn.ReLU): An activation function applied to the hidden states.
        dropout (nn.Dropout): A dropout layer applied to the hidden states.
        classifier (nn.Conv2d): A convolutional layer for generating the final logits.
        config: The configuration object containing hyperparameters and settings for the SegformerDecodeHead.

    Methods:
        forward(encoder_hidden_states: mindspore.Tensor) -> mindspore.Tensor:
            Constructs the decoding head and generates logits for semantic segmentation based on the given encoder
            hidden states.

            Args:

            - encoder_hidden_states (mindspore.Tensor): A tensor containing the hidden states of the encoder blocks.

            Returns:

            - mindspore.Tensor: The logits for semantic segmentation.

    Note:
        - The `SegformerDecodeHead` class requires an instance of `SegformerPreTrainedModel` as its parent class.
        - The decoding head consists of multiple MLP modules, a fusion layer, batch normalization, activation, dropout,
        and a final classifier.
        - The `forward` method takes the encoder hidden states as input and performs the necessary computations to
        generate the logits.
        - The `SegformerDecodeHead` class is designed to be used in conjunction with the Segformer model for semantic
        segmentation tasks.
    """
    def __init__(self, config):
        """
        Initializes the SegformerDecodeHead class.

        Args:
            self: The instance of the SegformerDecodeHead class.
            config: A dictionary containing the configuration parameters for the SegformerDecodeHead,
                including the following keys:

                - num_encoder_blocks (int): The number of encoder blocks.
                - hidden_sizes (list of int): The list of hidden sizes for each encoder block.
                - decoder_hidden_size (int): The size of the hidden layer in the decoder.
                - classifier_dropout_prob (float): The dropout probability for the classifier.
                - num_labels (int): The number of output labels.

        Returns:
            None.

        Raises:
            None.
        """
        super().__init__(config)
        # linear layers which will unify the channel dimension of each of the encoder blocks to the same config.decoder_hidden_size
        mlps = []
        for i in range(config.num_encoder_blocks):
            mlp = SegformerMLP(config, input_dim=config.hidden_sizes[i])
            mlps.append(mlp)
        self.linear_c = nn.ModuleList(mlps)

        # the following 3 layers implement the ConvModule of the original implementation
        self.linear_fuse = nn.Conv2d(
            in_channels=config.decoder_hidden_size * config.num_encoder_blocks,
            out_channels=config.decoder_hidden_size,
            kernel_size=1,
            bias=False,
            pad_mode='valid'
        )
        self.batch_norm = nn.BatchNorm2d(config.decoder_hidden_size)
        self.activation = nn.ReLU()

        self.dropout = nn.Dropout(p=config.classifier_dropout_prob)
        self.classifier = nn.Conv2d(config.decoder_hidden_size, config.num_labels, kernel_size=1, pad_mode='valid', bias=True)

        self.config = config

    def forward(self, encoder_hidden_states: mindspore.Tensor) -> mindspore.Tensor:
        '''
        Constructs the decode head for Segformer.

        Args:
            self (SegformerDecodeHead): The instance of the SegformerDecodeHead class.
            encoder_hidden_states (mindspore.Tensor): The hidden states from the encoder.
                It is a tensor representing the hidden states from the encoder with shape (N, C, H, W).
                N represents the batch size, C represents the number of channels, H represents the height,
                and W represents the width.

        Returns:
            mindspore.Tensor: A tensor representing the logits for the segmentation task with shape (N, C', H', W').
                N represents the batch size, C' represents the number of classes, H' represents the height,
                and W' represents the width.

        Raises:
            ValueError: If the reshape_last_stage configuration is False and the encoder_hidden_state has 3 dimensions.
            RuntimeError: If there is an issue with the linear fusion operation.
            RuntimeError: If there is an issue with the batch normalization operation.
            RuntimeError: If there is an issue with the activation operation.
            RuntimeError: If there is an issue with the dropout operation.
            RuntimeError: If there is an issue with the classifier operation.
        '''
        batch_size = encoder_hidden_states[-1].shape[0]

        all_hidden_states = ()
        for encoder_hidden_state, mlp in zip(encoder_hidden_states, self.linear_c):
            if self.config.reshape_last_stage is False and encoder_hidden_state.ndim == 3:
                height = width = int(math.sqrt(encoder_hidden_state.shape[-1]))
                encoder_hidden_state = (
                    encoder_hidden_state.reshape(batch_size, height, width, -1).permute(0, 3, 1, 2)
                )

            # unify channel dimension
            height, width = encoder_hidden_state.shape[2], encoder_hidden_state.shape[3]
            encoder_hidden_state = mlp(encoder_hidden_state)
            encoder_hidden_state = encoder_hidden_state.permute(0, 2, 1)
            encoder_hidden_state = encoder_hidden_state.reshape(batch_size, -1, height, width)
            # upsample
            encoder_hidden_state = ops.interpolate(
                encoder_hidden_state, size=encoder_hidden_states[0].shape[2:], mode="bilinear", align_corners=False
            )
            all_hidden_states += (encoder_hidden_state,)

        hidden_states = self.linear_fuse(ops.cat(all_hidden_states[::-1], axis=1))
        hidden_states = self.batch_norm(hidden_states)
        hidden_states = self.activation(hidden_states)
        hidden_states = self.dropout(hidden_states)

        # logits are of shape (batch_size, num_labels, height/4, width/4)
        logits = self.classifier(hidden_states)

        return logits

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDecodeHead.__init__(config)

Initializes the SegformerDecodeHead class.

PARAMETER	DESCRIPTION
self	The instance of the SegformerDecodeHead class.
config	A dictionary containing the configuration parameters for the SegformerDecodeHead, including the following keys: num_encoder_blocks (int): The number of encoder blocks. hidden_sizes (list of int): The list of hidden sizes for each encoder block. decoder_hidden_size (int): The size of the hidden layer in the decoder. classifier_dropout_prob (float): The dropout probability for the classifier. num_labels (int): The number of output labels.

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config):
    """
    Initializes the SegformerDecodeHead class.

    Args:
        self: The instance of the SegformerDecodeHead class.
        config: A dictionary containing the configuration parameters for the SegformerDecodeHead,
            including the following keys:

            - num_encoder_blocks (int): The number of encoder blocks.
            - hidden_sizes (list of int): The list of hidden sizes for each encoder block.
            - decoder_hidden_size (int): The size of the hidden layer in the decoder.
            - classifier_dropout_prob (float): The dropout probability for the classifier.
            - num_labels (int): The number of output labels.

    Returns:
        None.

    Raises:
        None.
    """
    super().__init__(config)
    # linear layers which will unify the channel dimension of each of the encoder blocks to the same config.decoder_hidden_size
    mlps = []
    for i in range(config.num_encoder_blocks):
        mlp = SegformerMLP(config, input_dim=config.hidden_sizes[i])
        mlps.append(mlp)
    self.linear_c = nn.ModuleList(mlps)

    # the following 3 layers implement the ConvModule of the original implementation
    self.linear_fuse = nn.Conv2d(
        in_channels=config.decoder_hidden_size * config.num_encoder_blocks,
        out_channels=config.decoder_hidden_size,
        kernel_size=1,
        bias=False,
        pad_mode='valid'
    )
    self.batch_norm = nn.BatchNorm2d(config.decoder_hidden_size)
    self.activation = nn.ReLU()

    self.dropout = nn.Dropout(p=config.classifier_dropout_prob)
    self.classifier = nn.Conv2d(config.decoder_hidden_size, config.num_labels, kernel_size=1, pad_mode='valid', bias=True)

    self.config = config

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDecodeHead.forward(encoder_hidden_states)

Constructs the decode head for Segformer.

PARAMETER	DESCRIPTION
self	The instance of the SegformerDecodeHead class. TYPE: SegformerDecodeHead
encoder_hidden_states	The hidden states from the encoder. It is a tensor representing the hidden states from the encoder with shape (N, C, H, W). N represents the batch size, C represents the number of channels, H represents the height, and W represents the width. TYPE: Tensor

RETURNS	DESCRIPTION
Tensor	mindspore.Tensor: A tensor representing the logits for the segmentation task with shape (N, C', H', W'). N represents the batch size, C' represents the number of classes, H' represents the height, and W' represents the width.

RAISES	DESCRIPTION
ValueError	If the reshape_last_stage configuration is False and the encoder_hidden_state has 3 dimensions.
RuntimeError	If there is an issue with the linear fusion operation.
RuntimeError	If there is an issue with the batch normalization operation.
RuntimeError	If there is an issue with the activation operation.
RuntimeError	If there is an issue with the dropout operation.
RuntimeError	If there is an issue with the classifier operation.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, encoder_hidden_states: mindspore.Tensor) -> mindspore.Tensor:
    '''
    Constructs the decode head for Segformer.

    Args:
        self (SegformerDecodeHead): The instance of the SegformerDecodeHead class.
        encoder_hidden_states (mindspore.Tensor): The hidden states from the encoder.
            It is a tensor representing the hidden states from the encoder with shape (N, C, H, W).
            N represents the batch size, C represents the number of channels, H represents the height,
            and W represents the width.

    Returns:
        mindspore.Tensor: A tensor representing the logits for the segmentation task with shape (N, C', H', W').
            N represents the batch size, C' represents the number of classes, H' represents the height,
            and W' represents the width.

    Raises:
        ValueError: If the reshape_last_stage configuration is False and the encoder_hidden_state has 3 dimensions.
        RuntimeError: If there is an issue with the linear fusion operation.
        RuntimeError: If there is an issue with the batch normalization operation.
        RuntimeError: If there is an issue with the activation operation.
        RuntimeError: If there is an issue with the dropout operation.
        RuntimeError: If there is an issue with the classifier operation.
    '''
    batch_size = encoder_hidden_states[-1].shape[0]

    all_hidden_states = ()
    for encoder_hidden_state, mlp in zip(encoder_hidden_states, self.linear_c):
        if self.config.reshape_last_stage is False and encoder_hidden_state.ndim == 3:
            height = width = int(math.sqrt(encoder_hidden_state.shape[-1]))
            encoder_hidden_state = (
                encoder_hidden_state.reshape(batch_size, height, width, -1).permute(0, 3, 1, 2)
            )

        # unify channel dimension
        height, width = encoder_hidden_state.shape[2], encoder_hidden_state.shape[3]
        encoder_hidden_state = mlp(encoder_hidden_state)
        encoder_hidden_state = encoder_hidden_state.permute(0, 2, 1)
        encoder_hidden_state = encoder_hidden_state.reshape(batch_size, -1, height, width)
        # upsample
        encoder_hidden_state = ops.interpolate(
            encoder_hidden_state, size=encoder_hidden_states[0].shape[2:], mode="bilinear", align_corners=False
        )
        all_hidden_states += (encoder_hidden_state,)

    hidden_states = self.linear_fuse(ops.cat(all_hidden_states[::-1], axis=1))
    hidden_states = self.batch_norm(hidden_states)
    hidden_states = self.activation(hidden_states)
    hidden_states = self.dropout(hidden_states)

    # logits are of shape (batch_size, num_labels, height/4, width/4)
    logits = self.classifier(hidden_states)

    return logits

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDropPath

Bases: Module

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerDropPath(nn.Module):
    """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks)."""
    def __init__(self, drop_prob: Optional[float] = None) -> None:
        """
        Initialize an instance of SegformerDropPath.

        Args:
            self: The instance of the SegformerDropPath class.
            drop_prob (Optional[float]): The probability of dropping a connection during training. 
                If None, no connections are dropped. Default is None.

        Returns:
            None.

        Raises:
            None.
        """
        super().__init__()
        self.drop_prob = drop_prob

    def forward(self, hidden_states: mindspore.Tensor) -> mindspore.Tensor:
        """
        Constructs a drop path operation on the hidden states.

        Args:
            self (SegformerDropPath): An instance of the SegformerDropPath class.
            hidden_states (mindspore.Tensor): The input hidden states to apply drop path on.

        Returns:
            mindspore.Tensor: The modified hidden states after applying drop path operation.

        Raises:
            TypeError: If the input hidden_states is not a mindspore.Tensor object.
            ValueError: If the drop_prob is not a valid probability value.

        Note:
            Drop path is a regularization technique used in training deep neural networks.
            It randomly sets a fraction of the hidden states to zero during training,
            which helps in reducing overfitting.
        """
        return drop_path(hidden_states, self.drop_prob, self.training)

    def extra_repr(self) -> str:
        """
        This method returns a string representation of the drop probability for a SegformerDropPath instance.

        Args:
            self (SegformerDropPath):
                The instance of SegformerDropPath for which the drop probability is being represented.

        Returns:
            str: A string representing the drop probability of the SegformerDropPath instance.

        Raises:
            None.
        """
        return "p={}".format(self.drop_prob)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDropPath.__init__(drop_prob=None)

Initialize an instance of SegformerDropPath.

PARAMETER	DESCRIPTION
self	The instance of the SegformerDropPath class.
drop_prob	The probability of dropping a connection during training. If None, no connections are dropped. Default is None. TYPE: Optional[float] DEFAULT: None

RETURNS	DESCRIPTION
None	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, drop_prob: Optional[float] = None) -> None:
    """
    Initialize an instance of SegformerDropPath.

    Args:
        self: The instance of the SegformerDropPath class.
        drop_prob (Optional[float]): The probability of dropping a connection during training. 
            If None, no connections are dropped. Default is None.

    Returns:
        None.

    Raises:
        None.
    """
    super().__init__()
    self.drop_prob = drop_prob

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDropPath.extra_repr()

This method returns a string representation of the drop probability for a SegformerDropPath instance.

PARAMETER	DESCRIPTION
self	The instance of SegformerDropPath for which the drop probability is being represented. TYPE: SegformerDropPath

RETURNS	DESCRIPTION
str	A string representing the drop probability of the SegformerDropPath instance. TYPE: str

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def extra_repr(self) -> str:
    """
    This method returns a string representation of the drop probability for a SegformerDropPath instance.

    Args:
        self (SegformerDropPath):
            The instance of SegformerDropPath for which the drop probability is being represented.

    Returns:
        str: A string representing the drop probability of the SegformerDropPath instance.

    Raises:
        None.
    """
    return "p={}".format(self.drop_prob)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerDropPath.forward(hidden_states)

Constructs a drop path operation on the hidden states.

PARAMETER	DESCRIPTION
self	An instance of the SegformerDropPath class. TYPE: SegformerDropPath
hidden_states	The input hidden states to apply drop path on. TYPE: Tensor

RETURNS	DESCRIPTION
Tensor	mindspore.Tensor: The modified hidden states after applying drop path operation.

RAISES	DESCRIPTION
TypeError	If the input hidden_states is not a mindspore.Tensor object.
ValueError	If the drop_prob is not a valid probability value.

Note

Drop path is a regularization technique used in training deep neural networks. It randomly sets a fraction of the hidden states to zero during training, which helps in reducing overfitting.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states: mindspore.Tensor) -> mindspore.Tensor:
    """
    Constructs a drop path operation on the hidden states.

    Args:
        self (SegformerDropPath): An instance of the SegformerDropPath class.
        hidden_states (mindspore.Tensor): The input hidden states to apply drop path on.

    Returns:
        mindspore.Tensor: The modified hidden states after applying drop path operation.

    Raises:
        TypeError: If the input hidden_states is not a mindspore.Tensor object.
        ValueError: If the drop_prob is not a valid probability value.

    Note:
        Drop path is a regularization technique used in training deep neural networks.
        It randomly sets a fraction of the hidden states to zero during training,
        which helps in reducing overfitting.
    """
    return drop_path(hidden_states, self.drop_prob, self.training)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEfficientSelfAttention

Bases: Module

SegFormer's efficient self-attention mechanism. Employs the sequence reduction process introduced in the PvT paper.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerEfficientSelfAttention(nn.Module):
    """SegFormer's efficient self-attention mechanism. Employs the sequence reduction process introduced in the [PvT
    paper](https://arxiv.org/abs/2102.12122)."""
    def __init__(self, config, hidden_size, num_attention_heads, sequence_reduction_ratio):
        """Initializes an instance of the SegformerEfficientSelfAttention class.

        Args:
            self: The instance of the class.
            config: Configuration object containing various settings.
            hidden_size (int): The size of the hidden states.
            num_attention_heads (int): The number of attention heads.
            sequence_reduction_ratio (int): The ratio by which the sequence length is reduced.

        Returns:
            None

        Raises:
            ValueError: If the hidden_size is not a multiple of the num_attention_heads.

        """
        super().__init__()
        self.hidden_size = hidden_size
        self.num_attention_heads = num_attention_heads

        if self.hidden_size % self.num_attention_heads != 0:
            raise ValueError(
                f"The hidden size ({self.hidden_size}) is not a multiple of the number of attention "
                f"heads ({self.num_attention_heads})"
            )

        self.attention_head_size = int(self.hidden_size / self.num_attention_heads)
        self.all_head_size = self.num_attention_heads * self.attention_head_size

        self.query = nn.Linear(self.hidden_size, self.all_head_size)
        self.key = nn.Linear(self.hidden_size, self.all_head_size)
        self.value = nn.Linear(self.hidden_size, self.all_head_size)

        self.dropout = nn.Dropout(p=config.attention_probs_dropout_prob)

        self.sr_ratio = sequence_reduction_ratio
        if sequence_reduction_ratio > 1:
            self.sr = nn.Conv2d(
                hidden_size, hidden_size, kernel_size=sequence_reduction_ratio, stride=sequence_reduction_ratio, pad_mode='valid', bias=True
            )
            self.layer_norm = nn.LayerNorm(hidden_size)

    def swapaxes_for_scores(self, hidden_states):
        """
        Swaps axes and reshapes the input tensor for calculating attention scores in the
        SegformerEfficientSelfAttention class.

        Args:
            self (SegformerEfficientSelfAttention): An instance of the SegformerEfficientSelfAttention class.
            hidden_states (torch.Tensor): A tensor representing the hidden states. It should have a shape of
                (batch_size, sequence_length, hidden_size).

        Returns:
            torch.Tensor: A tensor representing the reshaped hidden states. The shape of the tensor will be
                (batch_size, num_attention_heads, sequence_length, attention_head_size).

        Raises:
            None.
        """
        new_shape = hidden_states.shape[:-1] + (self.num_attention_heads, self.attention_head_size)
        hidden_states = hidden_states.view(new_shape)
        return hidden_states.permute(0, 2, 1, 3)

    def forward(
        self,
        hidden_states,
        height,
        width,
        output_attentions=False,
    ):
        """
        Constructs the self-attention mechanism for the SegformerEfficientSelfAttention class.

        Args:
            self (object): The instance of the SegformerEfficientSelfAttention class.
            hidden_states (tensor): The input tensor representing the hidden states.
                Shape (batch_size, seq_len, num_channels).
            height (int): The height of the input tensor.
            width (int): The width of the input tensor.
            output_attentions (bool, optional): Flag indicating whether to output attentions. Defaults to False.

        Returns:
            tuple: A tuple containing the context layer tensor and attention probabilities tensor if output_attentions
                is True, otherwise only the context layer tensor.

        Raises:
            ValueError: If the shape of the hidden_states tensor is not compatible.
            TypeError: If the input parameters are not of the expected types.
            RuntimeError: If an error occurs during the computation.
        """
        query_layer = self.swapaxes_for_scores(self.query(hidden_states))

        if self.sr_ratio > 1:
            batch_size, seq_len, num_channels = hidden_states.shape
            # Reshape to (batch_size, num_channels, height, width)
            hidden_states = hidden_states.permute(0, 2, 1).reshape(batch_size, num_channels, height, width)
            # Apply sequence reduction
            hidden_states = self.sr(hidden_states)
            # Reshape back to (batch_size, seq_len, num_channels)
            hidden_states = hidden_states.reshape(batch_size, num_channels, -1).permute(0, 2, 1)
            hidden_states = self.layer_norm(hidden_states)

        key_layer = self.swapaxes_for_scores(self.key(hidden_states))
        value_layer = self.swapaxes_for_scores(self.value(hidden_states))

        # Take the dot product between "query" and "key" to get the raw attention scores.
        attention_scores = ops.matmul(query_layer, key_layer.swapaxes(-1, -2))

        attention_scores = attention_scores / math.sqrt(self.attention_head_size)

        # Normalize the attention scores to probabilities.
        attention_probs = ops.softmax(attention_scores, axis=-1)

        # This is actually dropping out entire tokens to attend to, which might
        # seem a bit unusual, but is taken from the original Transformer paper.
        attention_probs = self.dropout(attention_probs)

        context_layer = ops.matmul(attention_probs, value_layer)

        context_layer = context_layer.permute(0, 2, 1, 3)
        new_context_layer_shape = context_layer.shape[:-2] + (self.all_head_size,)
        context_layer = context_layer.view(new_context_layer_shape)

        outputs = (context_layer, attention_probs) if output_attentions else (context_layer,)

        return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEfficientSelfAttention.__init__(config, hidden_size, num_attention_heads, sequence_reduction_ratio)

Initializes an instance of the SegformerEfficientSelfAttention class.

PARAMETER	DESCRIPTION
self	The instance of the class.
config	Configuration object containing various settings.
hidden_size	The size of the hidden states. TYPE: int
num_attention_heads	The number of attention heads. TYPE: int
sequence_reduction_ratio	The ratio by which the sequence length is reduced. TYPE: int

RETURNS	DESCRIPTION
	None

RAISES	DESCRIPTION
ValueError	If the hidden_size is not a multiple of the num_attention_heads.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config, hidden_size, num_attention_heads, sequence_reduction_ratio):
    """Initializes an instance of the SegformerEfficientSelfAttention class.

    Args:
        self: The instance of the class.
        config: Configuration object containing various settings.
        hidden_size (int): The size of the hidden states.
        num_attention_heads (int): The number of attention heads.
        sequence_reduction_ratio (int): The ratio by which the sequence length is reduced.

    Returns:
        None

    Raises:
        ValueError: If the hidden_size is not a multiple of the num_attention_heads.

    """
    super().__init__()
    self.hidden_size = hidden_size
    self.num_attention_heads = num_attention_heads

    if self.hidden_size % self.num_attention_heads != 0:
        raise ValueError(
            f"The hidden size ({self.hidden_size}) is not a multiple of the number of attention "
            f"heads ({self.num_attention_heads})"
        )

    self.attention_head_size = int(self.hidden_size / self.num_attention_heads)
    self.all_head_size = self.num_attention_heads * self.attention_head_size

    self.query = nn.Linear(self.hidden_size, self.all_head_size)
    self.key = nn.Linear(self.hidden_size, self.all_head_size)
    self.value = nn.Linear(self.hidden_size, self.all_head_size)

    self.dropout = nn.Dropout(p=config.attention_probs_dropout_prob)

    self.sr_ratio = sequence_reduction_ratio
    if sequence_reduction_ratio > 1:
        self.sr = nn.Conv2d(
            hidden_size, hidden_size, kernel_size=sequence_reduction_ratio, stride=sequence_reduction_ratio, pad_mode='valid', bias=True
        )
        self.layer_norm = nn.LayerNorm(hidden_size)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEfficientSelfAttention.forward(hidden_states, height, width, output_attentions=False)

Constructs the self-attention mechanism for the SegformerEfficientSelfAttention class.

PARAMETER	DESCRIPTION
self	The instance of the SegformerEfficientSelfAttention class. TYPE: object
hidden_states	The input tensor representing the hidden states. Shape (batch_size, seq_len, num_channels). TYPE: tensor
height	The height of the input tensor. TYPE: int
width	The width of the input tensor. TYPE: int
output_attentions	Flag indicating whether to output attentions. Defaults to False. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
tuple	A tuple containing the context layer tensor and attention probabilities tensor if output_attentions is True, otherwise only the context layer tensor.

RAISES	DESCRIPTION
ValueError	If the shape of the hidden_states tensor is not compatible.
TypeError	If the input parameters are not of the expected types.
RuntimeError	If an error occurs during the computation.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(
    self,
    hidden_states,
    height,
    width,
    output_attentions=False,
):
    """
    Constructs the self-attention mechanism for the SegformerEfficientSelfAttention class.

    Args:
        self (object): The instance of the SegformerEfficientSelfAttention class.
        hidden_states (tensor): The input tensor representing the hidden states.
            Shape (batch_size, seq_len, num_channels).
        height (int): The height of the input tensor.
        width (int): The width of the input tensor.
        output_attentions (bool, optional): Flag indicating whether to output attentions. Defaults to False.

    Returns:
        tuple: A tuple containing the context layer tensor and attention probabilities tensor if output_attentions
            is True, otherwise only the context layer tensor.

    Raises:
        ValueError: If the shape of the hidden_states tensor is not compatible.
        TypeError: If the input parameters are not of the expected types.
        RuntimeError: If an error occurs during the computation.
    """
    query_layer = self.swapaxes_for_scores(self.query(hidden_states))

    if self.sr_ratio > 1:
        batch_size, seq_len, num_channels = hidden_states.shape
        # Reshape to (batch_size, num_channels, height, width)
        hidden_states = hidden_states.permute(0, 2, 1).reshape(batch_size, num_channels, height, width)
        # Apply sequence reduction
        hidden_states = self.sr(hidden_states)
        # Reshape back to (batch_size, seq_len, num_channels)
        hidden_states = hidden_states.reshape(batch_size, num_channels, -1).permute(0, 2, 1)
        hidden_states = self.layer_norm(hidden_states)

    key_layer = self.swapaxes_for_scores(self.key(hidden_states))
    value_layer = self.swapaxes_for_scores(self.value(hidden_states))

    # Take the dot product between "query" and "key" to get the raw attention scores.
    attention_scores = ops.matmul(query_layer, key_layer.swapaxes(-1, -2))

    attention_scores = attention_scores / math.sqrt(self.attention_head_size)

    # Normalize the attention scores to probabilities.
    attention_probs = ops.softmax(attention_scores, axis=-1)

    # This is actually dropping out entire tokens to attend to, which might
    # seem a bit unusual, but is taken from the original Transformer paper.
    attention_probs = self.dropout(attention_probs)

    context_layer = ops.matmul(attention_probs, value_layer)

    context_layer = context_layer.permute(0, 2, 1, 3)
    new_context_layer_shape = context_layer.shape[:-2] + (self.all_head_size,)
    context_layer = context_layer.view(new_context_layer_shape)

    outputs = (context_layer, attention_probs) if output_attentions else (context_layer,)

    return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEfficientSelfAttention.swapaxes_for_scores(hidden_states)

Swaps axes and reshapes the input tensor for calculating attention scores in the SegformerEfficientSelfAttention class.

PARAMETER	DESCRIPTION
self	An instance of the SegformerEfficientSelfAttention class. TYPE: SegformerEfficientSelfAttention
hidden_states	A tensor representing the hidden states. It should have a shape of (batch_size, sequence_length, hidden_size). TYPE: Tensor

RETURNS	DESCRIPTION
	torch.Tensor: A tensor representing the reshaped hidden states. The shape of the tensor will be (batch_size, num_attention_heads, sequence_length, attention_head_size).

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def swapaxes_for_scores(self, hidden_states):
    """
    Swaps axes and reshapes the input tensor for calculating attention scores in the
    SegformerEfficientSelfAttention class.

    Args:
        self (SegformerEfficientSelfAttention): An instance of the SegformerEfficientSelfAttention class.
        hidden_states (torch.Tensor): A tensor representing the hidden states. It should have a shape of
            (batch_size, sequence_length, hidden_size).

    Returns:
        torch.Tensor: A tensor representing the reshaped hidden states. The shape of the tensor will be
            (batch_size, num_attention_heads, sequence_length, attention_head_size).

    Raises:
        None.
    """
    new_shape = hidden_states.shape[:-1] + (self.num_attention_heads, self.attention_head_size)
    hidden_states = hidden_states.view(new_shape)
    return hidden_states.permute(0, 2, 1, 3)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEncoder

Bases: Module

SegformerEncoder is a neural network module that represents the encoder of the Segformer model. It takes input pixel values and produces a sequence of hidden states that can be used for various downstream tasks.

Inherits from

nn.Module

PARAMETER	DESCRIPTION
config	An instance of SegformerConfig that contains various hyperparameters for the encoder.

RAISES	DESCRIPTION
ValueError	If the input config is not an instance of SegformerConfig.

Example

>>> config = SegformerConfig()
>>> encoder = SegformerEncoder(config)
>>> pixel_values = mindspore.Tensor(np.zeros((1, 3, 224, 224)), mindspore.float32)
>>> outputs = encoder(pixel_values)

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerEncoder(nn.Module):

    """
    SegformerEncoder is a neural network module that represents the encoder of the Segformer model.
    It takes input pixel values and produces a sequence of hidden states that can be used for various downstream tasks.

    Inherits from:
        nn.Module

    Args:
        config: An instance of SegformerConfig that contains various hyperparameters for the encoder.

    Raises:
        ValueError: If the input config is not an instance of SegformerConfig.

    Example:
        ```python
        >>> config = SegformerConfig()
        >>> encoder = SegformerEncoder(config)
        >>> pixel_values = mindspore.Tensor(np.zeros((1, 3, 224, 224)), mindspore.float32)
        >>> outputs = encoder(pixel_values)
        ```
    """
    def __init__(self, config):
        """
        This method initializes a SegformerEncoder instance with the provided configuration.

        Args:
            self (SegformerEncoder): The SegformerEncoder instance.
            config (object): A configuration object containing various parameters for the SegformerEncoder.
                It should include the following attributes:

                - depths (List[int]): The number of layers in each encoder block.
                - drop_path_rate (float): The drop path rate for the network.
                - num_encoder_blocks (int): The number of encoder blocks.
                - patch_sizes (List[int]): The patch sizes for each encoder block.
                - strides (List[int]): The strides for each encoder block.
                - num_channels (int): The number of input channels.
                - hidden_sizes (List[int]): The hidden sizes for each encoder block.
                - num_attention_heads (List[int]): The number of attention heads for each encoder block.
                - sr_ratios (List[float]): The sequence reduction ratios for each encoder block.
                - mlp_ratios (List[float]): The MLP ratios for each encoder block.

        Returns:
            None.

        Raises:
            ValueError: If the provided configuration is invalid or incomplete.
            TypeError: If the provided configuration is of an unexpected type.
        """
        super().__init__()
        self.config = config

        # stochastic depth decay rule
        drop_path_decays = [x.item() for x in ops.linspace(0, config.drop_path_rate, sum(config.depths))]

        # patch embeddings
        embeddings = []
        for i in range(config.num_encoder_blocks):
            embeddings.append(
                SegformerOverlapPatchEmbeddings(
                    patch_size=config.patch_sizes[i],
                    stride=config.strides[i],
                    num_channels=config.num_channels if i == 0 else config.hidden_sizes[i - 1],
                    hidden_size=config.hidden_sizes[i],
                )
            )
        self.patch_embeddings = nn.ModuleList(embeddings)

        # Transformer blocks
        blocks = []
        cur = 0
        for i in range(config.num_encoder_blocks):
            # each block consists of layers
            layers = []
            if i != 0:
                cur += config.depths[i - 1]
            for j in range(config.depths[i]):
                layers.append(
                    SegformerLayer(
                        config,
                        hidden_size=config.hidden_sizes[i],
                        num_attention_heads=config.num_attention_heads[i],
                        drop_path=drop_path_decays[cur + j],
                        sequence_reduction_ratio=config.sr_ratios[i],
                        mlp_ratio=config.mlp_ratios[i],
                    )
                )
            blocks.append(nn.ModuleList(layers))

        self.block = nn.ModuleList(blocks)

        # Layer norms
        self.layer_norm = nn.ModuleList(
            [nn.LayerNorm(config.hidden_sizes[i]) for i in range(config.num_encoder_blocks)]
        )

    def forward(
        self,
        pixel_values: mindspore.Tensor,
        output_attentions: Optional[bool] = False,
        output_hidden_states: Optional[bool] = False,
        return_dict: Optional[bool] = True,
    ) -> Union[Tuple, BaseModelOutput]:
        """
        Method to forward the SegformerEncoder.

        Args:
            self: The instance of the SegformerEncoder class.
            pixel_values (mindspore.Tensor): The input pixel values as a Tensor.
            output_attentions (Optional[bool], optional): Whether to output attentions. Defaults to False.
            output_hidden_states (Optional[bool], optional): Whether to output hidden states. Defaults to False.
            return_dict (Optional[bool], optional): Whether to return the output as a dictionary. Defaults to True.

        Returns:
            Union[Tuple, BaseModelOutput]:
                The output value which can be either a Tuple or BaseModelOutput.

                - If return_dict is True, it returns a BaseModelOutput containing the last hidden state, hidden states,
                and attentions.
                - If return_dict is False, it returns a Tuple containing the hidden_states, all_hidden_states,
                and all_self_attentions.

        Raises:
            None
        """
        all_hidden_states = () if output_hidden_states else None
        all_self_attentions = () if output_attentions else None

        batch_size = pixel_values.shape[0]

        hidden_states = pixel_values
        for idx, x in enumerate(zip(self.patch_embeddings, self.block, self.layer_norm)):
            embedding_layer, block_layer, norm_layer = x
            # first, obtain patch embeddings
            hidden_states, height, width = embedding_layer(hidden_states)
            # second, send embeddings through blocks
            for i, blk in enumerate(block_layer):
                layer_outputs = blk(hidden_states, height, width, output_attentions)
                hidden_states = layer_outputs[0]
                if output_attentions:
                    all_self_attentions = all_self_attentions + (layer_outputs[1],)
            # third, apply layer norm
            hidden_states = norm_layer(hidden_states)
            # fourth, optionally reshape back to (batch_size, num_channels, height, width)
            if idx != len(self.patch_embeddings) - 1 or (
                idx == len(self.patch_embeddings) - 1 and self.config.reshape_last_stage
            ):
                hidden_states = hidden_states.reshape(batch_size, height, width, -1).permute(0, 3, 1, 2)
            if output_hidden_states:
                all_hidden_states = all_hidden_states + (hidden_states,)

        if not return_dict:
            return tuple(v for v in [hidden_states, all_hidden_states, all_self_attentions] if v is not None)
        return BaseModelOutput(
            last_hidden_state=hidden_states,
            hidden_states=all_hidden_states,
            attentions=all_self_attentions,
        )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEncoder.__init__(config)

This method initializes a SegformerEncoder instance with the provided configuration.

PARAMETER	DESCRIPTION
self	The SegformerEncoder instance. TYPE: SegformerEncoder
config	A configuration object containing various parameters for the SegformerEncoder. It should include the following attributes: depths (List[int]): The number of layers in each encoder block. drop_path_rate (float): The drop path rate for the network. num_encoder_blocks (int): The number of encoder blocks. patch_sizes (List[int]): The patch sizes for each encoder block. strides (List[int]): The strides for each encoder block. num_channels (int): The number of input channels. hidden_sizes (List[int]): The hidden sizes for each encoder block. num_attention_heads (List[int]): The number of attention heads for each encoder block. sr_ratios (List[float]): The sequence reduction ratios for each encoder block. mlp_ratios (List[float]): The MLP ratios for each encoder block. TYPE: object

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
ValueError	If the provided configuration is invalid or incomplete.
TypeError	If the provided configuration is of an unexpected type.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config):
    """
    This method initializes a SegformerEncoder instance with the provided configuration.

    Args:
        self (SegformerEncoder): The SegformerEncoder instance.
        config (object): A configuration object containing various parameters for the SegformerEncoder.
            It should include the following attributes:

            - depths (List[int]): The number of layers in each encoder block.
            - drop_path_rate (float): The drop path rate for the network.
            - num_encoder_blocks (int): The number of encoder blocks.
            - patch_sizes (List[int]): The patch sizes for each encoder block.
            - strides (List[int]): The strides for each encoder block.
            - num_channels (int): The number of input channels.
            - hidden_sizes (List[int]): The hidden sizes for each encoder block.
            - num_attention_heads (List[int]): The number of attention heads for each encoder block.
            - sr_ratios (List[float]): The sequence reduction ratios for each encoder block.
            - mlp_ratios (List[float]): The MLP ratios for each encoder block.

    Returns:
        None.

    Raises:
        ValueError: If the provided configuration is invalid or incomplete.
        TypeError: If the provided configuration is of an unexpected type.
    """
    super().__init__()
    self.config = config

    # stochastic depth decay rule
    drop_path_decays = [x.item() for x in ops.linspace(0, config.drop_path_rate, sum(config.depths))]

    # patch embeddings
    embeddings = []
    for i in range(config.num_encoder_blocks):
        embeddings.append(
            SegformerOverlapPatchEmbeddings(
                patch_size=config.patch_sizes[i],
                stride=config.strides[i],
                num_channels=config.num_channels if i == 0 else config.hidden_sizes[i - 1],
                hidden_size=config.hidden_sizes[i],
            )
        )
    self.patch_embeddings = nn.ModuleList(embeddings)

    # Transformer blocks
    blocks = []
    cur = 0
    for i in range(config.num_encoder_blocks):
        # each block consists of layers
        layers = []
        if i != 0:
            cur += config.depths[i - 1]
        for j in range(config.depths[i]):
            layers.append(
                SegformerLayer(
                    config,
                    hidden_size=config.hidden_sizes[i],
                    num_attention_heads=config.num_attention_heads[i],
                    drop_path=drop_path_decays[cur + j],
                    sequence_reduction_ratio=config.sr_ratios[i],
                    mlp_ratio=config.mlp_ratios[i],
                )
            )
        blocks.append(nn.ModuleList(layers))

    self.block = nn.ModuleList(blocks)

    # Layer norms
    self.layer_norm = nn.ModuleList(
        [nn.LayerNorm(config.hidden_sizes[i]) for i in range(config.num_encoder_blocks)]
    )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerEncoder.forward(pixel_values, output_attentions=False, output_hidden_states=False, return_dict=True)

Method to forward the SegformerEncoder.

PARAMETER	DESCRIPTION
self	The instance of the SegformerEncoder class.
pixel_values	The input pixel values as a Tensor. TYPE: Tensor
output_attentions	Whether to output attentions. Defaults to False. TYPE: Optional[bool] DEFAULT: False
output_hidden_states	Whether to output hidden states. Defaults to False. TYPE: Optional[bool] DEFAULT: False
return_dict	Whether to return the output as a dictionary. Defaults to True. TYPE: Optional[bool] DEFAULT: True

RETURNS	DESCRIPTION
Union[Tuple, BaseModelOutput]	Union[Tuple, BaseModelOutput]: The output value which can be either a Tuple or BaseModelOutput. If return_dict is True, it returns a BaseModelOutput containing the last hidden state, hidden states, and attentions. If return_dict is False, it returns a Tuple containing the hidden_states, all_hidden_states, and all_self_attentions.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(
    self,
    pixel_values: mindspore.Tensor,
    output_attentions: Optional[bool] = False,
    output_hidden_states: Optional[bool] = False,
    return_dict: Optional[bool] = True,
) -> Union[Tuple, BaseModelOutput]:
    """
    Method to forward the SegformerEncoder.

    Args:
        self: The instance of the SegformerEncoder class.
        pixel_values (mindspore.Tensor): The input pixel values as a Tensor.
        output_attentions (Optional[bool], optional): Whether to output attentions. Defaults to False.
        output_hidden_states (Optional[bool], optional): Whether to output hidden states. Defaults to False.
        return_dict (Optional[bool], optional): Whether to return the output as a dictionary. Defaults to True.

    Returns:
        Union[Tuple, BaseModelOutput]:
            The output value which can be either a Tuple or BaseModelOutput.

            - If return_dict is True, it returns a BaseModelOutput containing the last hidden state, hidden states,
            and attentions.
            - If return_dict is False, it returns a Tuple containing the hidden_states, all_hidden_states,
            and all_self_attentions.

    Raises:
        None
    """
    all_hidden_states = () if output_hidden_states else None
    all_self_attentions = () if output_attentions else None

    batch_size = pixel_values.shape[0]

    hidden_states = pixel_values
    for idx, x in enumerate(zip(self.patch_embeddings, self.block, self.layer_norm)):
        embedding_layer, block_layer, norm_layer = x
        # first, obtain patch embeddings
        hidden_states, height, width = embedding_layer(hidden_states)
        # second, send embeddings through blocks
        for i, blk in enumerate(block_layer):
            layer_outputs = blk(hidden_states, height, width, output_attentions)
            hidden_states = layer_outputs[0]
            if output_attentions:
                all_self_attentions = all_self_attentions + (layer_outputs[1],)
        # third, apply layer norm
        hidden_states = norm_layer(hidden_states)
        # fourth, optionally reshape back to (batch_size, num_channels, height, width)
        if idx != len(self.patch_embeddings) - 1 or (
            idx == len(self.patch_embeddings) - 1 and self.config.reshape_last_stage
        ):
            hidden_states = hidden_states.reshape(batch_size, height, width, -1).permute(0, 3, 1, 2)
        if output_hidden_states:
            all_hidden_states = all_hidden_states + (hidden_states,)

    if not return_dict:
        return tuple(v for v in [hidden_states, all_hidden_states, all_self_attentions] if v is not None)
    return BaseModelOutput(
        last_hidden_state=hidden_states,
        hidden_states=all_hidden_states,
        attentions=all_self_attentions,
    )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForImageClassification

Bases: SegformerPreTrainedModel

This class represents a Segformer model for image classification. It is a subclass of SegformerPreTrainedModel.

The SegformerForImageClassification class initializes and forwards a Segformer model for image classification. It takes in a configuration object as a parameter, which includes the number of labels for classification.

The forwardor (init) initializes the SegformerForImageClassification object by calling the superclass's forwardor with the provided configuration. It sets the number of labels and creates instances of the SegformerModel and nn.Linear classes. The post_init method is then called.

The forward method forwards the Segformer model for image classification. It takes in several optional parameters, including pixel_values (input image tensor), labels (classification labels), output_attentions (whether to output attention weights), output_hidden_states (whether to output hidden states), and return_dict (whether to return results as a dictionary). It returns a tuple or a SegFormerImageClassifierOutput object.

The labels parameter is an optional tensor that represents the classification labels for computing the image classification/regression loss. The indices in the labels tensor should be in the range of [0, ..., config.num_labels - 1]. If config.num_labels == 1, a regression loss (Mean-Square loss) is computed. If config.num_labels > 1, a classification loss (Cross-Entropy) is computed.

The method first calls the SegformerModel's forward method with the provided inputs and optional parameters. The output of the forward pass is stored in the sequence_output variable. If the reshape_last_stage configuration option is enabled, the sequence_output tensor is permuted and reshaped. Then, the mean of the sequence_output tensor is calculated along the second axis.

The logits tensor is obtained by passing the sequence_output tensor through the classifier module. The loss variable is initially set to None.

If the labels tensor is provided, the problem_type configuration option is checked to determine the type of loss calculation. If the problem_type is not set, it is inferred based on the number of labels and the data type of the labels tensor. For regression problems with a single label, the problem_type is set to 'regression'. For single-label classification problems, the problem_type is set to 'single_label_classification'. For multi-label classification problems, the problem_type is set to 'multi_label_classification'.

The loss is calculated based on the problem_type. For regression problems with a single label, the mean squared error (MSE) loss is computed. For single-label classification problems, the cross-entropy loss is computed. For multi-label classification problems, the binary cross-entropy with logits loss is computed.

Finally, the method returns the computed loss and other outputs depending on the value of the return_dict parameter. If return_dict is False, the method returns a tuple containing the logits and other outputs. If loss is None, the output tuple does not include the loss. If return_dict is True, the method returns a SegFormerImageClassifierOutput object containing the loss, logits, hidden states, and attentions.

Note

This docstring does not include the function signatures or any other code.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerForImageClassification(SegformerPreTrainedModel):

    """
    This class represents a Segformer model for image classification. It is a subclass of SegformerPreTrainedModel.

    The SegformerForImageClassification class initializes and forwards a Segformer model for image classification.
    It takes in a configuration object as a parameter, which includes the number of labels for classification.

    The forwardor (__init__) initializes the SegformerForImageClassification object by calling the superclass's
    forwardor with the provided configuration. It sets the number of labels and creates instances of the
    SegformerModel and nn.Linear classes. The post_init method is then called.

    The forward method forwards the Segformer model for image classification. It takes in several optional
    parameters, including pixel_values (input image tensor), labels (classification labels), output_attentions
    (whether to output attention weights), output_hidden_states (whether to output hidden states), and return_dict
    (whether to return results as a dictionary). It returns a tuple or a SegFormerImageClassifierOutput object.

    The labels parameter is an optional tensor that represents the classification labels for computing the image
    classification/regression loss. The indices in the labels tensor should be in the range of
    [0, ..., config.num_labels - 1]. If config.num_labels == 1, a regression loss (Mean-Square loss) is computed.
    If config.num_labels > 1, a classification loss (Cross-Entropy) is computed.

    The method first calls the SegformerModel's forward method with the provided inputs and optional parameters.
    The output of the forward pass is stored in the sequence_output variable. If the reshape_last_stage configuration
    option is enabled, the sequence_output tensor is permuted and reshaped. Then, the mean of the sequence_output tensor
    is calculated along the second axis.

    The logits tensor is obtained by passing the sequence_output tensor through the classifier module.
    The loss variable is initially set to None.

    If the labels tensor is provided, the problem_type configuration option is checked to determine the type of loss
    calculation. If the problem_type is not set, it is inferred based on the number of labels and the data type of the
    labels tensor. For regression problems with a single label, the problem_type is set to 'regression'.
    For single-label classification problems, the problem_type is set to 'single_label_classification'.
    For multi-label classification problems, the problem_type is set to 'multi_label_classification'.

    The loss is calculated based on the problem_type. For regression problems with a single label, the mean squared error
    (MSE) loss is computed. For single-label classification problems, the cross-entropy loss is computed.
    For multi-label classification problems, the binary cross-entropy with logits loss is computed.

    Finally, the method returns the computed loss and other outputs depending on the value of the return_dict parameter.
    If return_dict is False, the method returns a tuple containing the logits and other outputs.
    If loss is None, the output tuple does not include the loss. If return_dict is True, the method returns a
    SegFormerImageClassifierOutput object containing the loss, logits, hidden states, and attentions.

    Note:
        This docstring does not include the function signatures or any other code.
    """
    def __init__(self, config):
        """
        Initializes a new SegformerForImageClassification instance.

        Args:
            self: The instance of the SegformerForImageClassification class.
            config: An object containing configuration settings for the model.
                It should include the following attributes:

                - num_labels (int): The number of labels for classification.
                - hidden_sizes (list of int): A list of sizes for the hidden layers.

        Returns:
            None.

        Raises:
            TypeError: If the config parameter is not of the expected type.
            ValueError: If the config parameter is missing required attributes.
            RuntimeError: If there is an issue during the initialization process.
        """
        super().__init__(config)

        self.num_labels = config.num_labels
        self.segformer = SegformerModel(config)

        # Classifier head
        self.classifier = nn.Linear(config.hidden_sizes[-1], config.num_labels)

        # Initialize weights and apply final processing
        self.post_init()

    def forward(
        self,
        pixel_values: Optional[mindspore.Tensor] = None,
        labels: Optional[mindspore.Tensor] = None,
        output_attentions: Optional[bool] = None,
        output_hidden_states: Optional[bool] = None,
        return_dict: Optional[bool] = None,
    ) -> Union[Tuple, SegFormerImageClassifierOutput]:
        r"""
        Args:
            labels (`mindspore.Tensor` of shape `(batch_size,)`, *optional*):
                Labels for computing the image classification/regression loss. Indices should be in `[0, ...,
                config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
                `config.num_labels > 1` a classification loss is computed (Cross-Entropy).
        """
        return_dict = return_dict if return_dict is not None else self.config.use_return_dict

        outputs = self.segformer(
            pixel_values,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
        )

        sequence_output = outputs[0]

        # convert last hidden states to (batch_size, height*width, hidden_size)
        batch_size = sequence_output.shape[0]
        if self.config.reshape_last_stage:
            # (batch_size, num_channels, height, width) -> (batch_size, height, width, num_channels)
            sequence_output = sequence_output.permute(0, 2, 3, 1)
        sequence_output = sequence_output.reshape(batch_size, -1, self.config.hidden_sizes[-1])

        # global average pooling
        sequence_output = sequence_output.mean(axis=1)

        logits = self.classifier(sequence_output)

        loss = None
        if labels is not None:
            if self.config.problem_type is None:
                if self.num_labels == 1:
                    self.config.problem_type = "regression"
                elif self.num_labels > 1 and labels.dtype in (mindspore.int64, mindspore.int32):
                    self.config.problem_type = "single_label_classification"
                else:
                    self.config.problem_type = "multi_label_classification"

            if self.config.problem_type == "regression":
                if self.num_labels == 1:
                    loss = ops.mse_loss(logits.squeeze(), labels.squeeze())
                else:
                    loss = ops.mse_loss(logits, labels)
            elif self.config.problem_type == "single_label_classification":
                loss = ops.cross_entropy(logits.view(-1, self.num_labels), labels.view(-1))
            elif self.config.problem_type == "multi_label_classification":
                loss = ops.binary_cross_entropy_with_logits(logits, labels)
        if not return_dict:
            output = (logits,) + outputs[1:]
            return ((loss,) + output) if loss is not None else output

        return SegFormerImageClassifierOutput(
            loss=loss,
            logits=logits,
            hidden_states=outputs.hidden_states,
            attentions=outputs.attentions,
        )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForImageClassification.__init__(config)

Initializes a new SegformerForImageClassification instance.

PARAMETER	DESCRIPTION
self	The instance of the SegformerForImageClassification class.
config	An object containing configuration settings for the model. It should include the following attributes: num_labels (int): The number of labels for classification. hidden_sizes (list of int): A list of sizes for the hidden layers.

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
TypeError	If the config parameter is not of the expected type.
ValueError	If the config parameter is missing required attributes.
RuntimeError	If there is an issue during the initialization process.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config):
    """
    Initializes a new SegformerForImageClassification instance.

    Args:
        self: The instance of the SegformerForImageClassification class.
        config: An object containing configuration settings for the model.
            It should include the following attributes:

            - num_labels (int): The number of labels for classification.
            - hidden_sizes (list of int): A list of sizes for the hidden layers.

    Returns:
        None.

    Raises:
        TypeError: If the config parameter is not of the expected type.
        ValueError: If the config parameter is missing required attributes.
        RuntimeError: If there is an issue during the initialization process.
    """
    super().__init__(config)

    self.num_labels = config.num_labels
    self.segformer = SegformerModel(config)

    # Classifier head
    self.classifier = nn.Linear(config.hidden_sizes[-1], config.num_labels)

    # Initialize weights and apply final processing
    self.post_init()

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForImageClassification.forward(pixel_values=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)

PARAMETER	DESCRIPTION
labels	Labels for computing the image classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If config.num_labels > 1 a classification loss is computed (Cross-Entropy). TYPE: `mindspore.Tensor` of shape `(batch_size,)`, *optional* DEFAULT: None

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(
    self,
    pixel_values: Optional[mindspore.Tensor] = None,
    labels: Optional[mindspore.Tensor] = None,
    output_attentions: Optional[bool] = None,
    output_hidden_states: Optional[bool] = None,
    return_dict: Optional[bool] = None,
) -> Union[Tuple, SegFormerImageClassifierOutput]:
    r"""
    Args:
        labels (`mindspore.Tensor` of shape `(batch_size,)`, *optional*):
            Labels for computing the image classification/regression loss. Indices should be in `[0, ...,
            config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
            `config.num_labels > 1` a classification loss is computed (Cross-Entropy).
    """
    return_dict = return_dict if return_dict is not None else self.config.use_return_dict

    outputs = self.segformer(
        pixel_values,
        output_attentions=output_attentions,
        output_hidden_states=output_hidden_states,
        return_dict=return_dict,
    )

    sequence_output = outputs[0]

    # convert last hidden states to (batch_size, height*width, hidden_size)
    batch_size = sequence_output.shape[0]
    if self.config.reshape_last_stage:
        # (batch_size, num_channels, height, width) -> (batch_size, height, width, num_channels)
        sequence_output = sequence_output.permute(0, 2, 3, 1)
    sequence_output = sequence_output.reshape(batch_size, -1, self.config.hidden_sizes[-1])

    # global average pooling
    sequence_output = sequence_output.mean(axis=1)

    logits = self.classifier(sequence_output)

    loss = None
    if labels is not None:
        if self.config.problem_type is None:
            if self.num_labels == 1:
                self.config.problem_type = "regression"
            elif self.num_labels > 1 and labels.dtype in (mindspore.int64, mindspore.int32):
                self.config.problem_type = "single_label_classification"
            else:
                self.config.problem_type = "multi_label_classification"

        if self.config.problem_type == "regression":
            if self.num_labels == 1:
                loss = ops.mse_loss(logits.squeeze(), labels.squeeze())
            else:
                loss = ops.mse_loss(logits, labels)
        elif self.config.problem_type == "single_label_classification":
            loss = ops.cross_entropy(logits.view(-1, self.num_labels), labels.view(-1))
        elif self.config.problem_type == "multi_label_classification":
            loss = ops.binary_cross_entropy_with_logits(logits, labels)
    if not return_dict:
        output = (logits,) + outputs[1:]
        return ((loss,) + output) if loss is not None else output

    return SegFormerImageClassifierOutput(
        loss=loss,
        logits=logits,
        hidden_states=outputs.hidden_states,
        attentions=outputs.attentions,
    )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForSemanticSegmentation

Bases: SegformerPreTrainedModel

This class represents a Segformer model for semantic segmentation, specifically designed for image processing tasks. It is a subclass of SegformerPreTrainedModel.

The SegformerForSemanticSegmentation class includes methods for model initialization and forwardion. It utilizes the SegformerModel and SegformerDecodeHead classes for the main processing steps.

METHOD	DESCRIPTION
`__init__`	Initializes the SegformerForSemanticSegmentation instance with a given configuration. Parameters: config: The configuration object for the model.
`forward`	Constructs the semantic segmentation output based on the input pixel values. Parameters: pixel_values: A tensor containing the input pixel values for the image. labels (Optional): Ground truth semantic segmentation maps for computing the loss. Indices should be in the range [0, config.num_labels - 1]. If config.num_labels > 1, a classification loss is computed (Cross-Entropy). output_attentions (Optional): Boolean flag indicating whether to output attention weights. output_hidden_states (Optional): Boolean flag indicating whether to output hidden states. return_dict (Optional): Boolean flag indicating whether to return the output as a dictionary. Returns: If return_dict is False: If output_hidden_states is True: A tuple containing the logits and hidden states (logits, hidden_states). If output_hidden_states is False: A tuple containing the logits and attentions (logits, attentions). If return_dict is True: An instance of SemanticSegmenterOutput containing the loss, logits, hidden states (if output_hidden_states is True), and attentions.

Example

>>> from transformers import AutoImageProcessor, SegformerForSemanticSegmentation
>>> from PIL import Image
>>> import requests
...
>>> image_processor = AutoImageProcessor.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
>>> model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
...
>>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
>>> image = Image.open(requests.get(url, stream=True).raw)
...
>>> inputs = image_processor(images=image, return_tensors="pt")
>>> outputs = model(**inputs)
>>> logits = outputs.logits  # shape (batch_size, num_labels, height/4, width/4)
>>> list(logits.shape)
[1, 150, 128, 128]

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerForSemanticSegmentation(SegformerPreTrainedModel):

    """
    This class represents a Segformer model for semantic segmentation, specifically designed for image processing tasks.
    It is a subclass of SegformerPreTrainedModel.

    The SegformerForSemanticSegmentation class includes methods for model initialization and forwardion.
    It utilizes the SegformerModel and SegformerDecodeHead classes for the main processing steps.

    Methods:
        `__init__`:
            Initializes the SegformerForSemanticSegmentation instance with a given configuration.

            - Parameters:

                - `config`: The configuration object for the model.

        `forward`:
            Constructs the semantic segmentation output based on the input pixel values.

            Parameters:

            - `pixel_values`: A tensor containing the input pixel values for the image.
            - `labels` (Optional): Ground truth semantic segmentation maps for computing the loss.
            Indices should be in the range [0, config.num_labels - 1]. If config.num_labels > 1,
            a classification loss is computed (Cross-Entropy).
            - `output_attentions` (Optional): Boolean flag indicating whether to output attention weights.
            - `output_hidden_states` (Optional): Boolean flag indicating whether to output hidden states.
            - `return_dict` (Optional): Boolean flag indicating whether to return the output as a dictionary.

            Returns:

            - If return_dict is False:

                - If output_hidden_states is True:
                A tuple containing the logits and hidden states (logits, hidden_states).
                - If output_hidden_states is False:
                A tuple containing the logits and attentions (logits, attentions).

            - If return_dict is True:

                - An instance of SemanticSegmenterOutput containing the loss, logits, hidden states
                (if output_hidden_states is True), and attentions.

    Example:
        ```python
        >>> from transformers import AutoImageProcessor, SegformerForSemanticSegmentation
        >>> from PIL import Image
        >>> import requests
        ...
        >>> image_processor = AutoImageProcessor.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
        >>> model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
        ...
        >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
        >>> image = Image.open(requests.get(url, stream=True).raw)
        ...
        >>> inputs = image_processor(images=image, return_tensors="pt")
        >>> outputs = model(**inputs)
        >>> logits = outputs.logits  # shape (batch_size, num_labels, height/4, width/4)
        >>> list(logits.shape)
        [1, 150, 128, 128]
        ```

    """
    def __init__(self, config):
        """
        Initializes an instance of SegformerForSemanticSegmentation.

        Args:
            self: The instance of the SegformerForSemanticSegmentation class.
            config: A dictionary containing configuration parameters for the Segformer model.

        Returns:
            None.

        Raises:
            TypeError: If the config parameter is not a dictionary.
            ValueError: If the config parameter does not contain the required configuration parameters.
        """
        super().__init__(config)
        self.segformer = SegformerModel(config)
        self.decode_head = SegformerDecodeHead(config)

        # Initialize weights and apply final processing
        self.post_init()

    def forward(
        self,
        pixel_values: mindspore.Tensor,
        labels: Optional[mindspore.Tensor] = None,
        output_attentions: Optional[bool] = None,
        output_hidden_states: Optional[bool] = None,
        return_dict: Optional[bool] = None,
    ) -> Union[Tuple, SemanticSegmenterOutput]:
        r"""
        Args:
            labels (`mindspore.Tensor` of shape `(batch_size, height, width)`, *optional*):
                Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ...,
                config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy).

        Returns:
            Union[Tuple, SemanticSegmenterOutput]

        Example:
            ```python
            >>> from transformers import AutoImageProcessor, SegformerForSemanticSegmentation
            >>> from PIL import Image
            >>> import requests
            ...
            >>> image_processor = AutoImageProcessor.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
            >>> model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
            ...
            >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
            >>> image = Image.open(requests.get(url, stream=True).raw)
            ...
            >>> inputs = image_processor(images=image, return_tensors="pt")
            >>> outputs = model(**inputs)
            >>> logits = outputs.logits  # shape (batch_size, num_labels, height/4, width/4)
            >>> list(logits.shape)
            [1, 150, 128, 128]
            ```
        """
        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
        output_hidden_states = (
            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
        )

        outputs = self.segformer(
            pixel_values,
            output_attentions=output_attentions,
            output_hidden_states=True,  # we need the intermediate hidden states
            return_dict=return_dict,
        )

        encoder_hidden_states = outputs.hidden_states if return_dict else outputs[1]

        logits = self.decode_head(encoder_hidden_states)

        loss = None
        if labels is not None:
            # upsample logits to the images' original size
            upsampled_logits = ops.interpolate(
                logits, size=labels.shape[-2:], mode="bilinear", align_corners=False
            )
            if self.config.num_labels > 1:
                loss = ops.cross_entropy(upsampled_logits, labels, ignore_index=self.config.semantic_loss_ignore_index)
            elif self.config.num_labels == 1:
                valid_mask = ((labels >= 0) & (labels != self.config.semantic_loss_ignore_index)).float()
                loss = ops.binary_cross_entropy_with_logits(upsampled_logits.squeeze(1), labels.float(), reduction="none")
                loss = (loss * valid_mask).mean()
            else:
                raise ValueError(f"Number of labels should be >=0: {self.config.num_labels}")

        if not return_dict:
            if output_hidden_states:
                output = (logits,) + outputs[1:]
            else:
                output = (logits,) + outputs[2:]
            return ((loss,) + output) if loss is not None else output

        return SemanticSegmenterOutput(
            loss=loss,
            logits=logits,
            hidden_states=outputs.hidden_states if output_hidden_states else None,
            attentions=outputs.attentions,
        )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForSemanticSegmentation.__init__(config)

Initializes an instance of SegformerForSemanticSegmentation.

PARAMETER	DESCRIPTION
self	The instance of the SegformerForSemanticSegmentation class.
config	A dictionary containing configuration parameters for the Segformer model.

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
TypeError	If the config parameter is not a dictionary.
ValueError	If the config parameter does not contain the required configuration parameters.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config):
    """
    Initializes an instance of SegformerForSemanticSegmentation.

    Args:
        self: The instance of the SegformerForSemanticSegmentation class.
        config: A dictionary containing configuration parameters for the Segformer model.

    Returns:
        None.

    Raises:
        TypeError: If the config parameter is not a dictionary.
        ValueError: If the config parameter does not contain the required configuration parameters.
    """
    super().__init__(config)
    self.segformer = SegformerModel(config)
    self.decode_head = SegformerDecodeHead(config)

    # Initialize weights and apply final processing
    self.post_init()

mindnlp.transformers.models.segformer.modeling_segformer.SegformerForSemanticSegmentation.forward(pixel_values, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)

PARAMETER	DESCRIPTION
labels	Ground truth semantic segmentation maps for computing the loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels > 1, a classification loss is computed (Cross-Entropy). TYPE: `mindspore.Tensor` of shape `(batch_size, height, width)`, *optional* DEFAULT: None

RETURNS	DESCRIPTION
Union[Tuple, SemanticSegmenterOutput]	Union[Tuple, SemanticSegmenterOutput]

Example

>>> from transformers import AutoImageProcessor, SegformerForSemanticSegmentation
>>> from PIL import Image
>>> import requests
...
>>> image_processor = AutoImageProcessor.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
>>> model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
...
>>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
>>> image = Image.open(requests.get(url, stream=True).raw)
...
>>> inputs = image_processor(images=image, return_tensors="pt")
>>> outputs = model(**inputs)
>>> logits = outputs.logits  # shape (batch_size, num_labels, height/4, width/4)
>>> list(logits.shape)
[1, 150, 128, 128]

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(
    self,
    pixel_values: mindspore.Tensor,
    labels: Optional[mindspore.Tensor] = None,
    output_attentions: Optional[bool] = None,
    output_hidden_states: Optional[bool] = None,
    return_dict: Optional[bool] = None,
) -> Union[Tuple, SemanticSegmenterOutput]:
    r"""
    Args:
        labels (`mindspore.Tensor` of shape `(batch_size, height, width)`, *optional*):
            Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ...,
            config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy).

    Returns:
        Union[Tuple, SemanticSegmenterOutput]

    Example:
        ```python
        >>> from transformers import AutoImageProcessor, SegformerForSemanticSegmentation
        >>> from PIL import Image
        >>> import requests
        ...
        >>> image_processor = AutoImageProcessor.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
        >>> model = SegformerForSemanticSegmentation.from_pretrained("nvidia/segformer-b0-finetuned-ade-512-512")
        ...
        >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
        >>> image = Image.open(requests.get(url, stream=True).raw)
        ...
        >>> inputs = image_processor(images=image, return_tensors="pt")
        >>> outputs = model(**inputs)
        >>> logits = outputs.logits  # shape (batch_size, num_labels, height/4, width/4)
        >>> list(logits.shape)
        [1, 150, 128, 128]
        ```
    """
    return_dict = return_dict if return_dict is not None else self.config.use_return_dict
    output_hidden_states = (
        output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
    )

    outputs = self.segformer(
        pixel_values,
        output_attentions=output_attentions,
        output_hidden_states=True,  # we need the intermediate hidden states
        return_dict=return_dict,
    )

    encoder_hidden_states = outputs.hidden_states if return_dict else outputs[1]

    logits = self.decode_head(encoder_hidden_states)

    loss = None
    if labels is not None:
        # upsample logits to the images' original size
        upsampled_logits = ops.interpolate(
            logits, size=labels.shape[-2:], mode="bilinear", align_corners=False
        )
        if self.config.num_labels > 1:
            loss = ops.cross_entropy(upsampled_logits, labels, ignore_index=self.config.semantic_loss_ignore_index)
        elif self.config.num_labels == 1:
            valid_mask = ((labels >= 0) & (labels != self.config.semantic_loss_ignore_index)).float()
            loss = ops.binary_cross_entropy_with_logits(upsampled_logits.squeeze(1), labels.float(), reduction="none")
            loss = (loss * valid_mask).mean()
        else:
            raise ValueError(f"Number of labels should be >=0: {self.config.num_labels}")

    if not return_dict:
        if output_hidden_states:
            output = (logits,) + outputs[1:]
        else:
            output = (logits,) + outputs[2:]
        return ((loss,) + output) if loss is not None else output

    return SemanticSegmenterOutput(
        loss=loss,
        logits=logits,
        hidden_states=outputs.hidden_states if output_hidden_states else None,
        attentions=outputs.attentions,
    )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerLayer

Bases: Module

This corresponds to the Block class in the original implementation.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerLayer(nn.Module):
    """This corresponds to the Block class in the original implementation."""
    def __init__(self, config, hidden_size, num_attention_heads, drop_path, sequence_reduction_ratio, mlp_ratio):
        """
        Initializes a new instance of the SegformerLayer class.

        Args:
            self: The instance of the class.
            config: A configuration object specifying the settings for the SegformerLayer.
            hidden_size (int): The size of the hidden layer.
            num_attention_heads (int): The number of attention heads.
            drop_path (float): The probability of dropping a path during training. Must be between 0.0 and 1.0.
            sequence_reduction_ratio (float): The ratio by which the sequence length is reduced.
            mlp_ratio (float): The ratio by which the hidden size of the Multi-Layer Perceptron (MLP) is computed.

        Returns:
            None.

        Raises:
            None.
        """
        super().__init__()
        self.layer_norm_1 = nn.LayerNorm(hidden_size)
        self.attention = SegformerAttention(
            config,
            hidden_size=hidden_size,
            num_attention_heads=num_attention_heads,
            sequence_reduction_ratio=sequence_reduction_ratio,
        )
        self.drop_path = SegformerDropPath(drop_path) if drop_path > 0.0 else nn.Identity()
        self.layer_norm_2 = nn.LayerNorm(hidden_size)
        mlp_hidden_size = int(hidden_size * mlp_ratio)
        self.mlp = SegformerMixFFN(config, in_features=hidden_size, hidden_features=mlp_hidden_size)

    def forward(self, hidden_states, height, width, output_attentions=False):
        """
        This method forwards a Segformer layer by performing self-attention and multi-layer perceptron (mlp) operations.

        Args:
            self (object): The instance of the SegformerLayer class.
            hidden_states (tensor): The input tensor representing the hidden states of the layer.
            height (int): The height dimension of the input tensor.
            width (int): The width dimension of the input tensor.
            output_attentions (bool, optional): Flag indicating whether to output attentions. Defaults to False.

        Returns:
            tuple: A tuple containing the output layer and any additional outputs from the layer.

        Raises:
            None.
        """
        self_attention_outputs = self.attention(
            self.layer_norm_1(hidden_states),  # in Segformer, layernorm is applied before self-attention
            height,
            width,
            output_attentions=output_attentions,
        )

        attention_output = self_attention_outputs[0]
        outputs = self_attention_outputs[1:]  # add self attentions if we output attention weights

        # first residual connection (with stochastic depth)
        attention_output = self.drop_path(attention_output)
        hidden_states = attention_output + hidden_states

        mlp_output = self.mlp(self.layer_norm_2(hidden_states), height, width)

        # second residual connection (with stochastic depth)
        mlp_output = self.drop_path(mlp_output)
        layer_output = mlp_output + hidden_states

        outputs = (layer_output,) + outputs

        return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerLayer.__init__(config, hidden_size, num_attention_heads, drop_path, sequence_reduction_ratio, mlp_ratio)

Initializes a new instance of the SegformerLayer class.

PARAMETER	DESCRIPTION
self	The instance of the class.
config	A configuration object specifying the settings for the SegformerLayer.
hidden_size	The size of the hidden layer. TYPE: int
num_attention_heads	The number of attention heads. TYPE: int
drop_path	The probability of dropping a path during training. Must be between 0.0 and 1.0. TYPE: float
sequence_reduction_ratio	The ratio by which the sequence length is reduced. TYPE: float
mlp_ratio	The ratio by which the hidden size of the Multi-Layer Perceptron (MLP) is computed. TYPE: float

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config, hidden_size, num_attention_heads, drop_path, sequence_reduction_ratio, mlp_ratio):
    """
    Initializes a new instance of the SegformerLayer class.

    Args:
        self: The instance of the class.
        config: A configuration object specifying the settings for the SegformerLayer.
        hidden_size (int): The size of the hidden layer.
        num_attention_heads (int): The number of attention heads.
        drop_path (float): The probability of dropping a path during training. Must be between 0.0 and 1.0.
        sequence_reduction_ratio (float): The ratio by which the sequence length is reduced.
        mlp_ratio (float): The ratio by which the hidden size of the Multi-Layer Perceptron (MLP) is computed.

    Returns:
        None.

    Raises:
        None.
    """
    super().__init__()
    self.layer_norm_1 = nn.LayerNorm(hidden_size)
    self.attention = SegformerAttention(
        config,
        hidden_size=hidden_size,
        num_attention_heads=num_attention_heads,
        sequence_reduction_ratio=sequence_reduction_ratio,
    )
    self.drop_path = SegformerDropPath(drop_path) if drop_path > 0.0 else nn.Identity()
    self.layer_norm_2 = nn.LayerNorm(hidden_size)
    mlp_hidden_size = int(hidden_size * mlp_ratio)
    self.mlp = SegformerMixFFN(config, in_features=hidden_size, hidden_features=mlp_hidden_size)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerLayer.forward(hidden_states, height, width, output_attentions=False)

This method forwards a Segformer layer by performing self-attention and multi-layer perceptron (mlp) operations.

PARAMETER	DESCRIPTION
self	The instance of the SegformerLayer class. TYPE: object
hidden_states	The input tensor representing the hidden states of the layer. TYPE: tensor
height	The height dimension of the input tensor. TYPE: int
width	The width dimension of the input tensor. TYPE: int
output_attentions	Flag indicating whether to output attentions. Defaults to False. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
tuple	A tuple containing the output layer and any additional outputs from the layer.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states, height, width, output_attentions=False):
    """
    This method forwards a Segformer layer by performing self-attention and multi-layer perceptron (mlp) operations.

    Args:
        self (object): The instance of the SegformerLayer class.
        hidden_states (tensor): The input tensor representing the hidden states of the layer.
        height (int): The height dimension of the input tensor.
        width (int): The width dimension of the input tensor.
        output_attentions (bool, optional): Flag indicating whether to output attentions. Defaults to False.

    Returns:
        tuple: A tuple containing the output layer and any additional outputs from the layer.

    Raises:
        None.
    """
    self_attention_outputs = self.attention(
        self.layer_norm_1(hidden_states),  # in Segformer, layernorm is applied before self-attention
        height,
        width,
        output_attentions=output_attentions,
    )

    attention_output = self_attention_outputs[0]
    outputs = self_attention_outputs[1:]  # add self attentions if we output attention weights

    # first residual connection (with stochastic depth)
    attention_output = self.drop_path(attention_output)
    hidden_states = attention_output + hidden_states

    mlp_output = self.mlp(self.layer_norm_2(hidden_states), height, width)

    # second residual connection (with stochastic depth)
    mlp_output = self.drop_path(mlp_output)
    layer_output = mlp_output + hidden_states

    outputs = (layer_output,) + outputs

    return outputs

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMLP

Bases: Module

Linear Embedding.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerMLP(nn.Module):
    """
    Linear Embedding.
    """
    def __init__(self, config: SegformerConfig, input_dim):
        """
        Initializes the SegformerMLP class.

        Args:
            self (object): The instance of the SegformerMLP class.
            config (SegformerConfig): An instance of SegformerConfig containing configuration settings.
            input_dim (int): The dimensionality of the input data.

        Returns:
            None.

        Raises:
            TypeError: If the input arguments are not of the expected types.
            ValueError: If the input_dim is less than or equal to 0.
            RuntimeError: If there is an issue during the initialization process.
        """
        super().__init__()
        self.proj = nn.Linear(input_dim, config.decoder_hidden_size)

    def forward(self, hidden_states: mindspore.Tensor):
        """
        Constructs the SegformerMLP.

        Args:
            self (SegformerMLP): An instance of the SegformerMLP class.
            hidden_states (mindspore.Tensor): A tensor containing the hidden states.
                It should have a shape of (batch_size, sequence_length, hidden_size).

        Returns:
            None.

        Raises:
            None.
        """
        hidden_states = hidden_states.flatten(start_dim=2).swapaxes(1, 2)
        hidden_states = self.proj(hidden_states)
        return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMLP.__init__(config, input_dim)

Initializes the SegformerMLP class.

PARAMETER	DESCRIPTION
self	The instance of the SegformerMLP class. TYPE: object
config	An instance of SegformerConfig containing configuration settings. TYPE: SegformerConfig
input_dim	The dimensionality of the input data. TYPE: int

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
TypeError	If the input arguments are not of the expected types.
ValueError	If the input_dim is less than or equal to 0.
RuntimeError	If there is an issue during the initialization process.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config: SegformerConfig, input_dim):
    """
    Initializes the SegformerMLP class.

    Args:
        self (object): The instance of the SegformerMLP class.
        config (SegformerConfig): An instance of SegformerConfig containing configuration settings.
        input_dim (int): The dimensionality of the input data.

    Returns:
        None.

    Raises:
        TypeError: If the input arguments are not of the expected types.
        ValueError: If the input_dim is less than or equal to 0.
        RuntimeError: If there is an issue during the initialization process.
    """
    super().__init__()
    self.proj = nn.Linear(input_dim, config.decoder_hidden_size)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMLP.forward(hidden_states)

Constructs the SegformerMLP.

PARAMETER	DESCRIPTION
self	An instance of the SegformerMLP class. TYPE: SegformerMLP
hidden_states	A tensor containing the hidden states. It should have a shape of (batch_size, sequence_length, hidden_size). TYPE: Tensor

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states: mindspore.Tensor):
    """
    Constructs the SegformerMLP.

    Args:
        self (SegformerMLP): An instance of the SegformerMLP class.
        hidden_states (mindspore.Tensor): A tensor containing the hidden states.
            It should have a shape of (batch_size, sequence_length, hidden_size).

    Returns:
        None.

    Raises:
        None.
    """
    hidden_states = hidden_states.flatten(start_dim=2).swapaxes(1, 2)
    hidden_states = self.proj(hidden_states)
    return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMixFFN

Bases: Module

The SegformerMixFFN class represents a feed-forward neural network (FFN) module for the Segformer architecture. It is designed to process input features and generate output features using dense layers, depthwise convolution, activation functions, and dropout regularization. The class inherits from nn.Module and provides methods for initializing the module and forwarding the FFN computation graph.

ATTRIBUTE	DESCRIPTION
config	The configuration object containing parameters for the FFN module. TYPE: object
in_features	The number of input features. TYPE: int
hidden_features	The number of hidden features. If not provided, defaults to None. TYPE: int
out_features	The number of output features. If not provided, defaults to None. TYPE: int

METHOD	DESCRIPTION
__init__	Initializes the SegformerMixFFN module with the provided configuration and feature dimensions.
forward	Constructs the computation graph for the FFN module using the given input hidden_states and spatial dimensions (height and width).

The forwardion of the computation graph involves passing the input through dense layers, depthwise convolution, activation functions, and dropout layers to generate the output hidden states.

Note

This docstring is a representation of the class attributes and methods. Please refer to the source code for the most accurate and up-to-date information.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerMixFFN(nn.Module):

    """
    The SegformerMixFFN class represents a feed-forward neural network (FFN) module for the Segformer architecture.
    It is designed to process input features and generate output features using dense layers, depthwise convolution,
    activation functions, and dropout regularization.
    The class inherits from nn.Module and provides methods for initializing the module and forwarding the FFN
    computation graph.

    Attributes:
        config (object): The configuration object containing parameters for the FFN module.
        in_features (int): The number of input features.
        hidden_features (int, optional): The number of hidden features. If not provided, defaults to None.
        out_features (int, optional): The number of output features. If not provided, defaults to None.

    Methods:
        __init__:
            Initializes the SegformerMixFFN module with the provided configuration and feature dimensions.

        forward:
            Constructs the computation graph for the FFN module using the given input hidden_states and spatial
            dimensions (height and width).

    The forwardion of the computation graph involves passing the input through dense layers, depthwise convolution,
    activation functions, and dropout layers to generate the output hidden states.

    Note:
        This docstring is a representation of the class attributes and methods. Please refer to the source code for
        the most accurate and up-to-date information.
    """
    def __init__(self, config, in_features, hidden_features=None, out_features=None):
        """
        Initializes an instance of the SegformerMixFFN class.

        Args:
            self: The object itself.
            config (object): The configuration object containing various settings.
            in_features (int): The number of input features.
            hidden_features (int, optional): The number of hidden features. Defaults to None.
            out_features (int, optional): The number of output features. If not provided,
                it will be set equal to in_features.

        Returns:
            None

        Raises:
            None
        """
        super().__init__()
        out_features = out_features or in_features
        self.dense1 = nn.Linear(in_features, hidden_features)
        self.dwconv = SegformerDWConv(hidden_features)
        if isinstance(config.hidden_act, str):
            self.intermediate_act_fn = ACT2FN[config.hidden_act]
        else:
            self.intermediate_act_fn = config.hidden_act
        self.dense2 = nn.Linear(hidden_features, out_features)
        self.dropout = nn.Dropout(p=config.hidden_dropout_prob)

    def forward(self, hidden_states, height, width):
        '''
        This method forwards the feed-forward network for the SegformerMixFFN class.

        Args:
            self (object): The instance of the SegformerMixFFN class.
            hidden_states (tensor): The input hidden states for the feed-forward network.
            height (int): The height of the input feature map.
            width (int): The width of the input feature map.

        Returns:
            None.

        Raises:
            None
        '''
        hidden_states = self.dense1(hidden_states)
        hidden_states = self.dwconv(hidden_states, height, width)
        hidden_states = self.intermediate_act_fn(hidden_states)
        hidden_states = self.dropout(hidden_states)
        hidden_states = self.dense2(hidden_states)
        hidden_states = self.dropout(hidden_states)
        return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMixFFN.__init__(config, in_features, hidden_features=None, out_features=None)

Initializes an instance of the SegformerMixFFN class.

PARAMETER	DESCRIPTION
self	The object itself.
config	The configuration object containing various settings. TYPE: object
in_features	The number of input features. TYPE: int
hidden_features	The number of hidden features. Defaults to None. TYPE: int DEFAULT: None
out_features	The number of output features. If not provided, it will be set equal to in_features. TYPE: int DEFAULT: None

RETURNS	DESCRIPTION
	None

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config, in_features, hidden_features=None, out_features=None):
    """
    Initializes an instance of the SegformerMixFFN class.

    Args:
        self: The object itself.
        config (object): The configuration object containing various settings.
        in_features (int): The number of input features.
        hidden_features (int, optional): The number of hidden features. Defaults to None.
        out_features (int, optional): The number of output features. If not provided,
            it will be set equal to in_features.

    Returns:
        None

    Raises:
        None
    """
    super().__init__()
    out_features = out_features or in_features
    self.dense1 = nn.Linear(in_features, hidden_features)
    self.dwconv = SegformerDWConv(hidden_features)
    if isinstance(config.hidden_act, str):
        self.intermediate_act_fn = ACT2FN[config.hidden_act]
    else:
        self.intermediate_act_fn = config.hidden_act
    self.dense2 = nn.Linear(hidden_features, out_features)
    self.dropout = nn.Dropout(p=config.hidden_dropout_prob)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerMixFFN.forward(hidden_states, height, width)

This method forwards the feed-forward network for the SegformerMixFFN class.

PARAMETER	DESCRIPTION
self	The instance of the SegformerMixFFN class. TYPE: object
hidden_states	The input hidden states for the feed-forward network. TYPE: tensor
height	The height of the input feature map. TYPE: int
width	The width of the input feature map. TYPE: int

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states, height, width):
    '''
    This method forwards the feed-forward network for the SegformerMixFFN class.

    Args:
        self (object): The instance of the SegformerMixFFN class.
        hidden_states (tensor): The input hidden states for the feed-forward network.
        height (int): The height of the input feature map.
        width (int): The width of the input feature map.

    Returns:
        None.

    Raises:
        None
    '''
    hidden_states = self.dense1(hidden_states)
    hidden_states = self.dwconv(hidden_states, height, width)
    hidden_states = self.intermediate_act_fn(hidden_states)
    hidden_states = self.dropout(hidden_states)
    hidden_states = self.dense2(hidden_states)
    hidden_states = self.dropout(hidden_states)
    return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerModel

Bases: SegformerPreTrainedModel

A Python class representing a SegformerModel.

This class is a SegformerModel that inherits from SegformerPreTrainedModel. It is used for performing semantic segmentation tasks using the Segformer architecture.

The SegformerModel class provides methods for initializing the model, pruning model heads, and forwarding the model with input pixel values. It also allows for customization of the output, including attention maps and hidden states.

METHOD	DESCRIPTION
__init__	Initializes the SegformerModel instance with the provided configuration.
_prune_heads	Prunes specific heads of the model based on the provided dictionary.
forward	Constructs the model with the given pixel values and returns the output. Customization of output options is available.

Note

This class assumes the presence of the SegformerPreTrainedModel class.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerModel(SegformerPreTrainedModel):

    """
    A Python class representing a SegformerModel.

    This class is a SegformerModel that inherits from SegformerPreTrainedModel. It is used for performing semantic
    segmentation tasks using the Segformer architecture.

    The SegformerModel class provides methods for initializing the model, pruning model heads, and forwarding the
    model with input pixel values. It also allows for customization of the output, including attention maps and hidden
    states.

    Methods:
        __init__: Initializes the SegformerModel instance with the provided configuration.
        _prune_heads: Prunes specific heads of the model based on the provided dictionary.
        forward: Constructs the model with the given pixel values and returns the output.
            Customization of output options is available.

    Note:
        This class assumes the presence of the SegformerPreTrainedModel class.

    """
    def __init__(self, config):
        """
        Initializes an instance of the SegformerModel class.

        Args:
            self: The instance of the SegformerModel class.
            config (dict):
                A dictionary containing configuration parameters for initializing the SegformerModel.
                It should include the necessary configuration settings for the model.
                Required keys and their datatypes:

                - key1 (datatype): Description.
                - key2 (datatype): Description.
                - ...
                - (Add more keys and descriptions as needed)

        Returns:
            None.

        Raises:
            many exceptions:
                Any exceptions that may be raised during the initialization process should be documented here:

                - ExampleException: Description of the example exception that may be raised.
                - AnotherException: Description of another exception that may be raised.
                 (Add more exceptions and descriptions as needed)
        """
        super().__init__(config)
        self.config = config

        # hierarchical Transformer encoder
        self.encoder = SegformerEncoder(config)

        # Initialize weights and apply final processing
        self.post_init()

    def _prune_heads(self, heads_to_prune):
        """
        Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base
        class PreTrainedModel
        """
        for layer, heads in heads_to_prune.items():
            self.encoder.layer[layer].attention.prune_heads(heads)

    def forward(
        self,
        pixel_values: mindspore.Tensor,
        output_attentions: Optional[bool] = None,
        output_hidden_states: Optional[bool] = None,
        return_dict: Optional[bool] = None,
    ) -> Union[Tuple, BaseModelOutput]:
        """
        Constructs the SegformerModel.

        Args:
            self: The instance of the SegformerModel class.
            pixel_values (mindspore.Tensor): The input tensor containing pixel values.
                Shape: (batch_size, num_channels, image_height, image_width).
            output_attentions (Optional[bool], optional): Whether to include attention weights in the output.
                If not provided, it defaults to the value specified in the model's configuration.
                Defaults to None.
            output_hidden_states (Optional[bool], optional): Whether to include hidden states in the output.
                If not provided, it defaults to the value specified in the model's configuration.
                Defaults to None.
            return_dict (Optional[bool], optional): Whether to return outputs as a BaseModelOutput dictionary.
                If not provided, it defaults to the value specified in the model's configuration.
                Defaults to None.

        Returns:
            Union[Tuple, BaseModelOutput]: The output of the SegformerModel. If `return_dict` is False,
                it returns a tuple containing the sequence output and the encoder outputs.
                If `return_dict` is True, it returns a BaseModelOutput object with the following attributes:

                - last_hidden_state (mindspore.Tensor): The sequence output of the model.
                Shape: (batch_size, sequence_length, hidden_size).
                - hidden_states (Tuple[mindspore.Tensor]): The hidden states of all layers.
                Each tensor has shape (batch_size, sequence_length, hidden_size).
                - attentions (Tuple[mindspore.Tensor]): The attention weights of all layers.
                Each tensor has shape (batch_size, num_attention_heads, sequence_length, sequence_length).

        Raises:
            None
        """
        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
        output_hidden_states = (
            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
        )
        return_dict = return_dict if return_dict is not None else self.config.use_return_dict

        encoder_outputs = self.encoder(
            pixel_values,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
        )
        sequence_output = encoder_outputs[0]

        if not return_dict:
            return (sequence_output,) + encoder_outputs[1:]

        return BaseModelOutput(
            last_hidden_state=sequence_output,
            hidden_states=encoder_outputs.hidden_states,
            attentions=encoder_outputs.attentions,
        )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerModel.__init__(config)

Initializes an instance of the SegformerModel class.

PARAMETER	DESCRIPTION
self	The instance of the SegformerModel class.
config	A dictionary containing configuration parameters for initializing the SegformerModel. It should include the necessary configuration settings for the model. Required keys and their datatypes: key1 (datatype): Description. key2 (datatype): Description. ... (Add more keys and descriptions as needed) TYPE: dict

RETURNS	DESCRIPTION
	None.

RAISES	DESCRIPTION
many exceptions	Any exceptions that may be raised during the initialization process should be documented here: ExampleException: Description of the example exception that may be raised. AnotherException: Description of another exception that may be raised. (Add more exceptions and descriptions as needed)

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config):
    """
    Initializes an instance of the SegformerModel class.

    Args:
        self: The instance of the SegformerModel class.
        config (dict):
            A dictionary containing configuration parameters for initializing the SegformerModel.
            It should include the necessary configuration settings for the model.
            Required keys and their datatypes:

            - key1 (datatype): Description.
            - key2 (datatype): Description.
            - ...
            - (Add more keys and descriptions as needed)

    Returns:
        None.

    Raises:
        many exceptions:
            Any exceptions that may be raised during the initialization process should be documented here:

            - ExampleException: Description of the example exception that may be raised.
            - AnotherException: Description of another exception that may be raised.
             (Add more exceptions and descriptions as needed)
    """
    super().__init__(config)
    self.config = config

    # hierarchical Transformer encoder
    self.encoder = SegformerEncoder(config)

    # Initialize weights and apply final processing
    self.post_init()

mindnlp.transformers.models.segformer.modeling_segformer.SegformerModel.forward(pixel_values, output_attentions=None, output_hidden_states=None, return_dict=None)

Constructs the SegformerModel.

PARAMETER	DESCRIPTION
self	The instance of the SegformerModel class.
pixel_values	The input tensor containing pixel values. Shape: (batch_size, num_channels, image_height, image_width). TYPE: Tensor
output_attentions	Whether to include attention weights in the output. If not provided, it defaults to the value specified in the model's configuration. Defaults to None. TYPE: Optional[bool] DEFAULT: None
output_hidden_states	Whether to include hidden states in the output. If not provided, it defaults to the value specified in the model's configuration. Defaults to None. TYPE: Optional[bool] DEFAULT: None
return_dict	Whether to return outputs as a BaseModelOutput dictionary. If not provided, it defaults to the value specified in the model's configuration. Defaults to None. TYPE: Optional[bool] DEFAULT: None

RETURNS

DESCRIPTION

Union[Tuple, BaseModelOutput]

Union[Tuple, BaseModelOutput]: The output of the SegformerModel. If return_dict is False, it returns a tuple containing the sequence output and the encoder outputs. If return_dict is True, it returns a BaseModelOutput object with the following attributes: last_hidden_state (mindspore.Tensor): The sequence output of the model. Shape: (batch_size, sequence_length, hidden_size). hidden_states (Tuple[mindspore.Tensor]): The hidden states of all layers. Each tensor has shape (batch_size, sequence_length, hidden_size). attentions (Tuple[mindspore.Tensor]): The attention weights of all layers. Each tensor has shape (batch_size, num_attention_heads, sequence_length, sequence_length).

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(
    self,
    pixel_values: mindspore.Tensor,
    output_attentions: Optional[bool] = None,
    output_hidden_states: Optional[bool] = None,
    return_dict: Optional[bool] = None,
) -> Union[Tuple, BaseModelOutput]:
    """
    Constructs the SegformerModel.

    Args:
        self: The instance of the SegformerModel class.
        pixel_values (mindspore.Tensor): The input tensor containing pixel values.
            Shape: (batch_size, num_channels, image_height, image_width).
        output_attentions (Optional[bool], optional): Whether to include attention weights in the output.
            If not provided, it defaults to the value specified in the model's configuration.
            Defaults to None.
        output_hidden_states (Optional[bool], optional): Whether to include hidden states in the output.
            If not provided, it defaults to the value specified in the model's configuration.
            Defaults to None.
        return_dict (Optional[bool], optional): Whether to return outputs as a BaseModelOutput dictionary.
            If not provided, it defaults to the value specified in the model's configuration.
            Defaults to None.

    Returns:
        Union[Tuple, BaseModelOutput]: The output of the SegformerModel. If `return_dict` is False,
            it returns a tuple containing the sequence output and the encoder outputs.
            If `return_dict` is True, it returns a BaseModelOutput object with the following attributes:

            - last_hidden_state (mindspore.Tensor): The sequence output of the model.
            Shape: (batch_size, sequence_length, hidden_size).
            - hidden_states (Tuple[mindspore.Tensor]): The hidden states of all layers.
            Each tensor has shape (batch_size, sequence_length, hidden_size).
            - attentions (Tuple[mindspore.Tensor]): The attention weights of all layers.
            Each tensor has shape (batch_size, num_attention_heads, sequence_length, sequence_length).

    Raises:
        None
    """
    output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
    output_hidden_states = (
        output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
    )
    return_dict = return_dict if return_dict is not None else self.config.use_return_dict

    encoder_outputs = self.encoder(
        pixel_values,
        output_attentions=output_attentions,
        output_hidden_states=output_hidden_states,
        return_dict=return_dict,
    )
    sequence_output = encoder_outputs[0]

    if not return_dict:
        return (sequence_output,) + encoder_outputs[1:]

    return BaseModelOutput(
        last_hidden_state=sequence_output,
        hidden_states=encoder_outputs.hidden_states,
        attentions=encoder_outputs.attentions,
    )

mindnlp.transformers.models.segformer.modeling_segformer.SegformerOverlapPatchEmbeddings

Bases: Module

Construct the overlapping patch embeddings.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerOverlapPatchEmbeddings(nn.Module):
    """Construct the overlapping patch embeddings."""
    def __init__(self, patch_size, stride, num_channels, hidden_size):
        """Initialize the SegformerOverlapPatchEmbeddings class.

            Args:
                self: The object instance.
                patch_size (int): The size of the patches used for the convolutional layer.
                stride (int): The stride value for the convolutional layer.
                num_channels (int): The number of input channels.
                hidden_size (int): The number of output channels.

            Returns:
                None

            Raises:
                None
            """
        super().__init__()
        self.proj = nn.Conv2d(
            num_channels,
            hidden_size,
            kernel_size=patch_size,
            stride=stride,
            padding=patch_size // 2,
            pad_mode='pad',
            bias=True
        )

        self.layer_norm = nn.LayerNorm(hidden_size)

    def forward(self, pixel_values):
        """
        Constructs the overlap patch embeddings for the input pixel values.

        Args:
            self (SegformerOverlapPatchEmbeddings): An instance of the SegformerOverlapPatchEmbeddings class.
            pixel_values (torch.Tensor): A tensor representing the input pixel values.
                The shape of the tensor should be (batch_size, channels, height, width).

        Returns:
            tuple:
                A tuple containing the following elements:

                - embeddings (torch.Tensor): A tensor representing the forwarded embeddings.
                The shape of the tensor will be (batch_size, num_patches, embedding_dim).
                - height (int): The height of the embeddings tensor.
                - width (int): The width of the embeddings tensor.

        Raises:
            None.

        Note:
            - The 'proj' method referred to in the code should be a method defined in the
            SegformerOverlapPatchEmbeddings class.
            - The 'layer_norm' method referred to in the code should be a method defined in the
            SegformerOverlapPatchEmbeddings class.
        """
        embeddings = self.proj(pixel_values)
        _, _, height, width = embeddings.shape
        # (batch_size, num_channels, height, width) -> (batch_size, num_channels, height*width) -> (batch_size, height*width, num_channels)
        # this can be fed to a Transformer layer
        embeddings = embeddings.flatten(start_dim=2).swapaxes(1, 2)
        embeddings = self.layer_norm(embeddings)
        return embeddings, height, width

mindnlp.transformers.models.segformer.modeling_segformer.SegformerOverlapPatchEmbeddings.__init__(patch_size, stride, num_channels, hidden_size)

Initialize the SegformerOverlapPatchEmbeddings class.

PARAMETER	DESCRIPTION
self	The object instance.
patch_size	The size of the patches used for the convolutional layer. TYPE: int
stride	The stride value for the convolutional layer. TYPE: int
num_channels	The number of input channels. TYPE: int
hidden_size	The number of output channels. TYPE: int

RETURNS	DESCRIPTION
	None

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, patch_size, stride, num_channels, hidden_size):
    """Initialize the SegformerOverlapPatchEmbeddings class.

        Args:
            self: The object instance.
            patch_size (int): The size of the patches used for the convolutional layer.
            stride (int): The stride value for the convolutional layer.
            num_channels (int): The number of input channels.
            hidden_size (int): The number of output channels.

        Returns:
            None

        Raises:
            None
        """
    super().__init__()
    self.proj = nn.Conv2d(
        num_channels,
        hidden_size,
        kernel_size=patch_size,
        stride=stride,
        padding=patch_size // 2,
        pad_mode='pad',
        bias=True
    )

    self.layer_norm = nn.LayerNorm(hidden_size)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerOverlapPatchEmbeddings.forward(pixel_values)

Constructs the overlap patch embeddings for the input pixel values.

PARAMETER	DESCRIPTION
self	An instance of the SegformerOverlapPatchEmbeddings class. TYPE: SegformerOverlapPatchEmbeddings
pixel_values	A tensor representing the input pixel values. The shape of the tensor should be (batch_size, channels, height, width). TYPE: Tensor

RETURNS	DESCRIPTION
tuple	A tuple containing the following elements: embeddings (torch.Tensor): A tensor representing the forwarded embeddings. The shape of the tensor will be (batch_size, num_patches, embedding_dim). height (int): The height of the embeddings tensor. width (int): The width of the embeddings tensor.

Note

• The 'proj' method referred to in the code should be a method defined in the SegformerOverlapPatchEmbeddings class.
• The 'layer_norm' method referred to in the code should be a method defined in the SegformerOverlapPatchEmbeddings class.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, pixel_values):
    """
    Constructs the overlap patch embeddings for the input pixel values.

    Args:
        self (SegformerOverlapPatchEmbeddings): An instance of the SegformerOverlapPatchEmbeddings class.
        pixel_values (torch.Tensor): A tensor representing the input pixel values.
            The shape of the tensor should be (batch_size, channels, height, width).

    Returns:
        tuple:
            A tuple containing the following elements:

            - embeddings (torch.Tensor): A tensor representing the forwarded embeddings.
            The shape of the tensor will be (batch_size, num_patches, embedding_dim).
            - height (int): The height of the embeddings tensor.
            - width (int): The width of the embeddings tensor.

    Raises:
        None.

    Note:
        - The 'proj' method referred to in the code should be a method defined in the
        SegformerOverlapPatchEmbeddings class.
        - The 'layer_norm' method referred to in the code should be a method defined in the
        SegformerOverlapPatchEmbeddings class.
    """
    embeddings = self.proj(pixel_values)
    _, _, height, width = embeddings.shape
    # (batch_size, num_channels, height, width) -> (batch_size, num_channels, height*width) -> (batch_size, height*width, num_channels)
    # this can be fed to a Transformer layer
    embeddings = embeddings.flatten(start_dim=2).swapaxes(1, 2)
    embeddings = self.layer_norm(embeddings)
    return embeddings, height, width

mindnlp.transformers.models.segformer.modeling_segformer.SegformerPreTrainedModel

Bases: PreTrainedModel

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerPreTrainedModel(PreTrainedModel):
    """
    An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained
    models.
    """
    config_class = SegformerConfig
    base_model_prefix = "segformer"
    main_input_name = "pixel_values"

    def _init_weights(self, cell):
        """Initialize the weights"""
        if isinstance(cell, (nn.Linear, nn.Conv2d)):
            # Slightly different from the TF version which uses truncated_normal for initialization
            # cf https://github.com/pytorch/pytorch/pull/5617
            cell.weight.initialize(Normal(self.config.initializer_range))
            if cell.bias is not None:
                cell.bias.initialize('zeros')
        elif isinstance(cell, nn.Embedding):
            data = np.random.normal(0.0, self.config.initializer_range, size=cell.weight.shape)
            if cell.padding_idx is not None:
                data[cell.padding_idx] = 0
            cell.weight.set_data(Tensor(data, cell.weight.dtype))
        elif isinstance(cell, nn.LayerNorm):
            cell.bias.initialize('zeros')
            cell.weight.initialize('ones')

mindnlp.transformers.models.segformer.modeling_segformer.SegformerSelfOutput

Bases: Module

This class represents the self-output of a segmenter transformer model (Segformer) in a neural network architecture. It inherits from the nn.Module class.

ATTRIBUTE	DESCRIPTION
dense	A fully connected layer that applies linear transformation to the input hidden states. TYPE: Linear
dropout	A dropout layer that randomly zeros some of the elements of the input tensor. TYPE: Dropout

METHOD	DESCRIPTION
__init__	Initializes an instance of the SegformerSelfOutput class.
forward	Constructs the self-output of the Segformer model.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

class SegformerSelfOutput(nn.Module):

    """
    This class represents the self-output of a segmenter transformer model (Segformer) in a neural network architecture.
    It inherits from the nn.Module class.

    Attributes:
        dense (nn.Linear): A fully connected layer that applies linear transformation to the input hidden states.
        dropout (nn.Dropout): A dropout layer that randomly zeros some of the elements of the input tensor.

    Methods:
        __init__: Initializes an instance of the SegformerSelfOutput class.
        forward: Constructs the self-output of the Segformer model.

    """
    def __init__(self, config, hidden_size):
        """
        Initializes an instance of the SegformerSelfOutput class.

        Args:
            self: The instance of the class.
            config (object): The configuration object containing various settings.
            hidden_size (int): The size of the hidden layer.

        Returns:
            None.

        Raises:
            None.

        Description:
            This method is called when a new instance of the SegformerSelfOutput class is created.
            It initializes the instance by setting up the necessary components for self-attention and output computation.

            The 'config' parameter is an object that stores various settings and configurations for the model.
            It is used to access the hidden dropout probability, which is used in the dropout layer. The 'hidden_size'
            parameter specifies the size of the hidden layer in the model.

            Inside the method, the 'super().__init__()' statement calls the __init__() method of the parent class to ensure
            proper initialization.

            The 'self.dense' attribute is an instance of the nn.Linear class, which represents a fully connected layer.
            It takes the 'hidden_size' as both the input and output size. This layer is used for self-attention computation.

            The 'self.dropout' attribute is an instance of the nn.Dropout class. It takes the 'config.hidden_dropout_prob'
            as the dropout probability. This layer is used for regularization during training to prevent overfitting.

            Note that this method does not perform any computations and is solely responsible for setting up the necessary
            components for the SegformerSelfOutput class.
        """
        super().__init__()
        self.dense = nn.Linear(hidden_size, hidden_size)
        self.dropout = nn.Dropout(p=config.hidden_dropout_prob)

    def forward(self, hidden_states, input_tensor):
        """
        Constructs the output of the SegformerSelfOutput class.

        Args:
            self (SegformerSelfOutput): An instance of the SegformerSelfOutput class.
            hidden_states (tensor): The hidden states of the self-attention mechanism.
                These states are passed through a dense layer and a dropout layer.
            input_tensor (tensor): The input tensor to the self-attention mechanism.

        Returns:
            None.

        Raises:
            None.
        """
        hidden_states = self.dense(hidden_states)
        hidden_states = self.dropout(hidden_states)
        return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.SegformerSelfOutput.__init__(config, hidden_size)

Initializes an instance of the SegformerSelfOutput class.

PARAMETER	DESCRIPTION
self	The instance of the class.
config	The configuration object containing various settings. TYPE: object
hidden_size	The size of the hidden layer. TYPE: int

RETURNS	DESCRIPTION
	None.

Description

This method is called when a new instance of the SegformerSelfOutput class is created. It initializes the instance by setting up the necessary components for self-attention and output computation.

The 'config' parameter is an object that stores various settings and configurations for the model. It is used to access the hidden dropout probability, which is used in the dropout layer. The 'hidden_size' parameter specifies the size of the hidden layer in the model.

Inside the method, the 'super().init()' statement calls the init() method of the parent class to ensure proper initialization.

The 'self.dense' attribute is an instance of the nn.Linear class, which represents a fully connected layer. It takes the 'hidden_size' as both the input and output size. This layer is used for self-attention computation.

The 'self.dropout' attribute is an instance of the nn.Dropout class. It takes the 'config.hidden_dropout_prob' as the dropout probability. This layer is used for regularization during training to prevent overfitting.

Note that this method does not perform any computations and is solely responsible for setting up the necessary components for the SegformerSelfOutput class.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def __init__(self, config, hidden_size):
    """
    Initializes an instance of the SegformerSelfOutput class.

    Args:
        self: The instance of the class.
        config (object): The configuration object containing various settings.
        hidden_size (int): The size of the hidden layer.

    Returns:
        None.

    Raises:
        None.

    Description:
        This method is called when a new instance of the SegformerSelfOutput class is created.
        It initializes the instance by setting up the necessary components for self-attention and output computation.

        The 'config' parameter is an object that stores various settings and configurations for the model.
        It is used to access the hidden dropout probability, which is used in the dropout layer. The 'hidden_size'
        parameter specifies the size of the hidden layer in the model.

        Inside the method, the 'super().__init__()' statement calls the __init__() method of the parent class to ensure
        proper initialization.

        The 'self.dense' attribute is an instance of the nn.Linear class, which represents a fully connected layer.
        It takes the 'hidden_size' as both the input and output size. This layer is used for self-attention computation.

        The 'self.dropout' attribute is an instance of the nn.Dropout class. It takes the 'config.hidden_dropout_prob'
        as the dropout probability. This layer is used for regularization during training to prevent overfitting.

        Note that this method does not perform any computations and is solely responsible for setting up the necessary
        components for the SegformerSelfOutput class.
    """
    super().__init__()
    self.dense = nn.Linear(hidden_size, hidden_size)
    self.dropout = nn.Dropout(p=config.hidden_dropout_prob)

mindnlp.transformers.models.segformer.modeling_segformer.SegformerSelfOutput.forward(hidden_states, input_tensor)

Constructs the output of the SegformerSelfOutput class.

PARAMETER	DESCRIPTION
self	An instance of the SegformerSelfOutput class. TYPE: SegformerSelfOutput
hidden_states	The hidden states of the self-attention mechanism. These states are passed through a dense layer and a dropout layer. TYPE: tensor
input_tensor	The input tensor to the self-attention mechanism. TYPE: tensor

RETURNS	DESCRIPTION
	None.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def forward(self, hidden_states, input_tensor):
    """
    Constructs the output of the SegformerSelfOutput class.

    Args:
        self (SegformerSelfOutput): An instance of the SegformerSelfOutput class.
        hidden_states (tensor): The hidden states of the self-attention mechanism.
            These states are passed through a dense layer and a dropout layer.
        input_tensor (tensor): The input tensor to the self-attention mechanism.

    Returns:
        None.

    Raises:
        None.
    """
    hidden_states = self.dense(hidden_states)
    hidden_states = self.dropout(hidden_states)
    return hidden_states

mindnlp.transformers.models.segformer.modeling_segformer.drop_path(input, drop_prob=0.0, training=False)

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument.

Source code in mindnlp/transformers/models/segformer/modeling_segformer.py

def drop_path(input: mindspore.Tensor, drop_prob: float = 0.0, training: bool = False) -> mindspore.Tensor:
    """
    Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

    Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks,
    however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper...
    See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the
    layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the
    argument.
    """
    if drop_prob == 0.0 or not training:
        return input
    keep_prob = 1 - drop_prob
    shape = (input.shape[0],) + (1,) * (input.ndim - 1)  # work with diff dim tensors, not just 2D ConvNets
    random_tensor = keep_prob + ops.rand(shape, dtype=input.dtype)
    random_tensor = random_tensor.floor()  # binarize
    output = input.div(keep_prob) * random_tensor
    return output