Supported Models

The table below represents the current support in the library for each of those models, whether they have support in Pynative mode or Graph mode.

Model Pynative support Graph Support
ALBERT
ALIGN
AltCLIP
Audio Spectrogram Transformer
Autoformer ✅ (Inference only)
BaiChuan
Bark
BART
Barthez
Bartpho
BEIT
BERT
BERT Generation
BERT Japanese
BERTweet
BGE M3
Bigbird
Bigbird Pegasus
BioGPT
Bit
Blenderbot
Blenderbot Small
BLIP
BLIP2
BLOOM
BridgeTower
Bros
ByT5
ChatGLM
ChatGLM2
ChatGLM3
CLIP
CodeGen
ConvBERT
ConvNext
CVT
CPM
CPM-Ant
CPM-Bee
MiniCPM
Deberta
Electra TODO
EnCodec
ERNIE
ERNIEM
ESM
Falcon
OpenAI GPT
OpenAI GPT-2
Gemma
GPT Neo
GPT NeoX
GPT Pangu
GPTBigCode
Graphormer
Hubert
Jamba
JetMoE
Llama
Llama2
Llava
Llava_next
CodeLlama
Longformer
LongT5
LUKE
MaskFormer
Mamba
mBART-50
Megatron-BERT
Megatron-GPT2
Mistral
Mixtral
MobileBERT
Moss
MusicGen
MusicGen Melody
Nezha
Olmo
OpenELM
OPT
Phi2
Phi3
Pagasus
Pop2piano
Qwen2
Qwen2Moe
Reformer
RegNet Todo
ResNet
RoBERTa
RWKV
SAM
SeamlessM4T
SeamlessM4Tv2
Segformer
StarCoder
T5
Timesformer
Tinybert
Wav2vec
VAN
VipLlava
Whisper
XLM
XLM-RoBERTa
XLNet