pytorch transformers ....

transformers的预训练模型下载到本地特定位置，默认是在~/.cache/huggingface/transformers

model = GPT2LMHeadModel.from_pretrained('gpt2', cache_dir="...")

想知道transformers的模型都是什么结构的，比如bert模型：

transformers/models/bert/__init__.py

这里可以看到导入了

from .modeling_bert import (            BERT_PRETRAINED_MODEL_ARCHIVE_LIST,            BertForMaskedLM,            BertForMultipleChoice,            BertForNextSentencePrediction,            BertForPreTraining,            BertForQuestionAnswering,            BertForSequenceClassification,            BertForTokenClassification,            BertLayer,            BertLMHeadModel,            BertModel,            BertPreTrainedModel,            load_tf_weights_in_bert,        )

然后点进去就可以看了，可以看他们的forward函数等