Various utility functions used by the blurr package.
torch.cuda.set_device(1)
print(f'Using GPU #{torch.cuda.current_device()}: {torch.cuda.get_device_name()}')
Using GPU #1: GeForce GTX 1080 Ti

str_to_class[source]

str_to_class(classname)

converts string representation to class

class Singleton[source]

Singleton()

Singleton functions as python decorator. Use this above any class to turn that class into a singleton (see here for more info on the singleton pattern).

@Singleton
class TestSingleton: pass

a = TestSingleton()
b = TestSingleton()
test_eq(a,b)

ModelHelper

Singleton object at 0x7fbfa4a70550>[source]

Singleton object at 0x7fbfa4a70550>(*args, **kwargs)

ModelHelper is a Singleton (there exists only one instance, and the same instance is returned upon subsequent instantiation requests). You can get at via the BLURR_MODEL_HELPER constant below.

mh = ModelHelper()
mh2 = ModelHelper()
test_eq(mh, mh2)

Provide global helper constant

Users of this library can simply use BLURR_MODEL_HELPER to access all the ModelHelper capabilities without having to fetch an instance themselves.

Here's how you can get at the core huggingface objects you need to work with ...

... the task

ModelHelper.get_tasks[source]

ModelHelper.get_tasks(arch=None)

Get the type of tasks for which there is a custom model for (optional: by architecture). There are a number of customized models built for specific tasks like token classification, question/answering, LM, etc....

print(BLURR_MODEL_HELPER.get_tasks())
print('')
print(BLURR_MODEL_HELPER.get_tasks('bart'))
['CausalLM', 'Classification', 'ConditionalGeneration', 'Generation', 'LMHead', 'LMHeadModel', 'MaskedLM', 'MultipleChoice', 'NextSentencePrediction', 'PreTraining', 'QuestionAnswering', 'QuestionAnsweringSimple', 'Seq2SeqLM', 'SequenceClassification', 'TokenClassification']

['ConditionalGeneration', 'QuestionAnswering', 'SequenceClassification']

... the architecture

ModelHelper.get_architectures[source]

ModelHelper.get_architectures()

Used to get all the architectures supported by your Transformers install

print(BLURR_MODEL_HELPER.get_architectures())
['albert', 'albert_fast', 'auto', 'bart', 'bart_fast', 'bert', 'bert_fast', 'bert_generation', 'bert_japanese', 'bertweet', 'blenderbot', 'camembert', 'camembert_fast', 'ctrl', 'deberta', 'distilbert', 'distilbert_fast', 'dpr', 'dpr_fast', 'electra', 'electra_fast', 'encoder_decoder', 'flaubert', 'fsmt', 'funnel', 'funnel_fast', 'gpt2', 'gpt2_fast', 'herbert', 'herbert_fast', 'layoutlm', 'layoutlm_fast', 'longformer', 'longformer_fast', 'lxmert', 'lxmert_fast', 'marian', 'mbart', 'mbart_fast', 'mmbt', 'mobilebert', 'mobilebert_fast', 'openai', 'openai_fast', 'pegasus', 'pegasus_fast', 'phobert', 'prophetnet', 'rag', 'reformer', 'reformer_fast', 'retribert', 'retribert_fast', 'roberta', 'roberta_fast', 'squeezebert', 'squeezebert_fast', 't5', 't5_fast', 'transfo_xl', 'utils_base', 'utils_fast', 'xlm', 'xlm_prophetnet', 'xlm_roberta', 'xlm_roberta_fast', 'xlnet', 'xlnet_fast']

ModelHelper.get_model_architecture[source]

ModelHelper.get_model_architecture(model_name_or_enum)

Get the architecture for a given model name / enum

print(BLURR_MODEL_HELPER.get_model_architecture('RobertaForSequenceClassification'))
roberta

... the config for that particular task and architecture

ModelHelper.get_config[source]

ModelHelper.get_config(arch)

Used the locate the name of the configuration class for a given architecture

print(BLURR_MODEL_HELPER.get_config('bert'))
<class 'transformers.configuration_bert.BertConfig'>

... the available tokenizers for that architecture

ModelHelper.get_tokenizers[source]

ModelHelper.get_tokenizers(arch)

Used to get the available huggingface tokenizers for a given architecture. Note: There may be multiple tokenizers and so this returns a list.

print(BLURR_MODEL_HELPER.get_tokenizers('electra'))
[<class 'transformers.tokenization_electra.ElectraTokenizer'>]

... and lastly the models (optionally for a given task and/or architecture)

ModelHelper.get_models[source]

ModelHelper.get_models(arch=None, task=None)

The transformer models available for use (optional: by architecture | task)

print(L(BLURR_MODEL_HELPER.get_models())[:5])
[<class 'transformers.modeling_transfo_xl.AdaptiveEmbedding'>, <class 'transformers.modeling_albert.AlbertForMaskedLM'>, <class 'transformers.modeling_albert.AlbertForMultipleChoice'>, <class 'transformers.modeling_albert.AlbertForPreTraining'>, <class 'transformers.modeling_albert.AlbertForQuestionAnswering'>]
print(BLURR_MODEL_HELPER.get_models(arch='bert')[:5])
[<class 'transformers.modeling_bert.BertForMaskedLM'>, <class 'transformers.modeling_bert.BertForMultipleChoice'>, <class 'transformers.modeling_bert.BertForNextSentencePrediction'>, <class 'transformers.modeling_bert.BertForPreTraining'>, <class 'transformers.modeling_bert.BertForQuestionAnswering'>]
print(BLURR_MODEL_HELPER.get_models(task='TokenClassification')[:5])
[<class 'transformers.modeling_albert.AlbertForTokenClassification'>, <class 'transformers.modeling_auto.AutoModelForTokenClassification'>, <class 'transformers.modeling_bert.BertForTokenClassification'>, <class 'transformers.modeling_camembert.CamembertForTokenClassification'>, <class 'transformers.modeling_distilbert.DistilBertForTokenClassification'>]
print(BLURR_MODEL_HELPER.get_models(arch='bert', task='TokenClassification'))
[<class 'transformers.modeling_bert.BertForTokenClassification'>]

Here we define some helpful enums to make it easier to get at the architecture and task you're looking for.

print(L(HF_ARCHITECTURES)[:5])
[<HF_ARCHITECTURES.albert: 1>, <HF_ARCHITECTURES.albert_fast: 2>, <HF_ARCHITECTURES.auto: 3>, <HF_ARCHITECTURES.bart: 4>, <HF_ARCHITECTURES.bart_fast: 5>]
print('--- all tasks ---')
print(L(HF_TASKS_ALL))
print('\n--- auto only ---')
print(L(HF_TASKS_AUTO))
--- all tasks ---
[<HF_TASKS_ALL.CausalLM: 1>, <HF_TASKS_ALL.Classification: 2>, <HF_TASKS_ALL.ConditionalGeneration: 3>, <HF_TASKS_ALL.Generation: 4>, <HF_TASKS_ALL.LMHead: 5>, <HF_TASKS_ALL.LMHeadModel: 6>, <HF_TASKS_ALL.MaskedLM: 7>, <HF_TASKS_ALL.MultipleChoice: 8>, <HF_TASKS_ALL.NextSentencePrediction: 9>, <HF_TASKS_ALL.PreTraining: 10>, <HF_TASKS_ALL.QuestionAnswering: 11>, <HF_TASKS_ALL.QuestionAnsweringSimple: 12>, <HF_TASKS_ALL.Seq2SeqLM: 13>, <HF_TASKS_ALL.SequenceClassification: 14>, <HF_TASKS_ALL.TokenClassification: 15>]

--- auto only ---
[<HF_TASKS_AUTO.CausalLM: 1>, <HF_TASKS_AUTO.LMHead: 2>, <HF_TASKS_AUTO.MaskedLM: 3>, <HF_TASKS_AUTO.MultipleChoice: 4>, <HF_TASKS_AUTO.NextSentencePrediction: 5>, <HF_TASKS_AUTO.PreTraining: 6>, <HF_TASKS_AUTO.QuestionAnswering: 7>, <HF_TASKS_AUTO.Seq2SeqLM: 8>, <HF_TASKS_AUTO.SequenceClassification: 9>, <HF_TASKS_AUTO.TokenClassification: 10>]
HF_TASKS_ALL.Classification
<HF_TASKS_ALL.Classification: 2>

BLURR_MODEL_HELPER.get_classes_for_model can be used to get the config, tokenizer, and model classes you want

ModelHelper.get_classes_for_model[source]

ModelHelper.get_classes_for_model(model_name_or_cls)

Get tokenizers, config, and model for a given model name / class

config, tokenizers, model = BLURR_MODEL_HELPER.get_classes_for_model('RobertaForSequenceClassification')

print(config)
print(tokenizers[0])
print(model)
<class 'transformers.configuration_roberta.RobertaConfig'>
<class 'transformers.tokenization_roberta.RobertaTokenizer'>
<class 'transformers.modeling_roberta.RobertaForSequenceClassification'>
config, tokenizers, model = BLURR_MODEL_HELPER.get_classes_for_model(DistilBertModel)

print(config)
print(tokenizers[0])
print(model)
<class 'transformers.configuration_distilbert.DistilBertConfig'>
<class 'transformers.tokenization_distilbert.DistilBertTokenizer'>
<class 'transformers.modeling_distilbert.DistilBertModel'>

Methods for loading pre-trained (configs, tokenizer, model) hugginface objects

ModelHelper.get_hf_objects[source]

ModelHelper.get_hf_objects(pretrained_model_name_or_path, task=None, config=None, tokenizer_cls=None, model_cls=None, config_kwargs={}, tokenizer_kwargs={}, model_kwargs={}, cache_dir=None)

Returns the architecture (str), config (obj), tokenizer (obj), and model (obj) given at minimum a pre-trained model name or path. Specify a task to ensure the right "AutoModelFor" is used to create the model.

Optionally, you can pass a config (obj), tokenizer (class), and/or model (class) (along with any related kwargs for each) to get as specific as you want w/r/t what huggingface objects are returned.

arch, config, tokenizer, model = BLURR_MODEL_HELPER.get_hf_objects("bert-base-cased-finetuned-mrpc",
                                                                   task=HF_TASKS_AUTO.MaskedLM)

print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
bert
<class 'transformers.configuration_bert.BertConfig'>
<class 'transformers.tokenization_bert.BertTokenizer'>
<class 'transformers.modeling_bert.BertForMaskedLM'>
arch, tokenizer, config, model = BLURR_MODEL_HELPER.get_hf_objects("fmikaelian/flaubert-base-uncased-squad",
                                                                   task=HF_TASKS_AUTO.QuestionAnswering)

print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
flaubert
<class 'transformers.tokenization_flaubert.FlaubertTokenizer'>
<class 'transformers.configuration_flaubert.FlaubertConfig'>
<class 'transformers.modeling_flaubert.FlaubertForQuestionAnsweringSimple'>
arch, tokenizer, config, model = BLURR_MODEL_HELPER.get_hf_objects("bert-base-cased-finetuned-mrpc",
                                                                   config=None,
                                                                   tokenizer_cls=BertTokenizer, 
                                                                   model_cls=BertForNextSentencePrediction)
print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
bert
<class 'transformers.tokenization_bert.BertTokenizer'>
<class 'transformers.configuration_bert.BertConfig'>
<class 'transformers.modeling_bert.BertForNextSentencePrediction'>

Cleanup