Distilling VIT-L from VIT-G on imagenet #374
-
Hi, How can modify the config files to Distill from a ViT-g model pretrained on Imagenet21k& finetuned on Imagenet 1k to a VIT-L model. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 25 replies
-
Hi @arpitsahni04 Do you just want to change models in some ImageNet configs?
For the teacher and student names, I referred to https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer.py Note that if you'll need to tune hyperparameters if you reuse existing yaml config files under |
Beta Was this translation helpful? Give feedback.
Hi @arpitsahni04
Do you just want to change models in some ImageNet configs?
If so, with the current PyPI package (torchdistill==0.3.3), your model entry in a yaml config will look like this as shown in README.md
For the teacher and student names, I referred to https://github.com/huggingface/pytorch-image-models/blob/main/…