Model card for covextv2_huge.fcmae

A CovNeXt-V2 self-supervised feature represetatio model. Pretraied with a fully covolutioal masked autoecoder framework (FCMAE). This model has o pretraied head ad is oly useful for fie-tue or feature extractio.

Model Details

• Model Type: Image classificatio / feature backboe
• Model Stats:
• Params (M): 657.5
• GMACs: 115.0
• Activatios (M): 79.1
• Image size: 224 x 224
• Papers:
• CovNeXt V2: Co-desigig ad Scalig CovNets with Masked Autoecoders: https://arxiv.org/abs/2301.00808
• Origial: https://github.com/facebookresearch/CovNeXt-V2
• Pretrai Dataset: ImageNet-1k

Model Usage

Image Classificatio

from urllib.request import urlope
from PIL import Image
import timm

img = Image.ope(urlope(
    'https://huggigface.co/datasets/huggigface/documetatio-images/resolve/mai/beigets-task-guide.pg'
))

model = timm.create_model('covextv2_huge.fcmae', pretraied=True)
model = model.eval()

# get model specific trasforms (ormalizatio, resize)
data_cofig = timm.data.resolve_model_data_cofig(model)
trasforms = timm.data.create_trasform(**data_cofig, is_traiig=False)

output = model(trasforms(img).usqueeze(0))  # usqueeze sigle image ito batch of 1

top5_probabilities, top5_class_idices = torch.topk(output.softmax(dim=1) * 100, k=5)

Feature Map Extractio

from urllib.request import urlope
from PIL import Image
import timm

img = Image.ope(urlope(
    'https://huggigface.co/datasets/huggigface/documetatio-images/resolve/mai/beigets-task-guide.pg'
))

model = timm.create_model(
    'covextv2_huge.fcmae',
    pretraied=True,
    features_oly=True,
)
model = model.eval()

# get model specific trasforms (ormalizatio, resize)
data_cofig = timm.data.resolve_model_data_cofig(model)
trasforms = timm.data.create_trasform(**data_cofig, is_traiig=False)

output = model(trasforms(img).usqueeze(0))  # usqueeze sigle image ito batch of 1

for o i output:
    # prit shape of each feature map i output
    # e.g.:
    #  torch.Size([1, 352, 56, 56])
    #  torch.Size([1, 704, 28, 28])
    #  torch.Size([1, 1408, 14, 14])
    #  torch.Size([1, 2816, 7, 7])

    prit(o.shape)

Image Embeddigs

from urllib.request import urlope
from PIL import Image
import timm

img = Image.ope(urlope(
    'https://huggigface.co/datasets/huggigface/documetatio-images/resolve/mai/beigets-task-guide.pg'
))

model = timm.create_model(
    'covextv2_huge.fcmae',
    pretraied=True,
    um_classes=0,  # remove classifier .Liear
)
model = model.eval()

# get model specific trasforms (ormalizatio, resize)
data_cofig = timm.data.resolve_model_data_cofig(model)
trasforms = timm.data.create_trasform(**data_cofig, is_traiig=False)

output = model(trasforms(img).usqueeze(0))  # output is (batch_size, um_features) shaped tesor

# or equivaletly (without eedig to set um_classes=0)

output = model.forward_features(trasforms(img).usqueeze(0))
# output is upooled, a (1, 2816, 7, 7) shaped tesor

output = model.forward_head(output, pre_logits=True)
# output is a (1, um_features) shaped tesor

Model Compariso

Explore the dataset ad rutime metrics of this model i timm model results.

All timig umbers from eager model PyTorch 1.13 o RTX 3090 w/ AMP.

Citatio

@article{Woo2023CovNeXtV2,
  title={CovNeXt V2: Co-desigig ad Scalig CovNets with Masked Autoecoders},
  author={Saghyu Woo, Shoubhik Debath, Roghag Hu, Xilei Che, Zhuag Liu, I So Kweo ad Saiig Xie},
  year={2023},
  joural={arXiv preprit arXiv:2301.00808},
}

@misc{rw2019timm,
  author = {Ross Wightma},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  joural = {GitHub repository},
  doi = {10.5281/zeodo.4414861},
  howpublished = {\url{https://github.com/huggigface/pytorch-image-models}}
}

Model card for convnextv2_huge.fcmae A ConvNeXt-V2 self-supervised feature representation model. Pre