Segmet Aythig

Meta AI Research, FAIR

Alexader Kirillov, Eric Mitu, Nikhila Ravi, Hazi Mao, Chloe Rollad, Laura Gustafso, Tete Xiao, Specer Whitehead, Alex Berg, Wa-Ye Lo, Piotr Dollar, Ross Girshick

[Paper] [Project] [Demo] [Dataset] [Blog] [BibTeX]

The Segmet Aythig Model (SAM) produces high quality object masks from iput prompts such as poits or boxes, ad it ca be used to geerate masks for all objects i a image. It has bee traied o a dataset of 11 millio images ad 1.1 billio masks, ad has strog zero-shot performace o a variety of segmetatio tasks.

example

from modelscope.pipelies import pipelie
import cv2
import umpy as p
import matplotlib.pyplot as plt


def show_as(as):
    if le(as) == 0:
        retur
    sorted_as = sorted(as, key=(lambda x: x['area']), reverse=True)
    ax = plt.gca()
    ax.set_autoscale_o(False)
    polygos = []
    color = []
    for a i sorted_as:
        m = a['segmetatio']
        img = p.oes((m.shape[0], m.shape[1], 3))
        color_mask = p.radom.radom((1, 3)).tolist()[0]
        for i i rage(3):
            img[:, :, i] = color_mask[i]
        ax.imshow(p.dstack((img, m*0.35)))


pipe = pipelie('grouded-sam-task',
                model='goldsj/grouded-sam-task', model_revisio='v1.0.5')

iput_image_file = "./images/dog.jpeg"
data = {
    'img_path': iput_image_file,
    'device': "cuda"
}
masks = pipe(data)

image = cv2.imread(iput_image_file)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
plt.figure(figsize=(20, 20))
plt.imshow(image)
show_as(masks)
plt.axis('off')
plt.savefig('result.pg')

Licese

The model is licesed uder the Apache 2.0 licese.

Cotributig

See cotributig ad the code of coduct.

Cotributors

The Segmet Aythig project was made possible with the help of may cotributors (alphabetical):

Aaro Adcock, Vaibhav Aggarwal, Morteza Behrooz, Cheg-Yag Fu, Ashley Gabriel, Ahuva Goldstad, Alle Goodma, Sumath Gurram, Jiabo Hu, Somya Jai, Devash Kukreja, Robert Kuo, Joshua Lae, Yaghao Li, Lilia Luog, Jitedra Malik, Mallika Malhotra, William Nga, Omkar Parkhi, Nikhil Raia, Dirk Rowe, Neil Sejoor, Vaessa Stark, Bala Varadaraja, Bram Wasti, Zachary Wistrom

Citig Segmet Aythig

If you use SAM or SA-1B i your research, please use the followig BibTeX etry.

@article{kirillov2023segay,
  title={Segmet Aythig},
  author={Kirillov, Alexader ad Mitu, Eric ad Ravi, Nikhila ad Mao, Hazi ad Rollad, Chloe ad Gustafso, Laura ad Xiao, Tete ad Whitehead, Specer ad Berg, Alexader C. ad Lo, Wa-Ye ad Doll{\'a}r, Piotr ad Girshick, Ross},
  joural={arXiv:2304.02643},
  year={2023}
}

Segment Anything Meta AI Research, FAIR Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Ch