autodistill / autodistill-grounded-edgesam

EdgeSAM model for use with Autodistill.

Autodistill Grounded EdgeSAM Module

This repository contains the code supporting the Grounded EdgeSAM base model for use with Autodistill.

EdgeSAM, introduced in the "EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM" paper, is a faster version of the Segment Anything model.

Grounded EdgeSAM combines Grounding DINO and EdgeSAM, allowing you to identify objects and generate segmentation masks for them.

Read the full Autodistill documentation to learn more about Autodistill.

Installation

To use Grounded EdgeSAM with autodistill, you need to install the following dependency:

pip3 install autodistill-grounded-edgesam

Quickstart

from autodistill_clip import CLIP

# define an ontology to map class names to our GroundingDINO prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
from autodistill_grounded_edgesam import GroundedEdgeSAM
from autodistill.detection import CaptionOntology
from autodistill.utils import plot
import cv2

# define an ontology to map class names to our GroundedSAM prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
base_model = GroundedEdgeSAM(
    ontology=CaptionOntology(
        {
            "person": "person",
            "forklift": "forklift",
        }
    )
)

# run inference on a single image
results = base_model.predict("logistics.jpeg")

plot(
    image=cv2.imread("logistics.jpeg"),
    classes=base_model.ontology.classes(),
    detections=results
)

# label a folder of images
base_model.label("./context_images", extension=".jpeg")

License

This repository is released under an S-Lab License 1.0 license.

🏆 Contributing

We love your input! Please see the core Autodistill contributing guide to get started. Thank you 🙏 to all our contributors!

autodistill-grounded-edgesam's Issues

Key error initializing GroundedEdgeSAM

I receive the following error.

base_model = GroundedEdgeSAM(
... ontology=CaptionOntology(
... {
... "person": "person",
... "forklift": "forklift",
... }
... )
... )
Loading EdgeSAM...
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\mpatm\AppData\Local\Programs\Python\Python310\lib\site-packages\autodistill_grounded_edgesam\grounded_edgesam_model.py", line 77, in init
self.predictor = check_dependencies()
File "C:\Users\mpatm\AppData\Local\Programs\Python\Python310\lib\site-packages\autodistill_grounded_edgesam\grounded_edgesam_model.py", line 59, in check_dependencies
sam = sam_model_registrySAM_ENCODER_VERSION.to(
KeyError: 'edge_sam'

I've confirmed that, indeed, the sam_model_registry dictionary does not contain a key named edge_sam. This code ran without error the first time I ran it, but not the second:

https://github.com/autodistill/autodistill-grounded-edgesam?ref=blog.roboflow.com

I worry that the automatic installs in grounded_edgesam_model.py messed up my configuration. It seems like a bad idea to embed clones and installs in that script. It would be better to trust your users to do the appropriate installs, at most raising an error if there is an issue; this is in line with pythonic principles.