Segment Anything Model (SAM)Translation site

Segment Anything Model (SAM)Developed by Meta AI (formerly Facebook AI Research team)General Image Segmentation Basic ModelIts most striking feature is:Supports real-time interactive segmentation of any object, any scene, and any prompt.And with its massive training dataset SA-1B (more than 110,000 images and 1 billion segmentation masks), it has set a new benchmark in the industry.

• Official website:segment-anything.com
• Open source project repository:GitHub: facebookresearch/segment-anything
• AI training papers:arXiv:2304.02643

This project not only released authoritative model weights, a detailed SDK, and a demo website, but also published the largest dataset ever for visual segmentation, bringing tremendous impetus to both applications and academia.

1. Supports prompt-driven segmentation.

SAM can accept different types of prompts (such as...)Point selection, box selection, coarse mask, etc.This allows for flexible segmentation, which greatly enhances the user experience and model adaptability.

Common prompt types are as follows:

• Select (click Prompt).Target segmentation is achieved by clicking on a point in the image.
• Box PromptDraw a rectangle, and the model identifies the target within the rectangle.
• Mask PromptRoughly outline the region of interest, then AI refines it into precise segmentation.

2. Zero-Shot Generalization

SAM is trained using large-scale, multi-type data and has excellent zero-shot capability.It can automatically achieve accurate segmentation on images or object categories that have never been seen before, without the need for special “fine-tuning” or further training.

3. Support for multiple model versions

SAM offers models of different sizes (such as ViT-B, ViT-L, ViT-H) to adapt to different...Computing power environment and segmentation accuracy requirementsIt supports ideal flexible deployment.

Performance Comparison Table of Common Models(from)Official GitHub）：

Chat endlessly with AI characters and start your own story.

Interact with a vast array of 2D and 3D characters and experience truly unlimited AI role-playing dialogue. Join now! New users receive 6000 points upon login!

How to play?Try it for free now →

4. High-quality segmentation mask output

By fusing multi-scale visual features using the Transformer architectureSAM can automatically generate fine-grained segmentation masks with clear and usable boundaries.This significantly reduces the burden of manual corrections in the later stages.

5. Interactive Web Demo and API

In addition to the standard SDK, SAM also provides...Web-based DemoWith RESTful API, it allows users from different backgrounds to quickly try and integrate it.

For more content and demonstrations, please visit [website address]. SAM Official Demo Page。

Currently, the SAM project itselfCompletely open source and freeThe Meta team has open-sourced the model weights, training code, and examples, which developers and scholars worldwide can download and study for free.(Open source license details)。

Table: SAM Official Pricing and Solutions

Remark: For large-scale online API inference requirements, deployment on a third-party cloud AI company (such as AWS, Azure, or Huggingface) is recommended. Costs vary depending on the specific vendor and configuration; please refer to [link/reference needed].Deployment of Hub SAM on Huggingface。

SAM's official documentation and community tutorials provide developers with a variety of resources.Local deployment and API interfaceSelection. The basic process is as follows:

A. Installation and Environmental Preparation

1. Obtain source code and weight
   

   git clone https://github.com/facebookresearch/segment-anything.git

   
   

   Download ViT-B/L/H weight files as needed.

   
   
2. Install dependencies
   

   pip install -e segment-anything

   
   

B. Python Call Examples

from segment_anything import SamPredictor, sam_model_registry checkpoint = "./sam_vit_b_01ec64.pth" sam = sam_model_registry["vit_b"](checkpoint=checkpoint) # Select model type predictor = SamPredictor(sam) # Load image import cv2 image = cv2.imread('path/to/image.jpg') predictor.set_image(image) # Segment by point/box hints input_point = [[100, 150]] # [(x, y)] input_label = [1] # 1 is the foreground point masks, scores, logits = predictor.predict( point_coords=input_point, point_labels=input_label, multimask_output=True )

C. Online Web DEMO Segmentation

• Direct accessSegment Anything Demo
• Upload an image, select the “point selection” or “box selection” tool, and obtain the segmentation result.

D. Integrating Label Studio data annotation
SAM has been integrated into mainstream open-source annotation platforms.Label StudioThis significantly improves data annotation efficiency. For detailed steps, please refer to [link/reference needed].Official integration tutorial。

SAM's unique design makes it compatible with a very wide range of applications:

• AI/Vision ResearchersUsed to explore “zero-shot generalization” and overcome academic challenges such as multi-class/few-shot generalization.
• Computer vision/image recognition engineerSignificantly improves the efficiency and accuracy of automatic/semi-automatic annotation and segmentation schemes.
• Medical image analysis/bioinformatics expertIt can be used for cell and tissue cutting without complex optimization, saving on domain adaptation costs.
• Intelligent Manufacturing/Autonomous Driving R&D TeamRapidly achieve object segmentation in multiple scenes, optimizing data for subsequent perception or upstream AI tasks.
• AI Data Service CompanyBy integrating SAM into platforms such as Label Studio, we can create automated batch annotation pipelines for our clients.

Examples of applicable industry scenarios

SAM is able to achieve this“Segment Anything（任意分割）”，主要得益于以下三大创新模块：

• Image Encoder（图像编码器）：采用大规模Vision Transformer架构，有效提取图片的高维特征，为分割提供基础
• Prompt Encoder（提示编码器）：无论是点、框、掩码还是后续可扩展的‘文本’提示，均可得到相应特征向量
• Mask Decoder（掩码解码器）：高效推理生成多样、清晰的分割掩码，具备模型内置信息熵评估（可输出多个备选掩码）

阅读原论文详细技术方案

多年来，分割领域主流依然是U-Net、Mask R-CNN、DeepLab等，但它们普遍存在泛化能力弱、成本高、交互式支持差等问题。

对比表：SAM与主流分割AI训练模型

欲查看更多实际对比与业界反馈，请关注 Huggingface Segment Anything 页面 的开源测试讨论。

答：可以。SAM输出的掩码高质量、边界清晰，非常适合作为下游语义分割、实例分割、目标检测等任务的数据标注来源。已有众多数据标注平台（如Label Studio）将SAM集成为半自动掩码标注工具，有效提升了训练数据制作效率。

Label Studio调用SAM详细教程

answer:SAM的资源消耗与模型类型有关：

• ViT-B（Base）版：8GB及以上显存即可较流畅推理；
• ViT-L（Large）/ViT-H（Huge）版：建议12GB~24GB+显存
• 对CPU有限支持，也有社区版本的移动端（MobileSAM等）可用于低算力环境。

详细部署建议见官方GitHub说明

答：目前SAM原生版本主要支持点、框、掩码作为Prompt，正在研发文本提示功能。不过，业界已有团队将CLIP等文本-视觉模型与SAM结合，实现了“文本自动分割”能力，相关成果在Semantic Segment Anything (SSA)和CLIP-SAM等项目中有演示。

• SAM官方主页
• Meta官方博客
• SA-1B开放数据集介绍
• 教学文章 LearnOpenCV：《Segment Anything – A Foundation Model for Image Segmentation》
• Label Studio集成案例博客

Chat endlessly with AI characters and start your own story.

Interact with a vast array of 2D and 3D characters and experience truly unlimited AI role-playing dialogue. Join now! New users receive 6000 points upon login!

How to play?Try it for free now →