Scene Graph Generator API

This repository provides an API endpoint for generating scene graphs from images. Upload an image, and the API returns the annotated image, a visual graph representation, and the detected relationships between objects.

API Usage

Endpoint

POST https://dixisouls-scene-graph-generator.hf.space/generate

Parameters

image: The image file to analyze (multipart/form-data)
confidence_threshold: A value between 0 and 1 (default: 0.5)
use_fixed_boxes: Boolean value (default: false)

Response

The API returns a JSON response with:

{
  "objects": [
    {
      "label": "person",
      "label_id": 1,
      "score": 0.91,
      "bbox": [0.3, 0.4, 0.1, 0.3]
    },
    ...
  ],
  "relationships": [
    {
      "subject": "person",
      "predicate": "riding",
      "object": "bicycle",
      "score": 0.82,
      "subject_id": 0,
      "object_id": 1,
      "predicate_id": 5
    },
    ...
  ],
  "annotated_image": "base64_encoded_image_data",
  "graph_image": "base64_encoded_image_data"
}

Example Usage

Python

import requests
import base64
from PIL import Image
import io

# Prepare the image
image_path = "your_image.jpg"
files = {'image': open(image_path, 'rb')}

# Set parameters
data = {
    'confidence_threshold': 0.5,
    'use_fixed_boxes': False
}

# Make the API call
api_url = "https://dixisouls-scene-graph-generator.hf.space/generate"
response = requests.post(api_url, files=files, data=data)

# Process the results
if response.status_code == 200:
    result = response.json()
    
    # Decode and save the images
    annotated_image = Image.open(io.BytesIO(base64.b64decode(result['annotated_image'])))
    annotated_image.save("annotated_image.jpg")
    
    graph_image = Image.open(io.BytesIO(base64.b64decode(result['graph_image'])))
    graph_image.save("graph_image.jpg")
    
    # Print information about objects and relationships
    print(f"Found {len(result['objects'])} objects and {len(result['relationships'])} relationships")
else:
    print(f"Error: {response.text}")

cURL

curl -X POST \
  -F "image=@your_image.jpg" \
  -F "confidence_threshold=0.5" \
  -F "use_fixed_boxes=false" \
  https://dixisouls-scene-graph-generator.hf.space/generate

Model Information

This API uses:

YOLOv8 for object detection
A custom neural network for relationship prediction
PyTorch as the deep learning framework

License

This project is licensed under the MIT License.

Author

Created by dixisouls