add-auto generated mask

mask_sqaure.py

@@ -1,9 +1,9 @@
 from PIL import Image, ImageDraw
 
 def create_square_mask(
-    width: int = 1536, 
-    height: int = 1024, 
-    square_size: int = 512, 
+    width: int = 1200, 
+    height: int = 692, 
+    square_size: int = 300, 
     center_x: int = None, 
     center_y: int = None
 ) -> Image:
@@ -60,5 +60,5 @@ if __name__ == "__main__":
         center_x=768,     # Center of 1536
         center_y=512      # Center of 1024
     )
-    mask.save("square_mask.png", "PNG")
+    mask.save("square_mask_tank.png", "PNG")
     print("Square mask image saved as 'square_mask.png'")

response.md

@@ -0,0 +1,34 @@
+The image features a large tank positioned in the middle of a field, surrounded by trees. The tank appears to be a military vehicle, possibly a tank or a tank destroyer, and it is facing towards the right side of the image. The field is mostly brown, with some patches of green grass, and the sky above is filled with clouds, giving the scene a somewhat gloomy atmosphere.
+
+
+The image depicts a vast, open field with a cloudy sky overhead. The field is covered in tall, dry grass, giving it a somewhat barren appearance. In the distance, a line of trees can be seen, providing a sense of depth to the scene. The overall atmosphere of the image is serene and somewhat melancholic, with the cloudy sky and the dry grass adding a touch of desolation to the landscape.
+
+
+
+--
+
+
+Based on the two descriptions you provided of images from a battlefield, let’s identify the most important military-related items present in each scene and then consider their significance. Since you’ve described two slightly different scenes, I’ll analyze them separately and then tie them together.
+First Description:
+
+    Large Tank: The standout military item here is the tank, described as a "large tank" or possibly a "tank destroyer," positioned in the middle of a field and facing right. Tanks are critical in military contexts, serving as heavily armored, mobile firepower capable of engaging enemy forces, breaking through defenses, or holding ground. The specific type (e.g., a main battle tank like an M1 Abrams or a tank destroyer like an M10) isn’t confirmed, but its presence dominates the scene as the primary military asset.
+    Field Setting: While not a "military item" itself, the field’s condition (brown with patches of green) could imply a battlefield after combat—perhaps churned by vehicle tracks or affected by weather, which is tactically relevant for military operations like maneuverability or concealment.
+    Surrounding Trees: These could serve a military purpose, such as providing cover or camouflage for troops or vehicles, though they’re peripheral compared to the tank.
+
+Key Military Item: The tank is unequivocally the most important military element here. It’s the central focus, a symbol of armored warfare, and likely the linchpin of any military activity in this scene.
+Second Description:
+
+    Vast Open Field: This scene lacks an explicit military vehicle like the tank from the first description. However, the "vast, open field" with tall, dry grass could still be a battlefield or staging area. Open terrain is militarily significant for visibility, artillery range, or tank maneuvers, though no specific equipment is mentioned.
+    Line of Trees in the Distance: Similar to the first description, the trees could offer strategic value—cover for infantry, observation posts, or a natural barrier—but they’re distant and secondary.
+    Cloudy Sky: While atmospheric, this isn’t a military item, though it could affect military operations (e.g., air support visibility or troop morale).
+
+Key Military Item: This description lacks a clear, tangible military object like the tank. The field itself, as a potential battlefield, is the most relevant military-related element, but it’s more abstract without equipment or personnel explicitly present.
+Synthesis and Battlefield Context:
+Since you mentioned these are outputs from a battlefield, I’ll assume they’re two perspectives of the same or related scenes. The first image introduces a concrete military asset (the tank), while the second might depict the broader environment around it, perhaps before or after the tank’s presence. Here’s what stands out as most important militarily across both:
+
+    The Tank: From the first description, this is the most significant military item. Tanks are pivotal in modern warfare, capable of offense, defense, and psychological impact. Its placement in the field suggests active engagement or a strategic hold.
+    The Field: Combining both descriptions, the open field (with dry grass, patches of green, and surrounding trees) is the battlefield itself. It’s the stage for military action, influencing how tanks, infantry, or artillery would operate. The barren, desolate vibe could hint at prior combat or abandonment.
+
+Conclusion:
+The tank is the most important military item when considering both descriptions together, as it’s the only explicit piece of military hardware mentioned. The field, as the battlefield context, is a close second, shaping the tactical environment. If these images are meant to represent a cohesive battlefield narrative, the tank likely defines the military presence, while the field sets the scene’s scope and tone. 
+If you’ve got more details—like whether these are literal images you want analyzed further or just text outputs to interpret—let me know, and I can refine this further! What’s your take on these scenes?

runway.py

@@ -1,6 +1,6 @@
 from fastapi import FastAPI, File, UploadFile, HTTPException
 from fastapi.responses import StreamingResponse
-from PIL import Image
+from PIL import Image, ImageDraw
 import io
 import torch
 import numpy as np
@@ -68,32 +68,53 @@ def soften_mask(mask_image: Image, softness: int = 5) -> Image:
     from PIL import ImageFilter
     return mask_image.filter(ImageFilter.GaussianBlur(radius=softness))
 
+def generate_rectangular_mask(image_size: tuple, x1: int = 100, y1: int = 100, x2: int = 200, y2: int = 200) -> Image:
+    """
+    Generate a rectangular mask matching the image dimensions.
+    - White (255) for unmasked areas, black (0) for masked areas.
+    
+    Args:
+        image_size (tuple): Tuple of (width, height) of the original image.
+        x1, y1 (int): Top-left corner coordinates of the rectangle.
+        x2, y2 (int): Bottom-right corner coordinates of the rectangle.
+    
+    Returns:
+        Image: The generated mask in grayscale (L mode).
+    """
+    # Create a blank white mask (255 = unmasked)
+    mask = Image.new("L", image_size, 255)
+    draw = ImageDraw.Draw(mask)
+    
+    # Draw a black rectangle (0 = masked)
+    draw.rectangle([x1, y1, x2, y2], fill=0)
+    
+    return mask
+
 @app.post("/inpaint/")
 async def inpaint_image(
     image: UploadFile = File(...),
-    mask: UploadFile = File(...),
-    prompt: str = "Fill the masked area with appropriate content."
+    prompt: str = "Fill the masked area with appropriate content.",
+    mask_x1: int = 100,
+    mask_y1: int = 100,
+    mask_x2: int = 200,
+    mask_y2: int = 200
 ):
     """
-    Endpoint for image inpainting using a text prompt.
+    Endpoint for image inpainting using a text prompt and autogenerated mask.
     - `image`: Original image file (PNG/JPG).
-    - `mask`: Mask file indicating areas to inpaint (black for masked areas, white for unmasked).
     - `prompt`: Text prompt describing the desired output.
+    - `mask_x1, mask_y1, mask_x2, mask_y2`: Coordinates for the rectangular mask (default: 100,100 to 200,200).
     
     Returns:
     - The inpainted image as a PNG file.
     """
     try:
-        # Load the uploaded image and mask
+        # Load the uploaded image
         image_bytes = await image.read()
-        mask_bytes = await mask.read()
-
         original_image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
-        mask_image = Image.open(io.BytesIO(mask_bytes)).convert("L")
 
-        # Ensure dimensions match between image and mask
-        if original_image.size != mask_image.size:
-            raise HTTPException(status_code=400, detail="Image and mask dimensions must match.")
+        # Generate the mask based on image dimensions and provided coordinates
+        mask_image = generate_rectangular_mask(original_image.size, mask_x1, mask_y1, mask_x2, mask_y2)
 
         # Perform inpainting using the pipeline
         result = pipe(prompt=prompt, image=original_image, mask_image=mask_image).images[0]
@@ -116,36 +137,38 @@ async def inpaint_image(
 @app.post("/inpaint-with-reference/")
 async def inpaint_with_reference(
     image: UploadFile = File(...),
-    mask: UploadFile = File(...),
     reference_image: UploadFile = File(...),
-    prompt: str = "Integrate the reference content naturally into the masked area, matching style and lighting."
+    prompt: str = "Integrate the reference content naturally into the masked area, matching style and lighting.",
+    mask_x1: int = 100,
+    mask_y1: int = 100,
+    mask_x2: int = 200,
+    mask_y2: int = 200
 ):
     """
-    Endpoint for replacing masked areas with reference image content, refined to look natural.
+    Endpoint for replacing masked areas with reference image content, refined to look natural, using an autogenerated mask.
     - `image`: Original image file (PNG/JPG).
-    - `mask`: Mask file (black for areas to replace, white for areas to keep).
     - `reference_image`: Reference image to guide the replacement (PNG/JPG).
     - `prompt`: Text prompt for inpainting refinement.
+    - `mask_x1, mask_y1, mask_x2, mask_y2`: Coordinates for the rectangular mask (default: 100,100 to 200,200).
     
     Returns:
     - The resulting image as a PNG file.
     """
     try:
-        # Load the uploaded image, mask, and reference image
+        # Load the uploaded image and reference image
         image_bytes = await image.read()
-        mask_bytes = await mask.read()
         reference_bytes = await reference_image.read()
 
         original_image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
-        mask_image = Image.open(io.BytesIO(mask_bytes)).convert("L")
         reference_image = Image.open(io.BytesIO(reference_bytes)).convert("RGB")
 
-        # Ensure dimensions match between image, mask, and reference image
-        if original_image.size != mask_image.size:
-            raise HTTPException(status_code=400, detail="Image and mask dimensions must match.")
+        # Ensure reference image matches original image dimensions
         if original_image.size != reference_image.size:
             reference_image = reference_image.resize(original_image.size, Image.Resampling.LANCZOS)
 
+        # Generate the mask based on image dimensions and provided coordinates
+        mask_image = generate_rectangular_mask(original_image.size, mask_x1, mask_y1, mask_x2, mask_y2)
+
         # Soften the mask for smoother transitions
         softened_mask = soften_mask(mask_image, softness=5)