Edit model card

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Card for Model ID

The gemma-StableDiffusion-prompt-generator-v1 model is designed to enhance the process of generating prompts for Stable Diffusion models. Stable Diffusion models are widely used for creating high-quality images based on textual descriptions. This model specifically focuses on transforming a given word or sentence into a well-crafted and effective prompt, which can then be used to guide Stable Diffusion models in producing visually appealing and contextually accurate images. By utilizing this prompt generator, users can improve the quality of their image generation tasks, making the entire process more efficient and effective.

Details

Model Description

  • Developed by: [Mert Sengil]
  • Model type: [Text-to-Text]
  • Language(s) (NLP): [English]
  • License: [MIT License]
  • Finetuned from model: [google/gemma-2b-it]

Uses

Below we share some code snippets on how to get quickly started with running the model. First make sure to

!pip install -U transformers

then copy the snippet from the section that is relevant for your usecase.

Using steps:

from transformers import AutoTokenizer, AutoModelForCausalLM

# upload finetuned model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("Sengil/gemma-StableDiffusion-prompt-generator-v1")
model = AutoModelForCausalLM.from_pretrained("Sengil/gemma-StableDiffusion-prompt-generator-v1")

import torch
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()  # Ensure model is in evaluation mode

def get_completion(query: str, model, tokenizer) -> str:

    # Updated prompt template to encourage generation
    prompt_template = "Generate a creative and descriptive stable diffusion prompt for the following query: {}"
    prompt = prompt_template.format(query)

    encodeds = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)
    model_inputs = encodeds.to(device)

    # Ensure model is in float32 precision
    model = model.float()

    # Additional generation parameters
    generation_params = {
        "max_length": model_inputs['input_ids'].shape[1] + 250,
        "do_sample": True,
        "temperature": 0.9,
        "top_k": 50,
        "top_p": 0.95,
        "pad_token_id": tokenizer.eos_token_id,
        "num_return_sequences": 1  # Adjust if you want multiple outputs
    }

    with torch.no_grad():  # Disable gradient calculation for inference
        generated_ids = model.generate(**model_inputs, **generation_params)

    decoded = tokenizer.decode(generated_ids[0], skip_special_tokens=True)

    # Extract the generated prompt from the output
    prompt_start = "Generate a creative and descriptive stable diffusion prompt for the following query: "
    if prompt_start in decoded:
        generated_prompt = decoded.split(prompt_start, 1)[1].strip()
    else:
        generated_prompt = decoded.strip()

    return generated_prompt

#enter your query
query = 'Colorful Cat'
result = get_completion(query, model, tokenizer)
print(f"Prompt:\n{result}")

Direct Use

from transformers import AutoTokenizer, AutoModelForCausalLM

# upload finetuned model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("Sengil/gemma-StableDiffusion-prompt-generator-v1")
model = AutoModelForCausalLM.from_pretrained("Sengil/gemma-StableDiffusion-prompt-generator-v1")

Inputs and Outputs

Input:

A single word or a sentence that serves as a keyword or a phrase.

Output:

A generated prompt in English, optimized for use with Stable Diffusion. The prompt will be coherent and contextually relevant to the input provided.

Framework versions

  • PEFT 0.11.2.dev0
Downloads last month
0
Safetensors
Model size
2.51B params
Tensor type
F32
·
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Sengil/gemma-StableDiffusion-prompt-generator-v1

Base model

google/gemma-2b-it
Finetuned
(48)
this model

Dataset used to train Sengil/gemma-StableDiffusion-prompt-generator-v1