Spaces:
Running
Running
| title: Text2svg Demo App | |
| emoji: 🚀 | |
| colorFrom: blue | |
| colorTo: yellow | |
| sdk: docker | |
| pinned: false | |
| app_port: 8501 | |
| # Drawing with LLM 🎨 | |
| A Streamlit application that converts text descriptions into SVG graphics using multiple AI models. | |
| Access demo app by this [link](https://huggingface.co/spaces/Timxjl/text2svg-demo-app) | |
| ## Overview | |
| This project allows users to create vector graphics (SVG) from text descriptions using three different approaches: | |
| 1. **ML Model** - Uses Stable Diffusion to generate images and vtracer to convert them to SVG | |
| 2. **DL Model** - Uses Stable Diffusion for initial image creation and StarVector for direct image-to-SVG conversion | |
| 3. **Naive Model** - Uses Phi-4 LLM to directly generate SVG code from text descriptions | |
| ## Features | |
| - Text-to-SVG generation with three different model approaches | |
| - Adjustable parameters for each model type | |
| - Real-time SVG preview and code display | |
| - SVG download functionality | |
| - GPU acceleration for faster generation | |
| ## Requirements | |
| - Python 3.11+ | |
| - CUDA-compatible GPU (recommended) | |
| - Dependencies listed in `requirements.txt` | |
| ## Installation | |
| ### Using Miniconda (Recommended) | |
| ```bash | |
| # Install Miniconda | |
| wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh | |
| bash miniconda.sh -b -p $HOME/miniconda | |
| echo 'export PATH="$HOME/miniconda/bin:$PATH"' >> ~/.bashrc | |
| source ~/.bashrc | |
| # Create and activate environment | |
| conda create -n svg-app python=3.11 -y | |
| conda activate svg-app | |
| # Install star-vector | |
| cd star-vector | |
| pip install -e . | |
| cd .. | |
| # Install other dependencies | |
| pip install -r requirements.txt | |
| ``` | |
| ### Using Docker | |
| ```bash | |
| # Build and run with Docker Compose | |
| docker-compose up -d | |
| ``` | |
| ## Usage | |
| Start the Streamlit application: | |
| ```bash | |
| streamlit run app.py | |
| ``` | |
| Or with the yes flag to automatically accept: | |
| ```bash | |
| yes | streamlit run app.py | |
| ``` | |
| The application will be available at http://localhost:8501 | |
| ## Models | |
| ### ML Model (vtracer) | |
| Uses Stable Diffusion to generate an image from the text prompt, then applies vtracer to convert the raster image to SVG. | |
| Configurable parameters: | |
| - Simplify SVG | |
| - Color Precision | |
| - Filter Speckle | |
| - Path Precision | |
| ### DL Model (starvector) | |
| Uses Stable Diffusion for initial image creation followed by StarVector, a specialized model designed to convert images directly to SVG. | |
| ### Naive Model (phi-4) | |
| Directly generates SVG code using the Phi-4 language model with specialized prompting. | |
| Configurable parameters: | |
| - Max New Tokens | |
| ## Evaluation Data and Results | |
| ### Data | |
| The `data` directory contains synthetic evaluation data created using custom scripts: | |
| - The first 15 examples are from the Kaggle competition "Drawing with LLM" | |
| - `descriptions.csv` - Text descriptions for generating SVGs | |
| - `eval.csv` - Evaluation metrics | |
| - `gen_descriptions.py` - Script for generating synthetic descriptions | |
| - `gen_vqa.py` - Script for generating visual question answering data | |
| - Sample images (`gray_coat.png`, `purple_forest.png`) for reference | |
| ### Results | |
| The `results` directory contains evaluation results comparing different models: | |
| - Evaluation results for both Naive (Phi-4) and ML (vtracer) models | |
| - The DL model (StarVector) was not evaluated as it typically fails on transforming natural images, often returning blank SVGs | |
| - Performance visualizations: | |
| - `category_radar.png` - Performance comparison across categories | |
| - `complexity_performance.png` - Performance relative to prompt complexity | |
| - `quality_vs_time.png` - Quality-time tradeoff analysis | |
| - `generation_time.png` - Comparison of generation times | |
| - `model_comparison.png` - Overall model performance comparison | |
| - Generated SVGs and PNGs in respective subdirectories | |
| - Detailed results in JSON and CSV formats | |
| ## Project Structure | |
| ``` | |
| drawing-with-llm/ # Root directory | |
| │ | |
| ├── app.py # Main Streamlit application | |
| ├── requirements.txt # Python dependencies | |
| ├── Dockerfile # Docker container definition | |
| ├── docker-compose.yml # Docker Compose configuration | |
| │ | |
| ├── ml.py # ML model implementation (vtracer approach) | |
| ├── dl.py # DL model implementation (StarVector approach) | |
| ├── naive.py # Naive model implementation (Phi-4 approach) | |
| ├── gen_image.py # Common image generation using Stable Diffusion | |
| │ | |
| ├── eval.py # Evaluation script for model comparison | |
| ├── eval_analysis.py # Analysis script for evaluation results | |
| ├── metric.py # Metrics implementation for evaluation | |
| │ | |
| ├── data/ # Evaluation data directory | |
| │ ├── descriptions.csv # Text descriptions for evaluation | |
| │ ├── eval.csv # Evaluation metrics | |
| │ ├── gen_descriptions.py # Script for generating synthetic descriptions | |
| │ ├── gen_vqa.py # Script for generating VQA data | |
| │ ├── gray_coat.png # Sample image by GPT-4o | |
| │ └── purple_forest.png # Sample image by GPT-4o | |
| │ | |
| ├── results/ # Evaluation results directory | |
| │ ├── category_radar.png # Performance comparison across categories | |
| │ ├── complexity_performance.png # Performance by prompt complexity | |
| │ ├── quality_vs_time.png # Quality-time tradeoff analysis | |
| │ ├── generation_time.png # Comparison of generation times | |
| │ ├── model_comparison.png # Overall model performance comparison | |
| │ ├── summary_*.csv # Summary metrics in CSV format | |
| │ ├── results_*.json # Detailed results in JSON format | |
| │ ├── svg/ # Generated SVG outputs | |
| │ └── png/ # Generated PNG outputs | |
| │ | |
| ├── star-vector/ # StarVector dependency (installed locally) | |
| └── starvector/ # StarVector Python package | |
| ``` | |
| ## Acknowledgments | |
| This project utilizes several key technologies: | |
| - [Stable Diffusion](https://github.com/CompVis/stable-diffusion) for image generation | |
| - [StarVector](https://github.com/joanrod/star-vector) for image-to-SVG conversion | |
| - [vtracer](https://github.com/visioncortex/vtracer) for raster-to-vector conversion | |
| - [Phi-4](https://huggingface.co/microsoft/phi-4) for text-to-SVG generation | |
| - [Streamlit](https://streamlit.io/) for the web interface | |