Update README.md

README.md

@@ -1,12 +1,200 @@
----
-title: PSYCHOMETER 2.0
-emoji: 💻
-colorFrom: pink
-colorTo: pink
-sdk: gradio
-sdk_version: 5.28.0
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Enhanced Facial Analysis Application
+
+A comprehensive application for real-time analysis of facial expressions and audio in videos and webcam feeds. This application leverages AI models to interpret users' emotional and cognitive states, providing detailed insights for advertising and user experience research.
+
+## Features
+
+### 1. Live Metrics Visualization
+- **Real-time processing**: View facial metrics and analysis as they change in both video and webcam inputs
+- **Dynamic dashboard**: Visualize multiple metrics simultaneously with intuitive gauges
+- **Progress tracking**: Monitor video processing with detailed status updates
+
+### 2. Gemini 2.0 Flash Integration
+- **Advanced contextual analysis**: Uses Google's Gemini 2.0 Flash to analyze facial expressions in context
+- **Detailed user state reports**: Provides both summary labels and in-depth explanations of detected emotional states
+- **Ad context awareness**: Interprets reactions based on the advertisement type and content
+
+### 3. Audio Analysis Integration
+- **Multimodal sensing**: Combines facial expression and voice tone analysis for more accurate emotional detection
+- **Emotion classification**: Identifies emotions like happiness, sadness, anger, and fear from audio
+- **Confidence metrics**: Provides reliability scores for detected audio emotions
+
+### 4. Comprehensive Metrics
+- **Psychological dimensions**: Measures valence, arousal, dominance, cognitive load, and more
+- **Personality indicators**: Estimates traits like openness, agreeableness, neuroticism, etc.
+- **Engagement tracking**: Monitors attention and involvement levels throughout the content
+
+## Installation
+
+### Prerequisites
+- Python 3.8 or higher
+- Pip package manager
+- FFmpeg (for audio processing)
+
+### Step 1: Clone or download the repository
+```bash
+git clone https://github.com/yourusername/enhanced-facial-analysis.git
+cd enhanced-facial-analysis
+```
+
+### Step 2: Set up a virtual environment (recommended)
+```bash
+python -m venv venv
+source venv/bin/activate  # On Windows, use: venv\Scripts\activate
+```
+
+### Step 3: Install dependencies
+```bash
+pip install -r requirements.txt
+```
+
+### Step 4: Set up API keys
+Create a `.env` file in the project root and add your Gemini API key:
+```
+GEMINI_API_KEY=your_api_key_here
+```
+
+To obtain a Gemini API key:
+1. Visit [Google AI Studio](https://ai.google.dev/)
+2. Sign up or log in
+3. Navigate to the API keys section
+4. Create a new API key
+
+## Usage
+
+### Starting the Application
+```bash
+python app.py
+```
+This will launch a Gradio web interface accessible at http://localhost:7860
+
+### Video File Analysis
+1. Navigate to the "Video File API" tab
+2. Upload a video file (MP4, AVI, MOV, etc.)
+3. Enter optional ad context information:
+   - Ad Description: Brief description of the advertisement
+   - Ad Detail Focus: Specific aspect of the ad being analyzed
+   - Ad Type/Genre: Category (Video, Funny, Serious, etc.)
+4. Adjust sampling rate (lower values = more detailed analysis but slower processing)
+5. Click "Process Video"
+6. View results in the dashboard and download the CSV data for further analysis
+
+### Webcam Analysis
+1. Navigate to the "Webcam API" tab
+2. Ensure your webcam is connected and permissions are granted
+3. Enter optional ad context information
+4. Toggle "Record Audio" on if you want to include audio analysis
+5. Click "Start Session"
+6. The application will display:
+   - Processed webcam feed with facial landmarks
+   - Live metrics visualization
+   - Real-time user state analysis
+7. Click "End Session" to stop recording and save the data
+
+## Technical Details
+
+### Architecture
+- **Frontend**: Gradio web interface
+- **Video Processing**: OpenCV and MediaPipe Face Mesh
+- **Audio Processing**: Librosa and Transformers
+- **AI Models**:
+  - MediaPipe for facial landmark detection
+  - Custom metrics calculation algorithms
+  - Hugging Face audio emotion detection model
+  - Google Gemini 2.0 Flash for contextual interpretation
+
+### Data Flow
+1. Input source (video file/webcam) → Frame extraction
+2. Each frame → MediaPipe → Facial landmarks
+3. Landmarks → Custom algorithms → Facial metrics
+4. (Optional) Audio → Hugging Face model → Audio emotion metrics
+5. All metrics + Context → Gemini API → Detailed user state analysis
+6. Results → Visualization and CSV export
+
+### Metrics Calculation
+The application extracts several facial features:
+- Eye Aspect Ratio (EAR): Measures eye openness
+- Mouth Aspect Ratio (MAR): Measures mouth openness
+- Eyebrow Position: Detects raised or lowered eyebrows
+- Head Pose: Estimates vertical and horizontal head tilt
+
+These features are used to calculate higher-level metrics:
+- **Valence**: Emotional positivity/negativity (0-1)
+- **Arousal**: Level of excitement/activation (0-1)
+- **Dominance**: Feeling of control (0-1)
+- **Cognitive Load**: Mental effort (0-1)
+- **Emotional Stability**: Resilience vs. volatility (0-1)
+- **Stress Index**: Level of tension (0-1)
+- **Engagement**: Attention and involvement (0-1)
+
+## Output Files
+
+### CSV Data
+The application generates CSV files with the following columns:
+- `timestamp`: Time in seconds from start
+- `frame_number`: Sequential frame identifier
+- Facial metrics (valence, arousal, etc.)
+- Audio metrics (audio_valence, audio_emotion, etc.)
+- Ad context information
+- `user_state`: Short summary of detected state
+- `detailed_user_analysis`: In-depth interpretation from Gemini
+
+### Processed Video
+If enabled, the application saves an annotated video showing:
+- Facial landmarks
+- Current emotional state
+- Detected audio emotion (if available)
+
+## Customization
+
+### Audio Model Selection
+You can modify the audio classification model by changing this line in the code:
+```python
+model_name = "ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition"
+```
+Replace with any compatible Hugging Face audio classification model.
+
+### Sampling Rate
+Adjust the sampling rate to balance between processing speed and analysis detail:
+- Lower values (1-3): Process more frames, more detailed but slower
+- Higher values (10+): Process fewer frames, faster but less detailed
+
+### Metrics Visualization
+The application uses a gauge-style visualization for metrics. You can customize the appearance by modifying the `update_metrics_visualization` function.
+
+## Troubleshooting
+
+### Common Issues
+
+**No face detected**
+- Ensure adequate lighting
+- Position face within camera frame
+- Check if MediaPipe Face Mesh is properly initialized
+
+**Audio not being analyzed**
+- Verify microphone permissions
+- Check if audio recording is enabled
+- Ensure FFmpeg is installed (required for audio extraction)
+
+**Slow processing**
+- Increase sampling rate
+- Use a lower resolution video
+- Check system resources (CPU/GPU usage)
+
+**Gemini API errors**
+- Verify API key is correct and active
+- Check internet connection
+- Ensure you haven't exceeded API rate limits
+
+## Dependencies
+
+The application relies on several key libraries:
+- Gradio: Web interface
+- OpenCV: Video processing
+- MediaPipe: Facial landmark detection
+- Librosa: Audio processing
+- Transformers: Hugging Face models
+- Google Generative AI: Gemini API access
+- MoviePy: Audio extraction from video
+- Matplotlib: Data visualization
+- Pandas: Data manipu