huggingface support

Dockerfile

@@ -16,4 +16,5 @@ RUN mkdir -p /app/credentials
 EXPOSE 8050
 
 # Command to run the application
-CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8050"]
+# Use the default Hugging Face port 7860
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,85 +1,101 @@
+---
+title: OpenAI to Gemini Adapter
+emoji: 🔄☁️
+colorFrom: blue
+colorTo: green
+sdk: docker
+app_port: 7860
+---
+
 # OpenAI to Gemini Adapter
 
-This service provides an OpenAI-compatible API that translates requests to Google's Vertex AI Gemini models.
+This service provides an OpenAI-compatible API that translates requests to Google's Vertex AI Gemini models, allowing you to use Gemini models with tools expecting an OpenAI interface.
 
 ## Features
 
-- OpenAI-compatible API endpoints
-- Support for multiple Google Cloud service account credentials with automatic rotation
-- Streaming and non-streaming responses
-- Docker containerization for easy deployment
+-   OpenAI-compatible API endpoints (`/v1/chat/completions`, `/v1/models`).
+-   Supports Google Cloud credentials via `GOOGLE_CREDENTIALS_JSON` secret (recommended for Spaces) or local file methods.
+-   Supports credential rotation when using local files.
+-   Handles streaming and non-streaming responses.
+-   Configured for easy deployment on Hugging Face Spaces using Docker (port 7860) or locally via Docker Compose (port 8050).
 
-## Setup
+## Hugging Face Spaces Deployment (Recommended)
 
-### Prerequisites
+This application is ready for deployment on Hugging Face Spaces using Docker.
 
-- Docker and Docker Compose
-- Google Cloud service account credentials with Vertex AI access
+1.  **Create a new Space:** Go to Hugging Face Spaces and create a new Space, choosing "Docker" as the Space SDK.
+2.  **Upload Files:** Upload the `app/` directory, `Dockerfile`, and `app/requirements.txt` to your Space repository. You can do this via the web interface or using Git.
+3.  **Configure Secrets:** In your Space settings, navigate to the **Secrets** section and add the following secrets:
+    *   `API_KEY`: Your desired API key for authenticating requests to this adapter service. If not set, it defaults to `123456`.
+    *   `GOOGLE_CREDENTIALS_JSON`: The **entire content** of your Google Cloud service account JSON key file. Copy and paste the JSON content directly into the secret value field. **This is the required method for providing credentials on Hugging Face.**
+4.  **Deployment:** Hugging Face will automatically build and deploy the Docker container. The application will run on port 7860 as defined in the `Dockerfile` and this README's metadata.
 
-### Credential Setup
+Your adapter service will be available at the URL provided by your Hugging Face Space (e.g., `https://your-user-name-your-space-name.hf.space`).
 
-The service supports multiple Google Cloud service account credentials and will rotate between them for API requests. This helps distribute API usage across multiple accounts.
+## Local Docker Setup (for Development/Testing)
 
-1. Create a `credentials` directory in the project root:
-   ```
-   mkdir -p credentials
-   ```
+### Prerequisites
 
-2. Add your service account JSON files to the credentials directory:
-   ```
-   # Example with multiple credential files
-   cp /path/to/your/service-account1.json credentials/service-account1.json
-   cp /path/to/your/service-account2.json credentials/service-account2.json
-   cp /path/to/your/service-account3.json credentials/service-account3.json
-   ```
+-   Docker and Docker Compose
+-   Google Cloud service account credentials with Vertex AI access
 
-   The service will automatically detect all `.json` files in the credentials directory.
+### Credential Setup (Local Docker)
 
-3. For backward compatibility, you can still name one of your files `service-account.json`:
-   ```
-   cp /path/to/your/primary-account.json credentials/service-account.json
-   ```
+1.  Create a `credentials` directory in the project root:
+    ```bash
+    mkdir -p credentials
+    ```
+2.  Add your service account JSON files to the `credentials` directory:
+    ```bash
+    # Example with multiple credential files
+    cp /path/to/your/service-account1.json credentials/service-account1.json
+    cp /path/to/your/service-account2.json credentials/service-account2.json
+    ```
+    The service will automatically detect and rotate through all `.json` files in this directory if the `GOOGLE_CREDENTIALS_JSON` environment variable is *not* set.
+3.  Alternatively, set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable *in your local environment or `docker-compose.yml`* to the *path* of a single credential file (used as a fallback if the other methods fail).
 
-## Running the Service
+### Running Locally
 
 Start the service using Docker Compose:
 
-```
+```bash
 docker-compose up -d
 ```
 
-The service will be available at http://localhost:8050.
+The service will be available at `http://localhost:8050` (as defined in `docker-compose.yml`).
 
 ## API Usage
 
 The service implements OpenAI-compatible endpoints:
 
-- `GET /v1/models` - List available models
-- `POST /v1/chat/completions` - Create a chat completion
-- `GET /health` - Health check endpoint (includes credential status)
+-   `GET /v1/models` - List available models
+-   `POST /v1/chat/completions` - Create a chat completion
+-   `GET /health` - Health check endpoint (includes credential status)
 
 All endpoints require authentication using an API key in the Authorization header.
 
 ### Authentication
 
-The service requires an API key for authentication. The default API key is `123456`.
+The service requires an API key for authentication.
 
-To authenticate, include the API key in the Authorization header using the Bearer token format:
+To authenticate, include the API key in the `Authorization` header using the `Bearer` token format:
 
 ```
-Authorization: Bearer 123456
+Authorization: Bearer YOUR_API_KEY
 ```
 
-You can change the API key by setting the `API_KEY` environment variable in the docker-compose.yml file.
+Replace `YOUR_API_KEY` with the key you configured (either via the `API_KEY` secret/environment variable or the default `123456`).
 
 ### Example Requests
 
+*(Replace `YOUR_ADAPTER_URL` with your Hugging Face Space URL or `http://localhost:8050` if running locally)*
+
 #### Basic Request
 
 ```bash
-curl -X POST http://localhost:8050/v1/chat/completions \
+curl -X POST YOUR_ADAPTER_URL/v1/chat/completions \
   -H "Content-Type: application/json" \
-  -H "Authorization: Bearer 123456" \
+  -H "Authorization: Bearer YOUR_API_KEY" \
   -d '{
     "model": "gemini-1.5-pro",
     "messages": [
@@ -93,9 +109,9 @@ curl -X POST http://localhost:8050/v1/chat/completions \
 #### Grounded Search Request
 
 ```bash
-curl -X POST http://localhost:8050/v1/chat/completions \
+curl -X POST YOUR_ADAPTER_URL/v1/chat/completions \
   -H "Content-Type: application/json" \
-  -H "Authorization: Bearer 123456" \
+  -H "Authorization: Bearer YOUR_API_KEY" \
   -d '{
     "model": "gemini-2.5-pro-exp-03-25-search",
     "messages": [
@@ -110,70 +126,78 @@ curl -X POST http://localhost:8050/v1/chat/completions \
 
 The API supports the following Vertex AI Gemini models:
 
-| Model ID | Description |
-|----------|-------------|
-| `gemini-2.5-pro-exp-03-25` | Gemini 2.5 Pro Experimental (March 25) |
+| Model ID                       | Description                                    |
+| ------------------------------ | ---------------------------------------------- |
+| `gemini-2.5-pro-exp-03-25`     | Gemini 2.5 Pro Experimental (March 25)         |
 | `gemini-2.5-pro-exp-03-25-search` | Gemini 2.5 Pro with Google Search grounding |
-| `gemini-2.0-flash` | Gemini 2.0 Flash |
-| `gemini-2.0-flash-search` | Gemini 2.0 Flash with Google Search grounding |
-| `gemini-2.0-flash-lite` | Gemini 2.0 Flash Lite |
+| `gemini-2.0-flash`             | Gemini 2.0 Flash                             |
+| `gemini-2.0-flash-search`      | Gemini 2.0 Flash with Google Search grounding |
+| `gemini-2.0-flash-lite`        | Gemini 2.0 Flash Lite                          |
 | `gemini-2.0-flash-lite-search` | Gemini 2.0 Flash Lite with Google Search grounding |
-| `gemini-2.0-pro-exp-02-05` | Gemini 2.0 Pro Experimental (February 5) |
-| `gemini-1.5-flash` | Gemini 1.5 Flash |
-| `gemini-1.5-flash-8b` | Gemini 1.5 Flash 8B |
-| `gemini-1.5-pro` | Gemini 1.5 Pro |
-| `gemini-1.0-pro-002` | Gemini 1.0 Pro |
-| `gemini-1.0-pro-vision-001` | Gemini 1.0 Pro Vision |
-| `gemini-embedding-exp` | Gemini Embedding Experimental |
+| `gemini-2.0-pro-exp-02-05`     | Gemini 2.0 Pro Experimental (February 5)      |
+| `gemini-1.5-flash`             | Gemini 1.5 Flash                             |
+| `gemini-1.5-flash-8b`          | Gemini 1.5 Flash 8B                          |
+| `gemini-1.5-pro`             | Gemini 1.5 Pro                               |
+| `gemini-1.0-pro-002`           | Gemini 1.0 Pro                               |
+| `gemini-1.0-pro-vision-001`    | Gemini 1.0 Pro Vision                        |
+| `gemini-embedding-exp`         | Gemini Embedding Experimental                |
 
-Models with the `-search` suffix enable grounding with Google Search using dynamic retrieval, allowing the model to intelligently decide when to retrieve and reference up-to-date information from the web based on the query.
+Models with the `-search` suffix enable grounding with Google Search using dynamic retrieval.
 
 ### Supported Parameters
 
-The API supports the following OpenAI-compatible parameters that map to Vertex AI's GenerativeModel:
+The API supports common OpenAI-compatible parameters, mapping them to Vertex AI where possible:
 
-| OpenAI Parameter | Vertex AI Parameter | Description |
-|------------------|---------------------|-------------|
-| `temperature` | `temperature` | Controls randomness (0.0 to 1.0) |
-| `max_tokens` | `max_output_tokens` | Maximum number of tokens to generate |
-| `top_p` | `top_p` | Nucleus sampling parameter (0.0 to 1.0) |
-| `top_k` | `top_k` | Top-k sampling parameter |
-| `stop` | `stop_sequences` | List of strings that stop generation when encountered |
-| `presence_penalty` | `presence_penalty` | Penalizes repeated tokens |
-| `frequency_penalty` | `frequency_penalty` | Penalizes frequent tokens |
-| `seed` | `seed` | Random seed for deterministic generation |
-| `logprobs` | `logprobs` | Number of log probabilities to return |
-| `n` | `candidate_count` | Number of completions to generate |
+| OpenAI Parameter    | Vertex AI Parameter | Description                                       |
+| ------------------- | --------------------- | ------------------------------------------------- |
+| `temperature`       | `temperature`         | Controls randomness (0.0 to 1.0)                  |
+| `max_tokens`        | `max_output_tokens`   | Maximum number of tokens to generate              |
+| `top_p`             | `top_p`               | Nucleus sampling parameter (0.0 to 1.0)           |
+| `top_k`             | `top_k`               | Top-k sampling parameter                          |
+| `stop`              | `stop_sequences`      | List of strings that stop generation when encountered |
+| `presence_penalty`  | `presence_penalty`    | Penalizes repeated tokens                         |
+| `frequency_penalty` | `frequency_penalty`   | Penalizes frequent tokens                         |
+| `seed`              | `seed`                | Random seed for deterministic generation          |
+| `logprobs`          | `logprobs`            | Number of log probabilities to return           |
+| `n`                 | `candidate_count`     | Number of completions to generate               |
 
-## Credential Rotation
+## Credential Handling Priority
 
-The service automatically rotates through all available credential files for each request. If a credential fails, it will try the next one.
+The application loads Google Cloud credentials in the following order:
 
-You can check the status of credentials using the health endpoint:
+1.  **`GOOGLE_CREDENTIALS_JSON` Environment Variable / Secret:** Checks for the JSON *content* directly in this variable (Required for Hugging Face).
+2.  **`credentials/` Directory (Local Only):** Looks for `.json` files in the directory specified by `CREDENTIALS_DIR` (Default: `/app/credentials` inside the container). Rotates through found files. Used if `GOOGLE_CREDENTIALS_JSON` is not set.
+3.  **`GOOGLE_APPLICATION_CREDENTIALS` Environment Variable (Local Only):** Checks for a *file path* specified by this variable. Used as a fallback if the above methods fail.
+
+## Environment Variables / Secrets
+
+-   `API_KEY`: API key for authentication (Default: `123456`). **Required as Secret on Hugging Face.**
+-   `GOOGLE_CREDENTIALS_JSON`: **(Required Secret on Hugging Face)** The full JSON content of your service account key. Takes priority over other methods.
+-   `CREDENTIALS_DIR` (Local Only): Directory containing credential files (Default: `/app/credentials` in the container). Used if `GOOGLE_CREDENTIALS_JSON` is not set.
+-   `GOOGLE_APPLICATION_CREDENTIALS` (Local Only): Path to a *specific* credential file. Used as a fallback if the above methods fail.
+-   `PORT`: Not needed for `CMD` config (uses 7860). Hugging Face provides this automatically, `docker-compose.yml` maps 8050 locally.
+
+## Health Check
+
+You can check the status of the service using the health endpoint:
 
 ```bash
-curl http://localhost:8050/health
+curl YOUR_ADAPTER_URL/health -H "Authorization: Bearer YOUR_API_KEY"
 ```
 
-This will return information about the available credentials:
+This returns information about the credential status:
 
 ```json
 {
   "status": "ok",
   "credentials": {
-    "available": 3,
-    "files": ["service-account1.json", "service-account2.json", "service-account3.json"],
-    "current_index": 1
+    "available": 1, // Example: 1 if loaded via JSON secret, or count if loaded from files
+    "files": [], // Lists files only if using CREDENTIALS_DIR method
+    "current_index": 0
   }
 }
 ```
 
-## Environment Variables
-
-- `CREDENTIALS_DIR`: Directory containing credential files (default: `/app/credentials`)
-- `GOOGLE_APPLICATION_CREDENTIALS`: Path to a specific credential file (used as fallback)
-- `API_KEY`: API key for authentication (default: `123456`)
-
 ## License
 
-This project is licensed under the MIT License - see the LICENSE file for details.
+This project is licensed under the MIT License.

app/main.py

@@ -173,28 +173,45 @@ class OpenAIRequest(BaseModel):
 
 # Configure authentication
 def init_vertex_ai():
+    global client # Ensure we modify the global client variable
     try:
-        # First try to use the credential manager to get credentials
+        # Priority 1: Check for credentials JSON content in environment variable (Hugging Face)
+        credentials_json_str = os.environ.get("GOOGLE_CREDENTIALS_JSON")
+        if credentials_json_str:
+            try:
+                credentials_info = json.loads(credentials_json_str)
+                credentials = service_account.Credentials.from_service_account_info(credentials_info, scopes=['https://www.googleapis.com/auth/cloud-platform'])
+                project_id = credentials.project_id
+                client = genai.Client(vertexai=True, credentials=credentials, project=project_id, location="us-central1")
+                print(f"Initialized Vertex AI using GOOGLE_CREDENTIALS_JSON env var for project: {project_id}")
+                return True
+            except Exception as e:
+                print(f"Error loading credentials from GOOGLE_CREDENTIALS_JSON: {e}")
+                # Fall through to other methods if this fails
+
+        # Priority 2: Try to use the credential manager to get credentials from files
         credentials, project_id = credential_manager.get_next_credentials()
-  
+
         if credentials and project_id:
-            client = genai.Client(vertexai=True,credentials=credentials, project=project_id, location="us-central1")
-            # vertexai.init(credentials=credentials, project=project_id, location="us-central1")
-            print(f"Initialized Vertex AI with project: {project_id}")
+            client = genai.Client(vertexai=True, credentials=credentials, project=project_id, location="us-central1")
+            print(f"Initialized Vertex AI using Credential Manager for project: {project_id}")
             return True
         
-        # Fall back to environment variable if credential manager fails
+        # Priority 3: Fall back to GOOGLE_APPLICATION_CREDENTIALS environment variable (file path)
         file_path = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")
         if file_path and os.path.exists(file_path):
-            credentials = service_account.Credentials.from_service_account_file(file_path,scopes=['https://www.googleapis.com/auth/cloud-platform'])
-            project_id = credentials.project_id
-            client = genai.Client(vertexai=True,credentials=credentials, project=project_id, location="us-central1")
-            # vertexai.init(credentials=credentials, project=project_id, location="us-central1")
-            print(f"Initialized Vertex AI with project: {project_id} (using GOOGLE_APPLICATION_CREDENTIALS)")
-            return True
-        else:
-            print(f"Error: No valid credentials found. GOOGLE_APPLICATION_CREDENTIALS file not found at {file_path}")
-            return False
+            try:
+                credentials = service_account.Credentials.from_service_account_file(file_path, scopes=['https://www.googleapis.com/auth/cloud-platform'])
+                project_id = credentials.project_id
+                client = genai.Client(vertexai=True, credentials=credentials, project=project_id, location="us-central1")
+                print(f"Initialized Vertex AI using GOOGLE_APPLICATION_CREDENTIALS file path for project: {project_id}")
+                return True
+            except Exception as e:
+                 print(f"Error loading credentials from GOOGLE_APPLICATION_CREDENTIALS path {file_path}: {e}")
+        
+        # If none of the methods worked
+        print(f"Error: No valid credentials found. Tried GOOGLE_CREDENTIALS_JSON, Credential Manager ({credential_manager.credentials_dir}), and GOOGLE_APPLICATION_CREDENTIALS.")
+        return False
     except Exception as e:
         print(f"Error initializing authentication: {e}")
         return False
@@ -645,9 +662,9 @@ async def chat_completions(request: OpenAIRequest, api_key: str = Depends(get_ap
         
         # Initialize Vertex AI with the rotated credentials
         try:
-            client = genai.Client(vertexai=True,credentials=credentials, project=project_id, location="us-central1")
-            # vertexai.init(credentials=credentials, project=project_id, location="us-central1")
-            print(f"Using credentials for project: {project_id}")
+            # Re-initialize client for this request - credentials might have rotated
+            client = genai.Client(vertexai=True, credentials=credentials, project=project_id, location="us-central1")
+            print(f"Using credentials for project: {project_id} for this request")
         except Exception as auth_error:
             error_response = create_openai_error_response(
                 500, f"Failed to initialize authentication: {str(auth_error)}", "server_error"

docker-compose.yml

@@ -6,7 +6,8 @@ services:
       context: .
       dockerfile: Dockerfile
     ports:
-      - "8050:8050"
+      # Map host port 8050 to container port 7860 (for Hugging Face compatibility)
+      - "8050:7860"
     volumes:
       - ./credentials:/app/credentials
     environment: