---
license: apache-2.0
---

### DISTILBERT RUNNING ON [DEEPSPARSE](https://github.com/neuralmagic/deepsparse) GOES BRHMMMMMMMM. 🚀🚀🚀

This model is 👇

        ███████╗ ██████╗   █████╗  ██████╗  ███████╗ ███████╗
        ██╔════╝ ██╔══██╗ ██╔══██╗ ██╔══██╗  ██╔════╝ ██╔════╝
        ███████╗ ██████╔╝ ███████║ ██████╔╝  ███████╗ █████╗  
        ╚════██║ ██╔═══╝  ██╔══██║ ██╔══██╗ ╚════██║█ █╔══╝  
        ███████║ ██║      ██║  ██║ ██║  ██ ║███████║ ███████╗
        ╚══════╝ ╚═╝      ╚═╝  ╚═╝ ╚═╝  ╚═ ╝╚══════╝ ╚══════╝
                                                                                                         
![Alt Text](https://media.giphy.com/media/4Hmjz2sqdtASJ2gFMH/giphy.gif)


### LOOKS LIKE THIS 👇

![Imgur](https://imgur.com/gWfX811.jpg)

### Inference endpoints, outside of outliers (4ms) is avg. latency on 2 vCPUs:

![Imgur](https://i.imgur.com/qceSdjZ.png)


### Handler for access to inference endpoints

```python
class EndpointHandler:

    def __init__(self, path=""):

        self.pipeline = Pipeline.create(task="text-classification", model_path=path)

    def __call__(self, data: Dict[str, Any]) -> Dict[str, str]:
        """
        Args:
            data (:obj:): prediction input text
        """
        inputs = data.pop("inputs", data)

        start = perf_counter()
        prediction = self.pipeline(inputs)
        end = perf_counter()
        latency = end - start

        return {
            "labels": prediction.labels, 
            "scores": prediction.scores,
            "latency (secs.)": latency
        }
```
      
̷͈̍
̵̳͒R̶̙̓i̸̟͘c̴̻̆k̸̑͜ÿ̷̳́
̸̪̚
̷͖̀