codeparrot/apps_metric · Fix bug that _temp_run can't be pickled; Pass indices to allow evaluation on a subset of problems

Bug fix: I always got a runtime error when evaluating any solution. The reason seems to be that _temp_run is inside check_correctness in utils.py. Moving it out of check_correctness solves the problem.
Feature: The _compute function in apps_metric.py accepts an indices argument, which is a list of indices of problems to be evaluated. This can be useful if we only want to evaluate solutions to a few problems in APPS, but not all of them.

I have to admit that I haven't created a PR on HF before. I did fork this first (https://huggingface.co/spaces/shunzh/apps_metric), but it seems that PR is not based on a fork, and I can upload files directly here? Also, let me know if there's a template that I should use for PR (I couldn't find one) or if this message is clear. Thanks!