Skip to content
FreeEval Documentation

Installation

Installing FreeEval

FreeEval is designed to provide a reproducible evaluation framework for language models. Follow these steps to set up FreeEval in your environment.

Prerequisites

Before installing FreeEval, ensure you have:

  • Python 3.8 or higher
  • Git
  • (Recommended) A virtual environment tool like conda or venv

Docker Setup for Model Evaluation

If you plan to evaluate your own models (especially large language models), we strongly recommend installing Docker with GPU support:

  1. Install Docker following the official Docker installation guide
  2. For NVIDIA GPU users, install NVIDIA Container Toolkit following the official guide

This setup will allow you to efficiently deploy models using our provided utilities.

Installation Steps

FreeEval is currently available directly from the GitHub repository:

Terminal window
# Clone the repository
git clone https://github.com/WisdomShell/FreeEval.git


# Navigate to the FreeEval directory
cd FreeEval


# Install dependencies
pip install -r requirements.txt

Verifying Your Installation

To verify that FreeEval was installed correctly, try running one of the example evaluations:

Terminal window
python run.py -c ./config/examples/mmlu.json

If the installation was successful, you’ll see FreeEval beginning to process the evaluation configuration.

Next Steps

Now that you have FreeEval installed, move on to the Quick Start guide to understand how to configure and run your own evaluations!