How we test
Our protocol
Every tool gets the same task in four languages.
Scoring profiles
Each test uses a published scoring profile. A metric scores 10 at its best value and 0 at its worst; the language score is the weighted average, and the overall score is the mean of the four languages.
| Metric | Weight | Best | Worst | Measured |
|---|---|---|---|---|
| Setup time | 60% | 5 | 65 | per language |
| Monthly cost at 1,000 runs | 40% | 0 | 100 | once for all languages |