Local control
chats, working folders and API keys stay on the PC; prompts go only to the endpoint you pick
Product
small language model benchmarking on your PC
01
Pairwise LM helps you run your own llm benchmarks on Windows. You compare model performance metrics locally with full control. With a local model nothing leaves the PC; with a cloud endpoint your prompt text goes to that provider.
Real results
See actual command output.
Any model
Use local or cloud APIs, or the built-in model.
Features
- Filesread, write, edit, delete, list, grep, copy, move, make dir.
- Shellrun commands, see exit code, build, lint, test locally or via SSH.
- Websearch the web or fetch one page for documentation.
- Screenshots and imagesgenerate images or read text from screenshots with OCR.
- Orchestrationparallel sub-agents, run on bigger models, skills, version control.
- Approval modesAuto-run, Ask first, Plan only for your control.
- Cronschedule prompts to run automatically at set times.
- Modelscloud API, local PC model, or built-in model with test button.
How it helps
You write scripts, the agent runs them, and in Ask first mode you approve each change before it is applied.
Score the CSV file with exact match.
- write_file
score_results.pysaved - edit_file
score_results.py+2 - Shell
python score_results.pyexit 0 - read_file
results.csvread - write_file
report.jsonsaved
waiting for your OK before the next stepRejectAccept
Walkthroughs
Run a small language model test
Write your evaluation script with write_file, and Shell runs it against your CSV data to produce results for comparison and analysis of the output. Check the exit code. Read the output with read_file to verify the numbers carefully. This method suits benchmark for local llms for agentic coding tasks. A small language model is the model you test in this workflow. You can re evaluated models on open llm leaderboard to compare performance. Pairwise LM runs local benchmarks on your PC and keeps all data private. Use the agent to automate the scoring process today for better results.
Compare model performance metrics
Pairwise LM runs tools autonomously on Windows, and every action appears in the chat as a card with its real result for the user. Run tests. Compare two models using performance metrics like latency and throughput. The agent uses Shell to run your monitoring command on your GPU. It captures the tokens per second and peak memory usage. You can also use web_search to find the latest benchmark methodology. Repeat the run to check consistency for a smaller model. The agent writes a new script to automate the model evaluation in machine learning.
What Pairwise LM does not do
It is not a hosted service or plugin; it is meant only for your own files, PC and servers you are authorised to use.
