Editorial investigation
The Reasoning Gap: How Claude Opus 5.5 Redefined the Quality Ceiling
Review the current top model, its accepted benchmark evidence, and the score calculation.
Explore full analysisLoading the current published AI model ranking…
Quality 3.0 uses two-thirds no-tools HLE and one-third Terminal-Bench 4.0 on fixed 0–100 scales. Both inputs use the same evaluated configuration. Price and speed do not affect Quality.
Buyer-facing table
Score explanation
Why the quality score changes
Quality 3.0 is two-thirds no-tools HLE plus one-third Terminal-Bench 4.0 on fixed 0–100 scales, using the same configuration for both. Ranking uses the unrounded total.
Editorial investigation
Review the current top model, its accepted benchmark evidence, and the score calculation.
Explore full analysis"Quality is what still makes sense after the excitement fades."
PickAIModel Editorial
Quality note
Supporting benchmarks do not affect Quality or Value. AA-Omniscience uses index points; the other benchmarks use percentages. Missing published evidence is shown as n/a.
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Opus 5.5 is worth shortlisting for demanding coding work, such as tracking down a stubborn bug or reviewing a large change, where a wrong answer costs more than the tokens. Its API pricing sits towards the upper end of the market, so it makes the most sense when you can see the difference in the finished work. The independent results were measured at the model's highest reasoning setting, which may not match how you plan to run it. Try it on a few real tasks before committing a budget.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Quality score
Humanity's Last Exam: thousands of very hard, expert-written questions across maths, science and the humanities. A good test of deep reasoning and expert-level knowledge.
How to read it: Percentage of questions answered correctly. Higher is better; even the best models still get most questions wrong, so modest-looking scores are normal.
Measured with the model thinking step by step and no tools, so it cannot search the web or run code to find answers.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Sonnet 5.5 is a sensible first choice for everyday development work: reviewing code, fixing well-defined bugs and making contained changes. Its API rates sit in the middle of the market, which makes it a natural model to try before paying for a premium one. A lower price per token does not always mean a lower cost per finished job, so keep an eye on how much follow-up each change needs. Its independent results were measured at maximum reasoning effort, so lighter settings may behave differently.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Kimi K3.1 Pro is worth a trial for coding, document analysis and research tasks that involve several steps. You can use the hosted Kimi product or build an API integration. The API supports image input and tool calling, but a deployed agent still needs suitable tools, permissions and checks. Compare its answers and completed work with your own acceptance criteria before using it for important decisions. API charges and hosted membership allowances follow separate billing rules.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Gemini 4 Argon is worth adding to a shortlist for demanding technical work, such as complex coding or multi-step analysis, where a better answer saves real time. Its current API rates are introductory, so check Google's pricing page for when they change and budget for the standard rate rather than the launch price. The independent results were measured at a high reasoning setting, which may differ from your own configuration. As with any new release, test it on a few real tasks before moving important work across.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Fable 5.1 is best considered for the hardest end of the work: complex reasoning and coding problems where a better answer is worth paying for. Its API rates are among the highest of the models we rank, so it is difficult to justify for routine or high-volume tasks. The sensible approach is to test it on a handful of genuinely difficult problems and judge the finished results yourself. Bear in mind that the independent results were measured at the model's top reasoning setting.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
Astra is a premium-priced OpenAI model for complicated reasoning and development work. At its published rates, the main question is whether it earns its cost, so measure that directly: run a small set of real tasks and compare usable results, follow-up work and total spend against a cheaper OpenAI model. The independent results were measured at its highest reasoning setting and say nothing about response speed, writing style or fit for every business task.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
GPT-6.1 Sol is a practical place to start if you want an OpenAI model for coding and reasoning without paying premium rates. If you already use another OpenAI model, compare input and output prices side by side, because small differences in output pricing add up quickly on long responses. Price alone will not tell you which model gives better answers, so test both on the same tasks and settings before switching. Its independent results were measured at maximum reasoning effort.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
GPT-6.1 Terra is worth trialling when work combines reading images, drafting text and acting in software. OpenAI offers it through ChatGPT Work, Codex and the API. For an API integration, choose a reasoning setting that suits the task and measure the result on representative documents or screens. Computer actions need appropriate permissions and review before they affect important records. The published token rates cover API inference, while hosted access depends on your plan and workspace availability.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
GLM-5.3, from Z.ai, is a good candidate for a cost-conscious coding or reasoning trial. Its API rates are well below premium pricing, and Z.ai also offers a cheaper Flash version, so the real question is whether the larger model does enough extra for your work to justify the difference. Run both against the same checks. The independent results give a useful reference point, but app availability and response speed have not been verified here and are worth checking separately.
Long-document reading: the model gets very long documents and must answer questions that need information pulled together from different parts.
How to read it: Percentage of questions answered correctly. Higher is better. Useful if you plan to work with reports, contracts or large files.
Long-document reasoning across 100 questions, with three attempts per question. Average pass@1 percentage; higher is better. Version 1.1 is not directly comparable with older AA-LCR results. Supporting evidence only.
Benchmark methodologyTests general knowledge and honesty: does the model know the answer, and does it admit when it does not instead of making something up?
How to read it: Scored from −100 to +100. Right answers add points, confident wrong answers take points away, and “I don’t know” scores zero. Higher means more reliable facts.
Factual knowledge and hallucination restraint. Index from −100 to +100: correct answers earn credit, incorrect answers lose credit, and abstentions receive zero. Higher is better. Supporting evidence only.
Benchmark methodologyPractical computer tasks done at a command line, such as setting up software, fixing broken code or processing files. Shows how well the model can work as a hands-on coding or IT assistant.
How to read it: Percentage of tasks completed successfully. Higher is better.
Different Terminal-Bench versions use different task sets, so compare scores only within the same version.
Estimated cost of one typical exchange through the developer API: about 3,000 tokens, roughly 2,000 words of prompt and reply combined.
How to read it: Lower is cheaper. This is pay-as-you-go API pricing, separate from monthly app subscriptions such as a Plus or Pro plan.
Standard basket: 1,200 input tokens plus 1,800 output tokens at the published API rates.
3,000 tokens · USD
The context window: how much text the model can take in at once, including your conversation, instructions and any documents you share.
How to read it: Shown in tokens (K = thousand, M = million). A token is about three-quarters of a word. Bigger lets you work with longer documents, but does not by itself mean better answers.
Leaderboard verdict
Full verdict
This verdict covers one specific release, Qwen3.8 Max 0902, not whatever happens to be served under the Qwen3.8 Max name later. Its output pricing is moderate, which makes it worth a look for work that produces long responses. Two cautions apply. The listed prices are for Alibaba Cloud's international (Singapore) service and may differ in other regions, and the independent results only hold if you use the same release and setup. Confirm both before relying on the benchmarks or estimating costs.