SourceJul 12, 20262026-07-124 minVerified
Rethinking Medical LLM Evaluation for Clinical Reasoning
A survey argues medical LLMs should be judged by clinical reasoning capacity, not just benchmark accuracy.
CommunityJul 11, 20262026-07-115 minVerified
Do Higher LLM Scores Really Signal Approaching AGI
Public research suggests rising LLM scores reflect tools, memory, and planning systems, not a simple march toward AGI.
SourceJul 11, 20262026-07-114 minVerified
When LLM Agreement Fails as a Reliability Signal
Why LLM agreement can mislead evaluation, with correlated errors, shared wrong answers, and safer judging protocols.
CommunityJul 10, 20262026-07-104 minVerified
Three Axes for Comparing Korean LLM Performance
Korean LLMs are better judged by naturalness, pragmatic understanding, and instruction following than by one rank.
SourceJul 9, 20262026-07-093 minVerified
PCBWorld Redefines Evaluation for Engine-Grounded PCB Routing AI
An overview of PCBWorld, a KiCad-based environment for evaluating PCB routing AI with native actions and DRC feedback.
CommunityJul 7, 20262026-07-074 minVerified
How Question AIs Shift Search Toward Accuracy
Question-based AI speeds research, but answer accuracy and source verification remain critical for reliable work.
CommunityJul 6, 20262026-07-065 minVerified
Why Coding Leads LLM Positioning And Evaluation Today
Why LLM firms foreground coding as a core benchmark, and how that bias helps developers but raises barriers for nondevelopers.
SourceJul 4, 20262026-07-043 minVerified
Medical AI Beyond Tests to Clinical Reasoning
As multiple-choice medical benchmarks saturate, open-ended clinical reasoning and safety are becoming key measures.
SourceJul 4, 20262026-07-043 minVerified
PACE Tests Cheap Proxies For Agent Benchmark Performance
PACE examines whether low-cost non-agent benchmarks can predict expensive agent benchmark performance.
CommunityJul 3, 20262026-07-035 minVerified
How To Compare Code Models Beyond Benchmark Scores
Code model evaluation should weigh real task success, retries, latency, and token cost, not benchmark scores alone.
CommunityJun 29, 20262026-06-294 minVerified
Do Language Models Really Build Stable World Models
Strong language performance may not imply a stable world model. Reassessing LLMs through failures in time, space, and physics.
SourceJun 29, 20262026-06-293 minVerified
MMG-Pop Rethinks Social Popularity Prediction Across Platforms
MMG-Pop uses multimodal and temporal graph signals from Bluesky and Reddit to reassess social popularity prediction.
SourceJun 28, 20262026-06-284 minVerified
Why Benchmarks Miss Much of LLM Performance
How single-run LLM benchmarks can miss usable performance, and why model choice, retries, and cost matter.
SourceJun 27, 20262026-06-274 minVerified
OpenFinGym Reframes How Financial AI Systems Are Evaluated
OpenFinGym shifts financial AI evaluation from single-task accuracy to workflow-level testing across prediction, trading, and risk.
SourceJun 26, 20262026-06-264 minVerified
Temporal Validity Challenges in RAG and Evolving Knowledge
How RAG mixes past and current facts, causing stale-fact errors, and why temporal validity matters in retrieval.
SourceJun 25, 20262026-06-254 minVerified
Automating Benchmarks for Neural Relational Reasoning Generalization
Why automated LLM-built benchmarks for relational reasoning need difficulty control, reliable answers, and bias checks.
CommunityJun 25, 20262026-06-254 minVerified
Beyond RAG for Domain-Specific LLM Decision Tasks
RAGBench and LegalBench show why enterprise LLM evaluation must separate retrieval quality from domain-specific judgment.
SourceJun 24, 20262026-06-244 minVerified
HOLMES Challenges LLMs With Higher-Order Logic Reasoning
HOLMES probes higher-order logic reasoning beyond final answers, exposing limits in LLM rule, predicate, and constraint handling.
SourceJun 24, 20262026-06-244 minVerified
IV-CoT Separates Structure Planning From Visual Rendering
IV-CoT targets structural prompt fidelity in text-to-image generation by separating layout planning from appearance rendering.
CommunityJun 19, 20262026-06-194 minVerified
How Close Chinese LLMs Are to Frontier Models
Chinese LLM progress is best judged by benchmarks, independent evaluations, and cost efficiency rather than executive claims.
CommunityJun 18, 20262026-06-183 minVerified
Why LLM Reasoning Needs More Than Correct Answers
LLM reasoning should be judged not only by accuracy, but also by consistency, constraint tracking, and self-checking.
SourceJun 3, 20262026-06-034 minVerified
Rethinking Protein AI Evaluation With TadA-Bench Replay
TadA-Bench shifts protein AI evaluation from static prediction scores to experiment selection and chronology-preserving replay.
SourceJun 1, 20262026-06-013 minVerified
CodeGolf Bench Tests Concise Code Beyond Correctness Metrics
CodeGolf Bench measures concise code generation across 60 languages, but its scores should not be read as real-world engineering productivity.
SourceJun 1, 20262026-06-014 minVerified
SCALE and the Shift Toward Self-Exploring Web Agents
SCALE examines whether web agents can reduce reliance on expert demonstrations and learn through self-exploration.