SourceMay 28, 20262026-05-283 minVerified
Evaluating AI Agents for E-Commerce Dispute Resolution Tasks
CyberJurors evaluates agent systems on multi-round, multimodal evidence handling and platform rule adaptation in e-commerce disputes.
SourceMay 28, 20262026-05-285 minVerified
MOV-Bench Reveals Gaps in Multi-Hop Video Reasoning
MOV-Bench highlights evaluation gaps in multi-hop audio-visual reasoning and shows consistent gains from agentic search.
SourceMay 28, 20262026-05-284 minVerified
Turning Papers Into Benchmarks With Agentic Reproduction Workflows
How under-specified applied ML papers can become executable benchmarks through agentic workflows and slot-based reporting.
SourceMar 27, 20262026-03-275 minVerified
NeuroVLM-Bench Tests Clinical Reasoning in Neuroimaging VLMs
A neuroimaging benchmark comparing vision-enabled LLMs on MRI and CT, focusing on clinical reasoning, errors, and safety tradeoffs.
SourceMar 26, 20262026-03-263 minVerified
Rethinking LLM Agents as Adaptive Computation Graphs
View LLM agents as runtime-adaptive computation graphs to optimize accuracy, cost, latency, debugging, and control.
SourceMar 12, 20262026-03-124 minVerified
Measuring LLM Gaps With LatAm Context QA Benchmarks
A LatAm-focused QA set (26k+) links Wikidata and Wikipedia to measure LLM gaps by country and cultural context.
CommunityMar 12, 20262026-03-124 minVerified
Rethinking Research Automation Forecasts Beyond Speed And Accuracy Metrics
Don’t equate tokens/sec or speedups with research automation; fix success, time budget, retries, and verification to forecast.
SourceMar 10, 20262026-03-104 minVerified
RM-R1 Reward Models That Reason Before Scoring
RM-R1 proposes reward models that reason before scoring, reporting up to 4.9% gains on public RM benchmarks and highlighting safety evaluation gaps.
CommunityMar 8, 20262026-03-084 minVerified
Beyond Benchmark Scores: Reproducible, Multi-Metric Model Evaluation
Why tiny benchmark gaps mislead: evaluation settings, reproducible logs, and multi-metric, roadmap-driven model selection.
CommunityMar 8, 20262026-03-084 minVerified
When 4-Bit Quantization Beats FP16 Perplexity
Explain why 4-bit quantized models can show lower PPL than FP16, and outline a reproducible evaluation protocol.
SourceMar 7, 20262026-03-075 minVerified
Combustion Knowledgebase And QA Benchmark For LLM Pipelines
A 3.5B-token combustion knowledgebase and CombustionQA benchmark unify knowledge injection and evaluation into one pipeline.
SourceMar 7, 20262026-03-073 minVerified
EVMbench Benchmarks Detect Patch And Exploit Agent Workflows
EVMbench evaluates agent smart-contract security across detection, patching with tests, and exploit attempts in a sandboxed EVM.
SourceMar 5, 20262026-03-053 minVerified
AgentSelect Benchmark For Query-Conditioned Agent Configuration Recommendation
AgentSelect defines narrative-query to end-to-end agent configuration recommendation, proposing a benchmark with queries, agents, and interactions.
CommunityMar 5, 20262026-03-055 minVerified
Difficulty Illusions In LLMs And Evaluation Design
How LLMs create difficulty illusions, and how to design evaluation gates with scenarios, protocols, and multi-metric reporting.
CommunityMar 4, 20262026-03-044 minVerified
Evaluating LLM Self-Consistency Beyond Humanlike Mimicry
Separate humanlike mimicry from self-consistency in LLMs, and evaluate long-term memory and persona drift with benchmarks and protocols.
CommunityMar 4, 20262026-03-044 minVerified
When Image Preprocessing Breaks Multimodal Geolocation Reliability
Resizing, tiling, and tokenization can shift what models see, turning map/geography misreads into repeatable product risk.
CommunityMar 4, 20262026-03-044 minVerified
Make AGI Year Predictions Testable With Clear Scoring
How to turn AGI arrival-year claims into testable forecasts by specifying definitions, metrics, probabilities, and scoring rules.
CommunityMar 3, 20262026-03-034 minVerified
Benchmark MLX 4-Bit Local LLMs on Apple Silicon
Run MLX mxfp4 local LLMs with identical commands and prompts, logging tokens-per-sec and peak memory for reproducible comparisons.
CommunityMar 3, 20262026-03-034 minVerified
Untangling AGI Terms: Reasoning, Memory, Continual Learning Metrics
A decision memo separating reasoning, long-term memory, and continual learning into testable metrics to reduce AGI narrative confusion.
CommunityMar 1, 20262026-03-014 minVerified
Why Tiny Prompt Changes Can Break Robot Safety
How small prompt shifts can amplify into risky robot actions, and why alignment alone can’t guarantee physical safety.
CommunityFeb 17, 20262026-02-174 minVerified
Why Benchmark Gains Don’t Guarantee Real-World Model Quality
Static benchmark gains may not translate to real work quality. Covers contamination risks and a practical evaluation framework.
CommunityFeb 16, 20262026-02-165 minVerified
Is The Frontier LLM Gap Really Shrinking Lately
Tight leaderboard scores can hide uncertainty and evaluation drift. Public data alone rarely confirms 3–6 month trend slowdowns.
NewsFeb 14, 20262026-02-144 minVerified
Measuring Coding Agent Speed Beyond Tokens Per Second
Break coding agent latency into output, prefill, tool time, and network overhead to measure end-to-end duration.
NewsFeb 5, 20262026-02-052 minVerified
Why AI Performance Graphs Need Autonomous METR Capability Metrics
Explore why METR metrics for autonomous capability are more crucial than simple benchmark scores for evaluating AI models.