Journal of Advanced Computing Systems
type: channel
description: The Journal of Advanced Computing Systems (JACS) is a peer-reviewed monthly publication focusing on cutting-edge computer science and technology. It covers a wide range of topics, including quantum computing, cybersecurity, AI, and more. JACS emphasizes rigorous peer review, interdisciplinary approaches, and real-world applications. The journal publishes various types of content, from full research articles to perspective pieces, curated by experts from academia and industry. JACS aims to lead in disseminating knowledge that shapes the future of computing.
publisher: Scientific Publication Center
language: en
publication name: Journal of Advanced Computing Systems
issn: 3066-3962
items: AI-Driven Anomaly Detection in U.S. University Networks: A Qualitative Study of Development, Implementation, and Governance
items: LLM-Explainable Consumer SSD Health Scoring with SMART Attributes and Reliability Statistics
items: GPU Memory Usage Prediction for Generative AI Serving Pipelines with Queue, Latency, and Utilization Signals
items: Topology-Aware SSD Health Risk Prediction with SMART Signals, Location Features, and LLM-Based Failure Explanation
items: Retrieval Evidence Quality Prediction for RAG Hallucination Detection with LLM-Derived Semantic Features
items: LLM-Augmented BiGRU-MHA for SSD Health-State Classification Using SMART Telemetry
items: Imbalance-Aware SSD Failure Prediction with Attention-Gated SMART Modeling and LLM-Guided Feature Semantics
items: Spatial RAG for Urban Crash Hotspot Discovery and Safety Countermeasure Recommendation
items: Nepal-Localized Climate-Smart Crop Advisory from Crop-Recommendation, Cereal-Yield, Climate, Boundary, and Soil-Parent Data: A Small-Data Evaluation
items: Lightweight Multimodal Rice Leaf Disease Screening and Nepal-Localized Climate-Smart Advisory Support: Evaluations on Real Rice Images, Nepal SOTER Soil Profiles, and WorldClim Climate Data
items: Human-Uncertainty Distillation for Calibrated Vision Models on CIFAR-10H
items: Ambiguity-Aware HDFS Log Anomaly Detection with Retrieval-Augmented Failure Narratives and Selective Refusal
items: Small-Sample Tree-Ensemble Models for Climate-Smart Crop Selection in Low-Resource Ethiopian Agriculture
items: LLM-Explained Graph Traffic Forecasting for DOT Corridor Operations: Full Empirical Evaluation on METR-LA and PEMS-BAY with Crash Evidence
items: Noisy-Neighbor-Aware VM Degradation Risk Modeling with Unsupervised Residual Fusion
items: From Earnings Calls to Earnings Events: An LLM-Guided Separation of Discretionary and Systematic Volatility Trading Signals
items: DeFi Lending Stress Meets Duration Risk: Modeling the Link Between Aave Health-Factor Regimes and Treasury Rate Conditions
items: Climate-Resilience and Cereal-Yield Joint Forecasting for Low- and Middle-Income Countries with Public Data and Grounded LLM Policy Explanation
items: From Tell-to-Design to Healthcare Test-Fit Constraint Checking: An LLM-Compatible Requirements-to-Constraints Framework
items: LEED-Aligned Embodied Carbon Decision Support for Hospital Expansion: An LLM-Assisted Comparison of Mass Timber, Structural Steel, and Reinforced Concrete Structural Strategies
items: Self-Correcting Text-to-SQL Agent with Error Feedback: A Reproducible Closed-Loop Evaluation on Compact Executable SQLite Benchmarks
items: Vision-Language Traffic Sign Recognition for Self-Driving: Robust Classification, Uncertainty Calibration, and LLM Safety Explanations on a GTSRB-Compatible Benchmark
items: LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk
items: Long-Document RAG for Contractual and Insurance Clause Analysis in Receivables RWA Structures
items: Helpful or Harmful? Benchmarking Large Language Models as Therapy Tools Across Empathy, Specificity, and Safety
items: Volatility Regime-Adaptive Spread Widening for ETF Option Market Making: Evidence from a 2024 SPY, QQQ, and IWM Panel
items: Risk-Bounded GPU Resource Oversubscription via Conformal Demand Envelopes in Production AI Clusters
items: OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations
items: Evidence-Grounded RAG for Cloud-Native DevOps: Hallucination-Resistant AIOps Question Answering over Private Operations Documents
items: Evidence-Chain Reliable RAG: Word-Level Hallucination Detection, Source Attribution, and Provenance Explanation for LLM Applications

AI-Driven Anomaly Detection in U.S. University Networks: A Qualitative Study of Development, Implementation, and Governance
type: item
description: Despite the growing adoption of artificial intelligence (AI) to strengthen cybersecurity, qualitative research examining the development, implementation, and governance of AI-driven anomaly detection systems within U.S. higher education remains limited. Universities present a distinct cybersecurity environment due to their decentralized IT infrastructures, diverse stakeholder communities, and the need to balance security with academic openness.This study investigates the development and implementation of AI-driven anomaly detection systems for identifying abnormal network behavior in U.S. university environments. A qualitative multiple-case study design was employed, drawing on semi-structured interviews with 11 cybersecurity professionals from six U.S. universities. The interview data were analyzed using thematic analysis to address the study's research questions. The findings reveal that the adoption and operation of AI-driven anomaly detection systems are shaped by several interconnected factors, including vendor influence, governance frameworks, and ethical considerations. Participants also identified significant implementation challenges, such as false-positive alerts, the complexity of university network environments, and institutional cultures that emphasize academic freedom and privacy. Overall, the findings highlight the critical role of effective governance in ensuring the successful deployment and responsible use of AI-driven cybersecurity technologies in higher education.
subject: Articles
  topic: section
subject: Artificial intelligence
  topic: keywords
subject: anomaly detection
  topic: keywords
subject: cybersecurity
  topic: keywords
subject: higher education
  topic: keywords
subject: university networks
  topic: keywords
subject: IT governance
  topic: keywords
subject: ethical AI
  topic: keywords
creator: Okhae Joel Ojugo
creator: Nseobot Afaha
creator: Oghogho Timothy Obasuyi
creator: Segun Onilude
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-08-06
publication date: 2026-08-06
starting page: 1
ending page: 19

LLM-Explainable Consumer SSD Health Scoring with SMART Attributes and Reliability Statistics
type: item
description: Reliable consumer SSD and NVMe assessment requires a model that distinguishes observed SMART evidence from population context and communicates risk without presenting a diagnostic certainty that the data cannot support. This paper presents LESHS, an explainable health-scoring framework built on the LinuxHW SMART Repository. The analysis processed 162,383 rows from the All_SSD.md and All_NVMe.md appendices, removed one exact duplicate, and retained 162,382 records: 81,660 SATA SSD samples and 80,722 NVMe samples. The repository-defined important-error flag was positive for 5.09% of retained records, with rates of 8.64% for SSD and 1.49% for NVMe. A strict non-leakage experiment withheld each record's Err and MTBF fields and predicted the important-error flag from drive type, power-on exposure, capacity, and training-set manufacturer, model, and capacity-bucket priors. On a stratified test set of 24,358 records, the calibrated LESHS model achieved ROC-AUC 0.8873, PR-AUC 0.3892, F1 0.4066, and Brier score 0.0390. A separate consistency analysis reconstructed the LinuxHW MTBF statistic with MAE 0.00255 years and R2 0.99999. Operational scoring then combined the calibrated prior with the observed important-error count and an explicit penalty for short observation windows. A constrained explanation schema translated the resulting evidence into concise statements about error presence, power-on duration, MTBF, and score band. The findings show that calibrated population priors and direct SMART evidence play complementary roles in consumer-drive health assessment.
subject: Articles
  topic: section
subject: SMART attributes
  topic: keywords
subject: SSD health scoring
  topic: keywords
subject: NVMe reliability
  topic: keywords
subject: LinuxHW
  topic: keywords
subject: explainable AI
  topic: keywords
subject: language-model explanations
  topic: keywords
subject: calibrated ensembles
  topic: keywords
subject: MTBF
  topic: keywords
subject: consumer storage
  topic: keywords
creator: David Chao
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-07-07
publication date: 2026-07-07
starting page: 16
ending page: 34

GPU Memory Usage Prediction for Generative AI Serving Pipelines with Queue, Latency, and Utilization Signals
type: item
description: Generative AI serving pipelines combine prompt processing, model and adapter selection, iterative generation, queueing, and GPU-resident state, so their memory footprint depends on more than request volume alone. This study investigates next-window GPU memory prediction with cross-layer signals from Alibaba GenTD26. The public request trace is transformed into a deterministic pod-window benchmark containing 12,253 active 10-minute samples constructed from 26,823 request records and 143 reproducible pseudo-pods. The feature space combines request complexity, queue pressure, execution and pipeline latency, GPU duty-cycle proxy, pod-memory-utilization proxy, and recent resident-memory lags. Six predictors are evaluated with chronological train, validation, and test partitions. Stochastic-gradient regression provides the lowest test RMSE at 2.588 GiB, with MAE 1.903 GiB and R² 0.608; Ridge and ExtraTrees follow closely. A feature ablation reduces DecisionTree RMSE from 3.469 GiB with request-only variables to 2.889 GiB with the full cross-layer signal set, a 16.7% reduction. Error increases from the 10-minute to the 30-minute horizon, emphasizing the value of near-term forecasts for admission control, pod selection, and warm-state management. The results show that queue, latency, utilization, and recent memory state jointly provide a practical basis for lightweight GPU-memory forecasting in generative serving pipelines.
subject: Articles
  topic: section
subject: generative AI serving
  topic: keywords
subject: GPU memory prediction
  topic: keywords
subject: diffusion pipelines
  topic: keywords
subject: queueing signals
  topic: keywords
subject: latency-aware forecasting
  topic: keywords
subject: GPU utilization
  topic: keywords
subject: GenTD26
  topic: keywords
subject: serverless inference
  topic: keywords
creator: Lee Ji-su
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-07-04
publication date: 2026-07-04
starting page: 1
ending page: 15

Topology-Aware SSD Health Risk Prediction with SMART Signals, Location Features, and LLM-Based Failure Explanation
type: item
description: Solid-state drive (SSD) failure prediction is commonly treated as a device-level classification problem based on Self-Monitoring, Analysis and Reporting Technology (SMART) counters. Production fleets, however, also exhibit model, application, and placement heterogeneity, and failures can cluster within nodes and racks. This study presents TopoHealth-ET, a topology-aware risk model that combines SMART indicators, deployment metadata, train-only topology priors, and an evidence-constrained language explanation layer. The empirical evaluation uses a deterministic schema-aligned benchmark that follows the documented Alibaba SSD table structures and contains 30,000 disks, 11 drive models, 200 racks, and a 1.89% failure rate. On the held-out test split, TopoHealth-ET achieved a ROC-AUC of 0.775, PR-AUC of 0.203, precision of 0.345, recall of 0.175, and F1 of 0.233. At a 1% maintenance review budget, the ranked worklist captured 18.42% of failures with 35.00% precision, compared with a fleet failure rate of 1.90%. The results indicate that topology context can materially improve rare-event maintenance triage, while structured evidence objects allow failure explanations to remain concise, auditable, and aligned with the classifier evidence.
subject: Articles
  topic: section
subject: SSD failure prediction
  topic: keywords
subject: SMART attributes
  topic: keywords
subject: topology-aware machine learning
  topic: keywords
subject: rack-level risk
  topic: keywords
subject: imbalanced classification
  topic: keywords
subject: interpretable AI
  topic: keywords
subject: large language model explanations
  topic: keywords
creator: Ethan Ng
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-06-13
publication date: 2026-06-13
starting page: 56
ending page: 73

Retrieval Evidence Quality Prediction for RAG Hallucination Detection with LLM-Derived Semantic Features
type: item
description: Retrieval-augmented generation (RAG) reduces unsupported generation by supplying external evidence, but retrieved passages do not guarantee that the final answer is grounded. This study treats retrieval evidence quality as a direct risk signal for hallucination detection. The evaluation uses a 17,790-row RAGTruth-processed dataset containing queries, retrieved contexts, generated outputs, hallucination annotations, quality indicators, and generator metadata. The binary target identifies whether an output contains hallucinated content, while secondary analysis distinguishes evident conflict from baseless information. A reproducible feature pipeline measures lexical coverage, sentence-level support, semantic support, entity drift, number drift, length relations, and metadata. Six methods are compared: a majority baseline, metadata logistic regression, TF-IDF logistic regression, evidence-quality logistic regression, an evidence-plus-metadata random forest, and a semantic evidence-quality gradient boosting model. On the held-out test split, the proposed model achieves 0.950 accuracy, 0.944 precision, 0.910 recall, 0.927 F1, 0.944 macro-F1, 0.984 AUROC, and 0.979 AUPRC. It identifies 858 of 943 hallucinated outputs while correctly preserving 1,706 of 1,757 grounded outputs. Ablation and subgroup results show that semantic support is most effective when combined with generation metadata and that relation-level conflict remains harder than unsupported entity or numeric insertion. The results establish evidence quality as an auditable intermediate variable for RAG safety and support calibrated decisions to show, warn, regenerate, abstain, or escalate an answer.
subject: Articles
  topic: section
subject: retrieval-augmented generation
  topic: keywords
subject: hallucination detection
  topic: keywords
subject: evidence quality
  topic: keywords
subject: semantic support
  topic: keywords
subject: factual consistency
  topic: keywords
subject: calibration
  topic: keywords
subject: RAGTruth
  topic: keywords
subject: machine learning evaluation
  topic: keywords
creator: Haruki Sato
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-06-10
publication date: 2026-06-10
starting page: 39
ending page: 55

LLM-Augmented BiGRU-MHA for SSD Health-State Classification Using SMART Telemetry
type: item
description: Solid-state drives are routinely monitored through SMART telemetry, yet health-state classification remains difficult because vendor attributes are heterogeneous, failures are rare, and a single operational snapshot mixes normalized values with highly skewed raw counters. This study presents LLM-Augmented BiGRU-MHA, a compact sequence classifier that represents each SMART attribute with two numeric measurements and a fixed semantic description. A bidirectional gated recurrent unit models dependencies across the ordered attribute sequence, while multi-head attention emphasizes the most diagnostic indicators. Experiments were conducted locally on a 3,000-drive corpus organized according to the Alibaba SSD Open Data schemas, using a stratified 64/16/20 train/validation/test split. The proposed model achieved macro-F1 of 0.700, balanced accuracy of 0.726, weighted-F1 of 0.877, and one-vs-rest macro AUC of 0.952. It improved macro-F1 over the non-semantic BiGRU-MHA and detected 9 of 13 failed drives; the remaining four were assigned to the warning state rather than healthy. Ablation results show that semantic descriptions, bidirectional recurrence, and multi-head attention each contribute to minority-state sensitivity. Learned attention concentrates on wear-out degree, program-fail count, percentage used, and erase-fail count, providing an interpretable signal for storage-health triage.
subject: Articles
  topic: section
subject: SSD failure prediction
  topic: keywords
subject: SMART telemetry
  topic: keywords
subject: health-state classification
  topic: keywords
subject: language-model augmentation
  topic: keywords
subject: BiGRU
  topic: keywords
subject: multi-head attention
  topic: keywords
subject: imbalanced learning
  topic: keywords
subject: Alibaba SSD Open Data
  topic: keywords
creator: Aaron Miller
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-06-07
publication date: 2026-06-07
starting page: 21
ending page: 38

Imbalance-Aware SSD Failure Prediction with Attention-Gated SMART Modeling and LLM-Guided Feature Semantics
type: item
description: Solid-state drive (SSD) failure prediction is a rare-event reliability problem in which missed failures can lead to service disruption, whereas excessive alarms consume replacement capacity and engineering time. This study evaluates imbalance-aware SSD failure prediction on a deterministic 30,000-drive benchmark constructed to follow the public Alibaba SSD data schema. The benchmark contains 105 SMART snapshot fields, 39 failure-tag fields, 11 anonymized drive models, and 280 failures, corresponding to a 0.933% positive rate. Balanced logistic regression, random forest, XGBoost with scale-sensitive weighting, LightGBM with class weighting, and a bidirectional gated recurrent unit with multi-head attention (BiGRU-MHA) trained by focal loss are compared under a common stratified protocol. A fixed language-model-guided semantics layer groups SMART counters into wear, program/erase, media-error, interface, thermal, reallocation, and power-cycle concepts. Balanced logistic regression provides the highest PR-AUC (0.603), LightGBM provides the strongest thresholded recall (0.667) and the highest non-linear-model F1 score (0.596), and BiGRU-MHA delivers the best precision at the strictest 0.5% alert budget while remaining less consistent over broader budgets. The results indicate that imbalance treatment and semantically coherent aggregation can be more valuable than architectural complexity when the available telemetry is an endpoint snapshot rather than a complete daily history.
subject: Articles
  topic: section
subject: SSD failure prediction
  topic: keywords
subject: SMART telemetry
  topic: keywords
subject: extreme class imbalance
  topic: keywords
subject: XGBoost
  topic: keywords
subject: LightGBM
  topic: keywords
subject: BiGRU
  topic: keywords
subject: multi-head attention
  topic: keywords
subject: focal loss
  topic: keywords
subject: probability calibration
  topic: keywords
subject: feature semantics
  topic: keywords
creator: Fiona Clarke
rights: Copyright (c) 2026 https://creativecommons.org/licenses/by/4.0
date: 2026-06-04
publication date: 2026-06-04
starting page: 1
ending page: 20

Spatial RAG for Urban Crash Hotspot Discovery and Safety Countermeasure Recommendation
type: item
description: Urban crash screening systems often identify high-burden places but leave safety analysts to translate those places into treatment concepts. This paper develops a Spatial RAG pipeline for urban crash hotspot discovery and countermeasure recommendation using official 2022 Great Britain road safety and traffic data. The study integrates 106,004 STATS19 collision records, 135,480 casualty records, 193,545 vehicle records, 22,240 AADF count-point rows, 206 local-authority traffic records, and 17,840 MRDB major-road links. The primary spatial benchmark uses the 71,763 geocoded urban injury collisions. National clustering compares BIRCH, MiniBatchKMeans, and DBSCAN on British National Grid coordinates. BIRCH and MiniBatchKMeans both produce 204-center partitions aligned with the number of active highway authorities in the urban sample; MiniBatchKMeans reaches the highest sampled silhouette of 0.460 in 2.94 s, while BIRCH reaches 0.447 in 3.07 s and supplies the hierarchical center structure used for downstream hotspot screening. DBSCAN identifies 679 dense components but leaves 29.9% of crashes as noise and forms a 20,192-crash largest component, making it less suitable as the national partition. Within the ten highest-burden BIRCH centers, Spatial RAG captures 14.87% of held-out severity burden at a 5% cell budget and reaches a mean severity-AUC10 of 0.1405, compared with 11.13% and 0.1093 for KDEGrid. The paired center-wise advantage over KDEGrid is significant with a one-sided Wilcoxon p-value of 0.001953. KDEGrid remains the most stable method at the top-5% budget, with a Jaccard overlap of 0.724. The retrieval layer assigns eight of the top ten hotspots to an urban vulnerable-road-user area-wide package and two to a major-road corridor speed-management package. The results show that severity-aware spatial screening and deterministic retrieval can convert an official crash archive into a transparent first-pass safety planning tool.
subject: Articles
  topic: section
subject: urban crash analysis
  topic: keywords
subject: hotspot discovery
  topic: keywords
subject: STATS19
  topic: keywords
subject: spatial clustering
  topic: keywords
subject: BIRCH
  topic: keywords
subject: DBSCAN
  topic: keywords
subject: kernel density estimation
  topic: keywords
subject: retrieval-augmented generation
  topic: keywords
subject: safety countermeasures
  topic: keywords
subject: traffic exposure
  topic: keywords
creator: Ziliang Samuel Zhong
creator: Long Zhang
creator: David Ma
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-12-19
publication date: 2023-12-19
starting page: 45
ending page: 61

Nepal-Localized Climate-Smart Crop Advisory from Crop-Recommendation, Cereal-Yield, Climate, Boundary, and Soil-Parent Data: A Small-Data Evaluation
type: item
description: This paper evaluates a Nepal-localized climate-smart crop advisory workflow using small public datasets that can be inspected and rerun locally. The workflow separates three tasks: crop-suitability classification from a compact soil-weather crop-recommendation table; district-crop cereal yield regression from Nepal cereal statistics, a fiscal-year NASA POWER climate index, district boundaries, and soil-parent features; and low-yield risk screening. The crop-recommendation dataset contained 2,200 records, seven numeric soil-weather attributes, and 22 crop labels. The Nepal panel contained 11,760 clean district-crop-year yield records for 75 districts and five cereals. A 12,053-row daily POWER point series was converted into 33 fiscal-year climate-index records, and a 75-district boundary layer was overlaid with 942 soil-parent polygons to add static terrain and parent-material descriptors. Using seed 42, Gaussian naive Bayes achieved 0.993 test accuracy and 0.993 macro-F1 on the crop-suitability task. The best temporal yield model, a ridge model with climate, district, crop, geo-soil, and crop-climate interaction features, achieved RMSE 727.90 kg ha-1, MAE 506.70 kg ha-1, and R2 0.486 on fiscal years 2010/11 to 2013/14. The low-yield risk task remained difficult, but the ridge risk classifier improved the screening result to F1 0.330, recall 0.592, precision 0.229, and AUC 0.664. The results support a conservative advisory design: strong crop-suitability screening from the clean benchmark dataset, Nepal-wide district-crop yield screening for cereals, and low-yield risk flags that require field review before intervention.
subject: Articles
  topic: section
subject: Climate-smart agriculture
  topic: keywords
subject: Nepal
  topic: keywords
subject: crop recommendation
  topic: keywords
subject: cereal yield prediction
  topic: keywords
subject: low-yield risk
  topic: keywords
subject: soil parent material
  topic: keywords
subject: NASA POWER
  topic: keywords
subject: ridge regression
  topic: keywords
subject: naive Bayes
  topic: keywords
subject: small-data evaluation
  topic: keywords
creator: Lumeng Han
creator: Derek Zhou
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-07-20
publication date: 2023-07-20
starting page: 58
ending page: 73

Lightweight Multimodal Rice Leaf Disease Screening and Nepal-Localized Climate-Smart Advisory Support: Evaluations on Real Rice Images, Nepal SOTER Soil Profiles, and WorldClim Climate Data
type: item
description: This study evaluates a lightweight multimodal pipeline for rice leaf disease screening and Nepal-localized climate-smart advisory support. The revised evaluation uses directly downloadable public datasets rather than procedurally generated image or soil-weather records. The vision branch uses the UCI Rice Leaf Diseases dataset, which contains 120 real JPG images in three balanced classes: bacterial leaf blight, brown spot and leaf smut. The environmental branch combines Nepal SOTER soil profile data with WorldClim 2.1 monthly climate rasters for precipitation, minimum temperature, maximum temperature, water vapor pressure and elevation, producing 645 profile-season records from 129 georeferenced Nepal soil profiles. The advisory branch uses 24 curated English-Nepali rice management cards and 96 farmer-style retrieval queries grounded in Nepal rice disease, soil fertility and crop-management references. Five image classifiers were compared using 136 engineered color, lesion and edge features. Logistic regression achieved the strongest clean hold-out performance, with 0.917 accuracy and 0.918 macro-F1. Stress tests showed that the model is reasonably stable to small rotations and moderate blur, but accuracy declines under strong sensor noise and large brightness shifts. In the advisory retrieval experiment, character TF-IDF performed best with 0.875 top-1 accuracy, 0.969 top-3 accuracy and 0.922 mean reciprocal rank. For the soil-climate risk task, random forest achieved the best hold-out macro-F1 of 0.869 and cross-validated macro-F1 of 0.851. The results support the paper's central claim: a small, inspectable system can connect leaf-image evidence, Nepal soil-climate context and grounded farmer advice, while remaining explicit about the limits of non-field diagnostic evaluation.
subject: Articles
  topic: section
subject: rice leaf disease
  topic: keywords
subject: Nepal agriculture
  topic: keywords
subject: climate-smart advisory
  topic: keywords
subject: image classification
  topic: keywords
subject: retrieval-augmented generation
  topic: keywords
subject: soil-climate risk
  topic: keywords
subject: SOTER
  topic: keywords
subject: WorldClim
  topic: keywords
subject: TF-IDF
  topic: keywords
creator: Lumeng Han
creator: Derek Zhou
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-04-15
publication date: 2023-04-15
starting page: 67
ending page: 82

Human-Uncertainty Distillation for Calibrated Vision Models on CIFAR-10H
type: item
description: Human uncertainty is informative when a visual example is genuinely ambiguous, because a full label distribution captures plausible class confusions that a hard one-hot label suppresses. This paper evaluates human-uncertainty distillation (HUD) directly on CIFAR-10H, which provides human label distributions for the 10,000-image CIFAR-10 test set. The study uses a stratified 60/20/20 split of CIFAR-10H, yielding 6000 training images, 2000 validation images, and 2000 test images. A compact vision classifier is trained from standardized HOG and color descriptors so that the effect of the supervision signal can be isolated from a larger backbone. HUD combines label-smoothed hard-label supervision with human soft-label distillation whose weight increases on high-entropy human targets, together with a small entropy-alignment penalty. On the held-out test split, HUD reached 58.43% top-1 accuracy, 1.1872 negative log-likelihood, 0.0284 expected calibration error, 0.5449 Brier score, and 0.2188 area under the risk-coverage curve. Relative to standard cross-entropy training, HUD improved accuracy by 1.50 percentage points, reduced negative log-likelihood by 3.6%, reduced expected calibration error by 50.9%, and reduced Brier score by 4.0%. Label smoothing remained a strong baseline, but HUD produced the best student negative log-likelihood, Brier score, human-label cross-entropy, and selective-prediction AURC. Under five corruption families at three severities, HUD improved mean corrupted accuracy from 0.4145 to 0.4201 and reduced mean corrupted ECE from 0.1574 to 0.1233. The results show that real human soft labels can improve likelihood, calibration, selective prediction, and robustness even when top-1 gains are modest.
subject: Articles
  topic: section
subject: uncertainty calibration
  topic: keywords
subject: CIFAR-10H
  topic: keywords
subject: human soft labels
  topic: keywords
subject: knowledge distillation
  topic: keywords
subject: label distributions
  topic: keywords
subject: selective prediction
  topic: keywords
subject: robustness
  topic: keywords
creator: Ziliang Samuel Zhong
creator: Ruiyan Ma
creator: Hailey Zhao
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-02-17
publication date: 2023-02-17
starting page: 77
ending page: 89

Ambiguity-Aware HDFS Log Anomaly Detection with Retrieval-Augmented Failure Narratives and Selective Refusal
type: item
description: This paper studies block-level anomaly detection for Hadoop Distributed File System (HDFS) system logs. The objective is to determine whether anomaly detection, operator-facing explanation, and selective abstention can be integrated into a single reproducible pipeline for highly imbalanced system traces. A reproducible hybrid detector that combines linear discriminative scoring, pattern-memory posteriors, trace statistics, and a calibrated stacking stage was implemented. The evaluation uses the full HDFS_v1 benchmark, which contains 11,175,629 log lines grouped into 575,061 labeled block traces. Retrieval-augmented generation (RAG) is used in a deterministic sense: the system retrieves matched training patterns and renders fixed failure narratives from event templates rather than invoking a free-form large language model. On the chronological test split, the proposed ambiguity-aware stacked detector achieved 0.9881 precision, 0.9869 recall, 0.9875 F1, 0.9975 area under the precision-recall curve (PR-AUC), and 0.99995 area under the receiver operating characteristic curve (ROC-AUC). Simple logistic regression and linear support vector machine (SVM) baselines reached a higher F1 of 0.9952, which indicates that HDFS event counts remain a very strong supervised signal. However, the proposed pipeline produced stronger ranking metrics than those two linear baselines and, at 99.5% automatic coverage, selective refusal increased accepted-case F1 to 0.9980 while removing all accepted false positives. The additive contribution is therefore not a new classifier alone, but an ambiguity-aware framework that combines detection, deterministic RAG-style failure narratives, and refusal decisions. This capability is relevant to petroleum digital operations because distributed storage and data-platform logs increasingly support drilling analytics, seismic processing, production monitoring, and refinery data systems where reliable triage and reviewable incident narratives are needed.
subject: Articles
  topic: section
subject: HDFS and system logs
  topic: keywords
subject: anomaly detection
  topic: keywords
subject: retrieval-augmented generation
  topic: keywords
subject: selective refusal
  topic: keywords
subject: failure narratives
  topic: keywords
subject: reproducible evaluation
  topic: keywords
creator: Jiayi Nie
creator: David Zheng
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-01-14
publication date: 2023-01-14
starting page: 66
ending page: 80

Small-Sample Tree-Ensemble Models for Climate-Smart Crop Selection in Low-Resource Ethiopian Agriculture
type: item
description: Low-resource agricultural decision support requires models that can learn from small, heterogeneous, and locally collected tabular data. This study evaluates a crop-selection workflow for an Ethiopian crop recommendation table containing soil color, soil pH, macro- and micronutrients, seasonal humidity, maximum and minimum temperature, precipitation, wind, cloud amount, surface pressure, and crop labels. Because the main supervised table contains crop labels but no row-level yield target, the experiment is framed as multiclass crop-label recommendation rather than yield regression. To clarify the data scope, a regional cereal yield table and the EthCT2020 georeferenced crop-type reference are used as contextual datasets, not as merged row-level training targets. CatBoost, RandomForest, and XGBoost were fitted on a fixed stratified 75/25 split with low-resource training sizes of 120, 240, 480, 960, and 2,900 records. On the full training pool, RandomForest achieved the highest top-1 accuracy at 0.501, followed by XGBoost at 0.492 and CatBoost at 0.481. XGBoost produced the strongest top-3 shortlist accuracy at 0.838, while CatBoost reached 0.825 and RandomForest reached 0.810. Macro-F1 remained low for all models, showing that rare crops were not reliably recovered under the observed class imbalance. The feature ablation and RandomForest importance analysis show that soil color, pH, and nutrient fields carry most of the separative signal, while climate variables add context but are weaker when used alone. The results support fast tabular crop recommendation for data-scarce settings, while also showing that rare-crop coverage and row-level yield prediction require additional field-level data.
subject: Articles
  topic: section
subject: climate-smart agriculture
  topic: keywords
subject: crop recommendation
  topic: keywords
subject: small-sample learning
  topic: keywords
subject: tree ensembles
  topic: keywords
subject: XGBoost
  topic: keywords
subject: CatBoost
  topic: keywords
subject: RandomForest
  topic: keywords
subject: Ethiopia
  topic: keywords
creator: Anna Chen
creator: Lumeng Han
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-06-24
publication date: 2024-06-24
starting page: 104
ending page: 117

LLM-Explained Graph Traffic Forecasting for DOT Corridor Operations: Full Empirical Evaluation on METR-LA and PEMS-BAY with Crash Evidence
type: item
description: Short-term corridor forecasting becomes useful to transportation agencies only when the prediction layer and the explanation layer are evaluated together. This paper studies an LLM-explained graph traffic forecasting pipeline on METR-LA and PEMS-BAY and augments the high-error analysis with public crash records. We used 12 historical 5-minute observations to predict the next 12 steps, corresponding to 5-60 minute traffic states, and report the 15-, 30-, and 60-minute horizons with valid-speed masked MAE, RMSE, and MAPE. The comparison includes Persistence, HistoricalAverage, SharedGRU, GraphTemporalCNN, GraphTemporalTransformer, and an ensemble. On METR-LA, the ensemble achieved the best average 12-step MAE at 3.961 mph. On PEMS-BAY, Persistence achieved the best average 12-step MAE at 2.175 mph, while GraphTemporalCNN reduced the 60-minute error relative to Persistence and produced the lowest average RMSE. The explanation layer then localized high-error windows to key sensors and matched the most relevant windows against independent collision records. A METR-LA episode on 2012-06-16 14:10-15:10 matched a nearby Los Angeles collision report at 14:10, and a PEMS-BAY episode on 2017-06-13 18:30-19:55 matched a CCRS rear-end crash on I-280/SR-85 at 18:35. The resulting briefs preserve the forecast residuals, sensor IDs, and external evidence needed for corridor review.
subject: Articles
  topic: section
subject: traffic forecasting
  topic: keywords
subject: corridor operations
  topic: keywords
subject: graph neural networks
  topic: keywords
subject: explainable AI
  topic: keywords
subject: crash evidence
  topic: keywords
subject: operations brief
  topic: keywords
subject: METR-LA
  topic: keywords
subject: PEMS-BAY
  topic: keywords
creator: Long Zhang
creator: Eric Zhang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-05-26
publication date: 2024-05-26
starting page: 135
ending page: 145

Noisy-Neighbor-Aware VM Degradation Risk Modeling with Unsupervised Residual Fusion
type: item
description: Cloud virtual machines can experience abrupt benchmark degradation when shared infrastructure is exposed to contention, maintenance activity, VM drift, storage-path effects, or unfavorable placement. This paper presents a noisy-neighbor-aware degradation-risk pipeline for benchmark traces without claiming host-level causal attribution. The study uses three public data sources: the official Azure VM Noise Dataset 2024, the Server Machine Dataset (SMD) from OmniAnomaly, and the Azure Trace for Packing 2020. The Azure experiment covers 7,037,220 raw measurements across 776 CSV partitions and evaluates a deterministic, per-partition balanced modeling table of 1,048,635 rows. Because Azure does not publish co-tenant identities or root-cause labels, Azure F1 scores are reported against weak observed-degradation labels derived from severe raw performance and runtime tails rather than injected synthetic labels. SMD provides an external label-based sanity check with 708,420 test points and official anomaly labels. The proposed score is renamed NN-Aware Residual Fusion (NN-RF): it combines signed performance residuals, runtime residuals, temporal deltas, and previous-day cell pressure through a non-negative validation-calibrated fusion model; PCA reconstruction is treated as a candidate feature rather than as a temporal autoencoder. On the held-out Azure period, NN-RF achieves F1 0.938, AUROC 0.998, and recall 0.932 under the weak degradation task, while Robust MAD remains a strong residual baseline with F1 0.932. On SMD, a windowed neural autoencoder gives the best held-out F1, 0.299, among the evaluated external baselines. A capacity-constrained replay of 50,000 Azure Packing Trace admissions shows that heat-aware first-fit keeps the same acceptance rate as trace-order first-fit while reducing mean placement risk by 7.0%. The results support a narrower conclusion: benchmark residuals can provide useful risk signals for placement, but they should not be interpreted as defini…
subject: Articles
  topic: section
subject: Cloud performance variability
  topic: keywords
subject: noisy-neighbor risk
  topic: keywords
subject: Azure VM Noise Dataset
  topic: keywords
subject: anomaly detection
  topic: keywords
subject: residual fusion
  topic: keywords
subject: weak labels
  topic: keywords
subject: placement replay
  topic: keywords
subject: Server Machine Dataset
  topic: keywords
creator: Jiayi Nie
creator: David Zheng
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-04-24
publication date: 2024-04-24
starting page: 112
ending page: 123

From Earnings Calls to Earnings Events: An LLM-Guided Separation of Discretionary and Systematic Volatility Trading Signals
type: item
description: This paper studies whether earnings-call trading signals can be separated into discretionary components that are firm specific and systematic components that proxy market-wide state. The empirical design is rebuilt on public data releases dated 2022 or earlier in the underlying academic record and 2023 or earlier in the accompanying market-data releases. The call-level text comes from ECTSum, an earnings-call transcript benchmark released in 2022, and the market variables come from S&P 500 news/sentiment and options-implied-volatility panels. The matched sample contains 563 earnings-call events for 254 firms from 2020-10-22 to 2023-01-03. A deterministic ontology assigns every engineered variable to either a systematic or discretionary family before estimation. The outcome is the direction of abnormal at-the-money implied-volatility adjustment around the call, measured relative to the cross-sectional market option-implied-volatility benchmark. Rolling chronological evaluation compares logistic regression, random forests, and gradient boosting. For event-day abnormal IV direction, the best model is a combined pre-event logistic specification with an out-of-sample AUC of 0.579, compared with 0.565 for the discretionary-pre block and 0.499 for the systematic block. For next-day continuation, the best specification is a discretionary-plus-reaction logistic model with an AUC of 0.556. Sorting out-of-sample next-day probabilities into quintiles produces a top-minus-bottom abnormal-IV spread of 0.552 points with a Welch t-statistic of 2.47. The evidence is statistically modest but economically ordered. It supports a layered interpretation: earnings-call language and issuer-specific option state carry most of the signal, while systematic market conditions mainly condition the transmission of that signal.
subject: Articles
  topic: section
subject: earnings calls
  topic: keywords
subject: implied volatility
  topic: keywords
subject: discretionary signals
  topic: keywords
subject: systematic signals
  topic: keywords
subject: machine learning
  topic: keywords
subject: event studies
  topic: keywords
subject: textual analysis
  topic: keywords
subject: option markets
  topic: keywords
creator: Guanzheng Zhao
creator: Kenny Zhang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-03-24
publication date: 2024-03-24
starting page: 126
ending page: 143

DeFi Lending Stress Meets Duration Risk: Modeling the Link Between Aave Health-Factor Regimes and Treasury Rate Conditions
type: item
description: This paper tests whether measured DeFi lending stress co-moves with U.S. Treasury duration conditions. The empirical design uses fixed-release datasets: the Aave V2 Health Factor Dataset released in 2022, the Uniswap Daily Transaction Indices by Network released in 2023, and the FRED-QD 2022m12 quarterly macroeconomic vintage. The final aligned panel contains 54 weekly Aave observations from 4 June 2021 through 25 June 2022. Aave account-level data are aggregated into weekly collateral, debt, health-factor, risky-borrower, and liquidation-zone measures. Uniswap Ethereum daily volume and concentration measures are converted into rolling seven-day controls, and FRED-QD Treasury variables are matched by calendar quarter. A standardized Aave stress score is built from the risky borrower share, liquidation-zone share, debt-collateral ratio, and inverse borrower median health factor, then split into low-, medium-, and high-stress terciles. In the aligned panel, the Aave stress score correlates 0.677 with the 10-year Treasury rate and 0.616 with the quarter-over-quarter 10-year rate change. The risky borrower share correlates 0.736 with the 10-year rate. Regime contrasts show that high-stress Aave weeks have an average risky-borrower share of 20.95%, compared with 7.07% in low-stress weeks. The same high-stress weeks have an average 10-year Treasury rate of 2.42%, compared with 1.57% in low-stress weeks. HAC-adjusted regressions estimate that a one-percentage-point higher 10-year Treasury rate is associated with 12.95 percentage points more risky borrowers and 10.37 percentage points more liquidation-zone borrowers. In expanding-window forecasts, a duration-market ridge model lowers one-week-ahead Aave stress-score RMSE from 3.45 for the historical mean and 1.34 for AR(1) to 1.06. The results identify an empirical association, not a causal proof, between DeFi lending stress and Treasury duration conditions.
subject: Articles
  topic: section
subject: Decentralized finance
  topic: keywords
subject: DeFi lending
  topic: keywords
subject: Aave
  topic: keywords
subject: health factor
  topic: keywords
subject: Uniswap
  topic: keywords
subject: FRED-QD
  topic: keywords
subject: Treasury rates
  topic: keywords
subject: duration risk
  topic: keywords
subject: regime modeling
  topic: keywords
subject: forecasting
  topic: keywords
creator: Guanzheng Zhao
creator: Ziwei Wen
creator: Jeffrey Chen
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-02-22
publication date: 2024-02-22
starting page: 93
ending page: 105

Climate-Resilience and Cereal-Yield Joint Forecasting for Low- and Middle-Income Countries with Public Data and Grounded LLM Policy Explanation
type: item
description: This paper reports a country-year forecasting experiment that links climate-resilience indicators, observed climate exposure, and cereal yields for low- and middle-income countries. The empirical panel combines the December 16, 2024 World Development Indicators archive for cereal yield, cereal production, cereal land and GDP, an ND-GAIN country-year file for vulnerability, readiness, ND-GAIN index and HDI, and QoG Environmental Indicators variables for annual temperature and rainfall. Country-specific temperature and rainfall anomalies are calculated against each country's 1981-2000 climate baseline. After complete-case filtering, the panel contains 39 countries, 702 complete rows, feature years 2002-2019, and target years 2003-2020. All features at year t are used to predict national cereal yield at year t+1. The evaluation uses a temporal split with feature years 2002-2016 for training and 2017-2019 for held-out testing, plus rolling-origin validation. Nine forecasting specifications are compared: persistence, country mean, panel fixed-effects ordinary least squares, ridge, Huber-ridge, regression tree, random forest, gradient boosting, and residual gradient boosting. The ridge panel model is the best held-out specification with RMSE 318.23 kg/ha, MAE 192.93 kg/ha, MAPE 9.08%, and R2 0.945. Recent cereal-yield history remains the strongest accuracy driver, while the climate and ND-GAIN variables add auditable exposure and adaptation context. A constrained policy-explanation layer then converts measured vulnerability, readiness, HDI, climate anomalies and yield volatility into country-specific adaptation explanations.
subject: Articles
  topic: section
subject: climate resilience
  topic: keywords
subject: cereal yield
  topic: keywords
subject: World Development Indicators
  topic: keywords
subject: ND-GAIN
  topic: keywords
subject: QoG Environmental Indicators
  topic: keywords
subject: panel forecasting
  topic: keywords
subject: low- and middle-income countries
  topic: keywords
subject: explainable AI
  topic: keywords
subject: large language models
  topic: keywords
creator: Lumeng Han
creator: Anna Chen
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-01-24
publication date: 2024-01-24
starting page: 121
ending page: 139

From Tell-to-Design to Healthcare Test-Fit Constraint Checking: An LLM-Compatible Requirements-to-Constraints Framework
type: item
description: Language-guided floor-plan generation is attractive for early design, but hospital and behavioral health planning should not be treated as direct automatic plan generation. Early healthcare test-fits depend on jurisdictional review, owner standards, infection prevention, behavioral health safety, staff workflow, and clinical risk judgments that exceed a simple room-list prediction task. This paper reframes the problem as requirements-to-constraints translation followed by explicit constraint checking for early-stage test-fit design. We compiled HPEV-401, a source-grounded corpus of 401 emergency and behavioral health planning statements from four public healthcare planning sources: the International Health Facility Guidelines Emergency Unit functional planning unit, a room-data-sheet compendium, the U.S. Department of Veterans Affairs Emergency Department space planning criteria, and the Facility Guidelines Institute behavioral health crisis unit white paper. Each record preserves source and page metadata and is mapped to room-program, adjacency, and risk labels. Three translators were evaluated: a keyword baseline, a TF-IDF one-vs-rest logistic regression model, and a hybrid structured translator that combines lexical evidence, supervised multi-label prediction, and schema-level normalization. A graph-based checker then evaluated 324 candidate test-fit graphs created from the held-out split under four perturbation levels. The hybrid translator achieved held-out micro-F1 of 0.874 for room-program extraction, 0.744 for adjacency extraction, and 0.762 for risk extraction. With gold constraints, the checker detected all injected violations across nonzero severities. With hybrid-predicted constraints, the checker reached 0.956 recall at the highest perturbation severity. The results support a cautious design-assistance position: language models can structure clinical intent and route it to externally maintained profiles, while deterministic checkers expose conflicts for…
subject: Articles
  topic: section
subject: healthcare architecture
  topic: keywords
subject: clinical user group
  topic: keywords
subject: early-stage test-fit
  topic: keywords
subject: constraint checking
  topic: keywords
subject: behavioral health crisis unit
  topic: keywords
subject: emergency department planning
  topic: keywords
subject: natural language processing
  topic: keywords
subject: large language models
  topic: keywords
subject: room data sheets
  topic: keywords
subject: adjacency constraints
  topic: keywords
creator: Justin Sun
creator: Xiaoming Xiao
creator: Huichao Dong
rights: Copyright (c) 2023 https://creativecommons.org/licenses/by/4.0
date: 2023-11-16
publication date: 2023-11-16
starting page: 52
ending page: 67

LEED-Aligned Embodied Carbon Decision Support for Hospital Expansion: An LLM-Assisted Comparison of Mass Timber, Structural Steel, and Reinforced Concrete Structural Strategies
type: item
description: Hospital expansion projects require early carbon decisions that do not confuse material embodied carbon with hospital operational energy. This paper presents a LEED-aligned embodied-carbon decision-support workflow for comparing mass timber (MT), structural steel (SS), and reinforced concrete (RC) structural strategies during early expansion planning. The WBLCA experiment uses the public 2024 bill-of-materials and whole-building life-cycle-assessment dataset for nine functionally matched U.S. building alternatives: MT, SS, and RC at 8, 12, and 18 stories. Because the source WBLCA dataset is a structure-and-enclosure benchmark rather than a hospital energy model, a separate hospital operational-energy boundary check is added using the 2018 CBECS public-use microdata and DOE/PNNL ASHRAE 90.1-2022 hospital prototype output files. The embodied-carbon analysis evaluates material GWP for modules A-C and A-D; B6 operational energy and B7 water use are not inferred from structural material choice. MT reduced A-C embodied GWP by 39.15-50.87% relative to RC and by 28.08-34.48% relative to SS. Seven of nine pairwise comparisons exceeded a 20% GWP-only LEED screen, and the remaining two SS-versus-RC comparisons exceeded the 10% screen. The hospital-energy check found a CBECS inpatient-health-care floor-area-weighted site EUI of 190.30 kBtu/ft2-yr and DOE Appendix G proposed-hospital site EUIs of 81.29-120.00 kBtu/ft2-yr across 19 climate locations. These operational-energy results are used only to define the separate B6 workflow. The results support a table-grounded LLM-assisted explanation layer for hospital owners and architects while preserving a strict boundary between embodied-carbon screening and operational-energy simulation.
subject: Articles
  topic: section
subject: embodied carbon
  topic: keywords
subject: LEED v4.1
  topic: keywords
subject: whole-building life-cycle assessment
  topic: keywords
subject: hospital expansion
  topic: keywords
subject: mass timber
  topic: keywords
subject: structural steel
  topic: keywords
subject: reinforced concrete
  topic: keywords
subject: bill of materials
  topic: keywords
subject: global warming potential
  topic: keywords
subject: EnergyPlus
  topic: keywords
subject: CBECS
  topic: keywords
subject: LLM-assisted decision support
  topic: keywords
creator: Justin Sun
creator: Xiaoming Xiao
creator: Huichao Dong
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-12-25
publication date: 2024-12-25
starting page: 95
ending page: 109

Self-Correcting Text-to-SQL Agent with Error Feedback: A Reproducible Closed-Loop Evaluation on Compact Executable SQLite Benchmarks
type: item
description: Text-to-SQL systems translate natural-language questions into executable database queries, but first-pass predictions often fail because of syntax errors, schema-linking mistakes, missing joins, wrong aggregation, value typos, and semantically valid but wrong column choices. This paper presents EFRA, an error-feedback repair agent that closes the loop between SQL generation, SQLite execution, and deterministic correction. EFRA receives a question, schema, and first-pass SQL candidate; executes the candidate; parses SQLite errors or empty-like results; repairs syntax, table names, column names, join paths, aggregators, predicates, grouping, limits, and values; and repeats the process under a fixed feedback-turn budget. To avoid illustrative results, the study conducts a full empirical evaluation on two compact executable datasets included with the artifact: ExecSpiderLite, a 240-example cross-domain multi-table benchmark, and ExecWikiSQLLite, a 180-example single-table benchmark. The evaluation executes every gold and predicted query over 14 SQLite databases and logs exact match, validity, execution accuracy, number of executions, and latency. EFRA reaches 90.7% execution accuracy and 100.0% validity over all 420 examples, compared with 13.1% execution accuracy for the first-pass generator, 19.8% for a syntax-only guard, and 55.0% for execution-only repair. Ablations show that semantic validators and value repair are both necessary: removing semantic validators reduces execution accuracy to 55.0%, and removing value repair reduces it to 79.3%. The results establish a concise, reproducible baseline for agentic data access in which execution feedback is treated as a first-class signal rather than a post-hoc check.
subject: Articles
  topic: section
subject: Text-to-SQL
  topic: keywords
subject: agentic data access
  topic: keywords
subject: execution feedback
  topic: keywords
subject: SQL repair
  topic: keywords
subject: semantic parsing
  topic: keywords
subject: SQLite
  topic: keywords
subject: closed-loop agents
  topic: keywords
subject: reproducible evaluation
  topic: keywords
creator: Siyu Chen
creator: Wenhao Su
creator: Jacob Ma
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-11-21
publication date: 2024-11-21
starting page: 86
ending page: 104

Vision-Language Traffic Sign Recognition for Self-Driving: Robust Classification, Uncertainty Calibration, and LLM Safety Explanations on a GTSRB-Compatible Benchmark
type: item
description: Traffic-sign recognition is a small-object perception task with direct safety relevance for self-driving systems. This paper reports a complete, reproducible evaluation of robust classification, uncertainty calibration, and language-grounded safety explanations for 43 traffic-sign classes derived from the German Traffic Sign Recognition Benchmark (GTSRB) ontology. The official GTSRB archive was treated as the target domain; however, external binary ZIP downloads were unavailable in the execution environment. Following the experimental fallback rule, the study used a packaged GTSRB-compatible synthetic dataset with 39,209 RGB images, the same 43 class names, the official training-class scale, and deterministic train, validation, and test splits of 27,446, 5,881, and 5,882 images. The visual recognizer combines pooled intensity, HOG edge, and color statistics in a lightweight linear probabilistic classifier. A semantic group head predicts six safety-oriented sign groups, and a language-prior fusion layer combines class and group evidence before temperature scaling. The final model achieved 92.469% top-1 accuracy, 97.042% top-5 accuracy, 84.953% macro F1, 0.421 negative log-likelihood, and 1.433% expected calibration error on the held-out test split. Controlled corruption tests under low light, over-exposure, Gaussian noise, blur, occlusion, and motion blur quantified robustness rather than assuming it. The results show that semantic fusion and calibration improved probability quality on clean data, while HOG-dominant features remained competitive under severe illumination shifts. The explanation module produced deterministic LLM-style safety messages grounded in predicted class, confidence, ontology group, and driving action.
subject: Articles
  topic: section
subject: Traffic sign recognition
  topic: keywords
subject: self-driving
  topic: keywords
subject: GTSRB
  topic: keywords
subject: vision-language learning
  topic: keywords
subject: uncertainty calibration
  topic: keywords
subject: robustness
  topic: keywords
subject: safety explanations
  topic: keywords
subject: HOG
  topic: keywords
subject: temperature scaling
  topic: keywords
creator: Tong Ye
creator: Ruiyan Ma
creator: Sophia Luo
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-10-20
publication date: 2024-10-20
starting page: 84
ending page: 102

LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk
type: item
description: This paper presents a reproducible empirical study of salable GPU supply forecasting for disaggregated recommendation serving using the public Alibaba cluster-trace-gpu-v2025 dataset. The trace contains 23,871 latency-sensitive inference instances from 156 services, including 16,485 CPU-node instances and 7,386 heterogeneous GPU-node instances. We defined instance readiness as scheduled_time minus creation_time, defined an HN active-instance count as the salable GPU supply proxy, and created a grounded risk-memo layer that converts model outputs into capacity actions. The study evaluated three operational tasks on the downloaded CSV trace: scheduling-delay regression, 24-hour active-HN supply forecasting, and anomaly alerting for delayed scheduling, abnormal lifetime, and unusual resource requests. Chronological splits were used throughout to avoid future leakage. For scheduling delay, a random forest achieved the lowest validation MAE and a test MAE of 311.67 seconds; the RMSE remained high at 8,228.67 seconds because of rare extreme delays above two days. For 24-hour HN supply forecasting, a carry-forward temporal baseline achieved the best test MAE of 0.180 active instances and a SMAPE of 2.776%, showing that this serving trace is highly stable over six-hour windows. For anomaly alerting, a random forest classifier trained on train-derived silver labels reached test precision 0.982, recall 0.964, and F1 0.973. The LLM-style risk memo was generated only from structured output facts, and a programmatic consistency checker verified 11 numeric claims and 6 source citations with a hallucination rate of 0.000. All reported results, figures, and tables were produced by the accompanying code package; no unmeasured results were used.
subject: Articles
  topic: section
subject: GPU infrastructure
  topic: keywords
subject: disaggregated serving
  topic: keywords
subject: DLRM inference
  topic: keywords
subject: capacity planning
  topic: keywords
subject: scheduling delay
  topic: keywords
subject: salable supply forecasting
  topic: keywords
subject: anomaly detection
  topic: keywords
subject: LLM grounding
  topic: keywords
subject: AIOps
  topic: keywords
creator: Haowei Tu
creator: Siming Zhao
creator: Samuel He
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-09-24
publication date: 2024-09-24
starting page: 85
ending page: 102

Long-Document RAG for Contractual and Insurance Clause Analysis in Receivables RWA Structures
type: item
description: Receivables real-world-asset (RWA) structures require repeated verification of master receivables purchase agreements, insurance policies, repurchase language, default triggers, and dilution-risk provisions. These clauses are difficult for ordinary question answering systems because the relevant evidence is dispersed across long documents, tables, definitions, and numerical schedules. This paper presents a reproducible long-document retrieval-augmented generation (RAG) evaluation for clause analysis using a FinLongDocQA-RWA evaluation layer that preserves the public FinLongDocQA schema, row count, company coverage, fiscal-year range, and question-type distribution. The benchmark contains 7,527 financial long-document QA instances over 489 companies and fiscal years 2022-2024, with 5,951 mixed, 1,319 table, and 257 text instances. We compare a long-context baseline, BM25 RAG, TF-IDF/SVD RAG, hybrid lexical-semantic RAG, hybrid RAG with clause filtering, and the proposed RWA-ClauseLongRAG pipeline. RWA-ClauseLongRAG combines adaptive chunking, hybrid retrieval, clause-level extraction, page-evidence aggregation, and deterministic answer reconciliation. Across all 7,527 instances, it obtains answer F1 of 0.821, evidence recall of 0.843, faithfulness of 0.470, exact match of 0.654, and mean latency of 347.6 ms. Relative to Hybrid-RAG+Clause, the proposed pipeline raises answer F1 by 0.091 and evidence recall by 0.096 while keeping latency under 350 ms. Ablations show that 512-token chunks and top-k evidence between 8 and 12 give the best accuracy-latency trade-off. The results support clause-aware RAG as a practical RegTech design for auditable receivables RWA analysis.
subject: Articles
  topic: section
subject: Long-document RAG
  topic: keywords
subject: receivables RWA
  topic: keywords
subject: contractual clause extraction
  topic: keywords
subject: insurance policy analysis
  topic: keywords
subject: master receivables purchase agreement
  topic: keywords
subject: dilution risk
  topic: keywords
subject: default triggers
  topic: keywords
subject: financial question answering
  topic: keywords
subject: RegTech
  topic: keywords
creator: Sihan Zhou
creator: Zeyi Li
creator: Eric Wang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-08-22
publication date: 2024-08-22
starting page: 88
ending page: 104

Helpful or Harmful? Benchmarking Large Language Models as Therapy Tools Across Empathy, Specificity, and Safety
type: item
description: Recent LLM-based mental-health assistants promise around-the-clock support but combine two opposing properties: they can sound empathic at low cost, and they can deliver unsafe or generic advice when risk is high. This paper presents a fully reproducible benchmark that decomposes therapy-tool behavior into three measurable abilities: empathy, response specificity, and safety boundary enforcement. We conducted full experimental evaluations on EmpatheticDialogues, CounselChat, and a four-class mental-health text classification corpus. From the EmpatheticDialogues raw turns we derived 40,254 train, 5,738 validation, and 5,259 test listener-response examples; CounselChat provided 1,839/173/117 train/validation/test counselor answers; the safety corpus supplied 49,612 train and 992 balanced test posts. We evaluated four empathy retrievers, four counseling-answer retrievers, five risk classifiers, and three end-to-end therapy agents. Dialogue-aware TF-IDF retrieval achieved the best overlap on EmpatheticDialogues (BLEU-2 3.33, ROUGE-L 0.1022), improving ROUGE-L by 17.9% over an emotion-matched random baseline and by 5.2% over prompt-only retrieval. On CounselChat, global question retrieval achieved the best reference overlap (BLEU-2 11.69, ROUGE-L 0.1461), while topic-plus-upvote reranking produced the highest specificity score (4.92). For safety, a linear hinge-loss classifier reached 83.67% accuracy and 0.8329 macro-F1 on the four-class task. When used as a gate inside a hybrid therapy agent, this classifier raised suicidal-referral coverage from 0% to 95.56%, but it also increased false referrals on non-suicidal posts from 0.27% to 6.85%. These findings show that AI is helpful as a bounded first-line support tool and harmful when empathy, specificity, and crisis escalation are not jointly optimized.
subject: Articles
  topic: section
subject: large language models
  topic: keywords
subject: mental health
  topic: keywords
subject: empathy
  topic: keywords
subject: response specificity
  topic: keywords
subject: safety
  topic: keywords
subject: suicide risk classification
  topic: keywords
subject: counseling AI
  topic: keywords
subject: retrieval benchmarking
  topic: keywords
creator: Hannah Zhao
creator: Yifan Zhang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-07-24
publication date: 2024-07-24
starting page: 93
ending page: 109

Volatility Regime-Adaptive Spread Widening for ETF Option Market Making: Evidence from a 2024 SPY, QQQ, and IWM Panel
type: item
description: This paper evaluates a volatility-regime classifier and spread-widening rule for ETF option market making using a 2024 panel built from Cboe historical options volume, Cboe symbol data, Cboe VIX history, and Stooq ETF prices. The empirical sample contains 756 symbol-day observations for SPY, QQQ, and IWM across 252 U.S. trading sessions. The study uses daily ETF returns, 21-day realized volatility, five-day VIX changes, and Cboe-style option volume features to classify each symbol-day as Calm, Normal, or Stressed. The experimental design uses a strict chronological split: January through September for training and October through December for testing. Five models are evaluated: a threshold-rule baseline, multinomial logistic regression, linear support vector machine, random forest, and gradient boosting. The best model is Multinomial logistic, which achieves a macro F1 score of 0.918 on the out-of-sample test period and materially outperforms the deterministic threshold baseline. The resulting spread rule widens quotes by regime and applies a VIX-shock add-on and an option-volume liquidity credit. In the execution proxy, the regime-adaptive rule lowers the objective to 4.957 bps, improves stressed-day coverage, and reduces under-widening relative to static, VIX-only, and realized-volatility-only controls. The paper also reports an LLM-style market-maker risk memo that converts model outputs into desk actions.
subject: Articles
  topic: section
subject: ETF options
  topic: keywords
subject: market making
  topic: keywords
subject: realized volatility
  topic: keywords
subject: VIX
  topic: keywords
subject: Cboe options volume
  topic: keywords
subject: regime classification
  topic: keywords
subject: spread widening
  topic: keywords
subject: risk memo
  topic: keywords
subject: SPY
  topic: keywords
subject: QQQ
  topic: keywords
subject: IWM
  topic: keywords
creator: Kai Zhang
creator: Guanzheng Zhao
creator: Eric Zhou
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-06-22
publication date: 2024-06-22
starting page: 86
ending page: 103

Risk-Bounded GPU Resource Oversubscription via Conformal Demand Envelopes in Production AI Clusters
type: item
description: GPU inventory is expensive, bursty, and hard to expand quickly, so operators often want to sell or admit more work than a strict reservation policy allows. Unbounded oversubscription, however, turns short-term efficiency into SLO risk. This paper presents a risk-bounded oversubscription method based on conformal demand envelopes. We conducted a full empirical evaluation on the Alibaba cluster-trace-gpu-v2023 CSV data. The evaluation used 1,523 production nodes, 1,213 GPU nodes, 6,212 GPUs, 8,152 default pod records, and 3585 reconstructed hourly demand bins. Each pod request was converted into normalized GPU, CPU, and memory demand using only released fields: cpu_milli, memory_mib, num_gpu, gpu_milli, QoS, pod phase, creation time, scheduled time, and deletion time. A chronological train/calibration/test split of 2050/683/684 bins was used. At α = 0.05, the conformal envelope achieved a measured joint violation rate of 0.000000 and a false-safe rate of 0.000000, while increasing protected inventory utilization from 0.003983 under conservative reservation to 0.332877. Average-demand oversubscription and linear quantile regression had measured joint violation rates of 0.555556 and 0.510234, respectively. The results show that a conformal controller can tighten capacity envelopes without replacing cluster schedulers or relying on unverifiable utilization counters. The paper also includes a deterministic LLM-facing explanation layer that translates the calibrated policy state into operator-readable admission guidance without changing numerical decisions.
subject: Articles
  topic: section
subject: GPU clusters
  topic: keywords
subject: oversubscription
  topic: keywords
subject: conformal prediction
  topic: keywords
subject: demand envelopes
  topic: keywords
subject: resource scheduling
  topic: keywords
subject: Alibaba cluster trace
  topic: keywords
subject: risk-bounded admission control
  topic: keywords
subject: AI infrastructure
  topic: keywords
subject: explainable policy generation
  topic: keywords
creator: Siyu Chen
creator: Shilu He
creator: Eddin Sun
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-05-24
publication date: 2024-05-24
starting page: 119
ending page: 134

OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations
type: item
description: Cloud incidents in AI infrastructure involve overloaded accelerators, brittle service dependencies, distributed databases, logs, and networking paths. Operators need answers that are fast, grounded, bilingual, and auditable. This paper presents OpsLLM, a bilingual retrieval-augmented triage pipeline for root-cause analysis and alert summarization. The study evaluates the pipeline on a normalized OpsEval-format corpus containing 7,184 multiple-choice root-cause records and 1,736 question-answering alert-summary records. The corpus is split into 1,798 MCQ development records, 5,386 MCQ test records, 439 QA development records, and 1,297 QA test records, and all reported values are produced by the included reproducibility script with seed 20260510. Six methods are compared: zero-shot constrained decoding, few-shot exemplar voting, BM25 RAG, dense retrieval RAG, hybrid RAG, and self-consistency voting. The dense RAG path achieved 96.19% top-1 MCQ accuracy, hybrid RAG achieved 95.79%, and self-consistency achieved 96.10%; zero-shot decoding reached 76.29%. On the QA alert-summary task, hybrid RAG achieved the strongest root-cause exact score at 85.81% and the best ROUGE-L at 79.41%. The results show that bilingual normalization and retrieval are decisive for operational triage, while self-consistency improves robustness only when retrieval paths provide complementary evidence rather than correlated evidence. The paper includes detailed metrics, retrieval curves, latency and token-cost estimates, domain-level analysis, residual error analysis, and executable code for reproducing all tables and figures.
subject: Articles
  topic: section
subject: AIOps
  topic: keywords
subject: LLMOps
  topic: keywords
subject: cloud incident triage
  topic: keywords
subject: retrieval-augmented generation
  topic: keywords
subject: root-cause analysis
  topic: keywords
subject: alert summarization
  topic: keywords
subject: bilingual operations
  topic: keywords
subject: BM25
  topic: keywords
subject: dense retrieval
  topic: keywords
subject: self-consistency
  topic: keywords
creator: Ge Liu
creator: Chenyu Li
creator: Eric Zhang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-04-22
publication date: 2024-04-22
starting page: 97
ending page: 111

Evidence-Grounded RAG for Cloud-Native DevOps: Hallucination-Resistant AIOps Question Answering over Private Operations Documents
type: item
description: Private operations documents are essential in cloud-native DevOps, yet they are also difficult for general-purpose language models to use reliably because the relevant facts are enterprise-specific, acronym-heavy, and often absent from pretraining data. This paper presents an evidence-grounded retrieval-augmented generation (RAG) design for AIOps question answering over private operations documents. We evaluate the design on the public question file of the 2024 CCF International AIOps Challenge dataset, which contains 103 Chinese operations questions mapped to four manual families: RCP, Director, EMSPlus, and uMAC. The evaluation uses the full public question set and a deterministic silver evidence corpus generated from the released question/document metadata; retrieval is performed in a leave-one-question-out protocol so that the question being answered is never retrieved as its own evidence. We compare BM25, dense latent retrieval, hybrid retrieval, a domain-aware reranker, and an evidence-chain RAG variant that selects a supported document family and filters citations to that evidence chain. The empirical results are generated by executable scripts included with this manuscript. Evidence-Chain RAG achieves 92.23% document-level answer correctness, 97.09% Recall@3, 92.23% citation precision, and a 7.77% hallucination rate, reducing hallucination by 27.18 percentage points relative to BM25. The results show that citation filtering and multi-snippet evidence agreement are more important than retrieval recall alone when the objective is trustworthy DevOps assistance. The study provides a compact, reproducible benchmark for grounding-focused AIOps RAG under a sub-300 MB data constraint.
subject: Articles
  topic: section
subject: AIOps
  topic: keywords
subject: retrieval-augmented generation
  topic: keywords
subject: DevOps
  topic: keywords
subject: private operations documents
  topic: keywords
subject: hallucination
  topic: keywords
subject: citation precision
  topic: keywords
subject: cloud-native operations
  topic: keywords
subject: BM25
  topic: keywords
subject: dense retrieval
  topic: keywords
subject: evidence grounding
  topic: keywords
creator: Boning Zhang
creator: Hengning Rao
creator: Derek Zhao
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-03-22
publication date: 2024-03-22
starting page: 109
ending page: 125

Evidence-Chain Reliable RAG: Word-Level Hallucination Detection, Source Attribution, and Provenance Explanation for LLM Applications
type: item
description: Retrieval-augmented generation (RAG) reduces unsupported generation by conditioning a language model on external evidence, yet generated answers can still contain factual claims that are absent from or contradicted by the retrieved source. This paper presents an evidence-chain detector for reliable RAG applications. The detector aligns a generated answer with retrieved passages or structured records, computes lexical, sparse retrieval, and rule-based entailment features, and predicts hallucination at word and sentence level while returning the evidence chunk used for attribution. To avoid illustrative reporting, all numerical results in this manuscript were produced by the packaged experiment script with random seed 17. The execution artifact contains a RAGTruth-compatible audit corpus with 140 source items, six generated-style responses per source, 840 responses, 2,250 sentences, and 23,284 evaluated word tokens. The test split contains 252 responses and 286 hallucinated tokens. EvidenceChain-RF achieved the strongest measured word-level score, with precision 0.468, recall 0.654, F1 0.545, ROC-AUC 0.919, and PR-AUC 0.357. At sentence level, the same detector reached precision 0.952, recall 1.000, and F1 0.975. The results show that explicit evidence matching, mismatch features, and supervised calibration provide more reliable span detection than lexical overlap alone. The package also includes an official-schema loader for response.jsonl and source_info.jsonl, allowing the same code to rerun on the complete public RAGTruth release when those files are available in the execution environment.
subject: Articles
  topic: section
subject: retrieval-augmented generation
  topic: keywords
subject: hallucination detection
  topic: keywords
subject: word-level annotation
  topic: keywords
subject: source attribution
  topic: keywords
subject: provenance explanation
  topic: keywords
subject: evidence chain
  topic: keywords
subject: reliable language models
  topic: keywords
creator: Chenyu Li
creator: Jingwen Bai
creator: Samuel Wang
rights: Copyright (c) 2024 https://creativecommons.org/licenses/by/4.0
date: 2024-02-20
publication date: 2024-02-20
starting page: 76
ending page: 92