Measuring the geometric parameters of woven fabrics with AI chatbots: a benchmark and three paths to higher accuracy
Mariia Moskvina, Aleksei MoskvinPurpose
Publicly available multimodal chatbots could make fabric geometry measurement accessible in low-resource settings, yet their accuracy for this task has never been systematically evaluated. This study aims to benchmark six such chatbots and investigates whether protocol-level improvements can raise measurement accuracy without model fine-tuning, unavailable to end users.
Design/methodology/approach
One hundred woven fabrics were scanned and their ground-truth values – weave type, thread count, cover factor, twist direction and twist angle – were measured according to ISO/ASTM. Six chatbots were tested with five runs per fabric, yielding 3,000 tests. Three improvement paths were examined: intra-system averaging, prompt enrichment with known fabric data and inter-system aggregation. Errors were quantified via misclassification rate (MCR) and mean absolute percentage error (MAPE).
Findings
Baseline accuracy was far below metrological requirements (MCR = 15–46%, MAPE = 9–34%). Three repeated runs indicated variability of ± 2–4 percentage points. Intra-system averaging and prompt refinement produced small, heterogeneous changes within this variability. Inter-system aggregation reduced five errors: weave MCR 32% to 21%, thread count MAPE 14% to 7%, cover factor MAPE 12% to 7%, twist direction MCR 32% to 20% and twist angle MAPE 28% to 18%. Paired tests showed significant differences between Path 3 and matched conditions after correction.
Originality/value
Aggregating heterogeneous, non-specialised chatbot outputs reduced fabric-geometry measurement error without model training. The resulting applicability matrices illustrate the potential operational range of each parameter, providing a parameter-by-parameter framework for interpreting the trade-off between accuracy and accessibility in low-infrastructure textile screening.