Scholar4AI Updated 2026-10-26 11:39 UTC+09:00

Scaled dot-product attention (Transformer)

area: Machine learning · 6 papers in the last 7 days

← Most-used equations and models

Attention(Q,K,V)=softmax(QK⊤dk)V

The core operation of Transformer models: each token's output is a weighted average of values, weighted by query-key similarity.

LaTeX: \mathrm{Attention}(Q, K, V) = \mathrm{softmax}\Big(\frac{Q K^\top}{\sqrt{d_k}}\Big) V

Names searched for: self-attention, attention mechanism, multi-head attention, scaled dot-product attention, Transformer architecture, Transformer-based, vision transformer

Key findings

  • Scaled dot-product attention (Transformer) was mentioned in 56 papers in the last 30 days (2026-09-27 to 2026-10-26), ranking #7 of the 88 equations and models mentioned.
  • By journal category: AI (55) and Operations Research (1).
  • Most of them appeared in Neural Computing and Applications (9), Knowledge-Based Systems (8) and Pattern Recognition (8).
  • Often mentioned together with Convolutional neural network (CNN) (8), Graph neural network (graph convolution) (4) and Long short-term memory (LSTM) (4).
  • Datasets named in the same papers: ImageNet (1), German Credit (UCI Statlog) (1) and WTI crude oil (1).
Papers mentioning Scaled dot-product attention (Transformer), newest first. Newest 50 of 56 papers; all of them are in /data/papers.json.
DateTitleJournalAuthorsAffiliations
2026-10-23Neuro-native paradigms of brain foundation models: evolution, taxonomy, and perspectivesArtificial Intelligence ReviewZepeng Li, Molei Zhang, Tianyi Li et al.Lanzhou University
2026-10-22Sine function-guided master–slave collaborative network for salient object detection in optical remote sensing imagesApplied IntelligenceDong Hansong, 罗子江, Zhizhu Ge et al.Guizhou University; Shunde Polytechnic; Fudan University
2026-10-22Beyond the survey: a systematic empirical study of detection and association in visual MOTArtificial Intelligence ReviewLinh Van Ma, Juhua Hu, Wei Cheng et al.Gwangju Institute of Science and Technology; University of Washington Tacoma
2026-10-21ADGB-ONet: an attention-driven gradient balancing operator network for scattering pressure field predictionApplied IntelligenceAnqiang Tan, Jinhong Wu, Yankun Sheng et al.Huazhong University of Science and Technology
2026-10-21InfoDisent: Explainability of image classification models by information disentanglementKnowledge-Based SystemsŁukasz Struski, Dawid Rymarczyk, Jacek Tabor
2026-10-20Boosting the understanding ability of diffusion model with masked image modelingPattern RecognitionJianning Pei, Yingchao Feng, Hanbo Bi et al.Chinese Academy of Sciences; Aerospace Information Research Institute; University of Chinese Academy of Sciences
2026-10-18DCDA-Net: A depthwise convolution dilated attention network with temporal weighted loss for seizure predictionNeural NetworksXingchen Dong, Haotian Li, Qiangqiang Chen et al.Fudan University; Nanjing University of Science and Technology; Nanjing University; …
2026-10-17TNIA: Target-aware transformer-based node injection attacks on graph neural networksExpert Systems with ApplicationsQiongshan Pan, Peng BaoBeijing Jiaotong University
2026-10-17AgriEyeNet: Seeing smarter in agriculture with medium-difficulty aware loss and lightweight adaptive detectionApplied IntelligenceJipei Cao, Hanxiang Wang, Mengyao Ma et al.Qufu Normal University; University of Jinan; Sejong University; …
2026-10-17Language modeling on tabular data: a survey of foundations, techniques and evolutionArtificial Intelligence ReviewYucheng Ruan, Xiang Lan, Jingying Ma et al.National University of Singapore; Institute of Data Science
2026-10-15SAR image despeckling via diffusion models with latent-space contrastive alignmentExpert Systems with ApplicationsFuyu Bo, Jing Fang, Yi Li et al.Beijing Jiaotong University; Shandong Normal University; Guizhou Province Chemical Industry Research Institute
2026-10-14Mitigating spectral decay via wavelet sparse contextual reinforcement for lightweight image super-resolutionKnowledge-Based SystemsQian Wang, Mengyang Wang, Yao Tang et al.Xi’an University of Posts and Telecommunications; Shanghai Jiao Tong University
2026-10-14Domain-adversarial anatomical graph networks for cross-user human activity recognitionInformation FusionXiaozhou Ye, Kevin I‐Kai WangUniversity of Auckland
2026-10-13DPCformer: Dual-Prior Calibration Transformer for low-light image enhancementInformation Processing & ManagementJiaxiang Wang, Pufen Zhang, Meng Wan et al.University of Science and Technology Beijing; Chinese Academy of Sciences; Computer Network Information Center; …
2026-10-13A context-aware dynamic ensemble interval-valued crude oil price forecasting systemInformation Processing & ManagementWendong Yang, Junyuan Li, Shuai Wang et al.Shandong University of Finance and Economics; Shandong Normal University
2026-10-12Balancing novelty and familiarity: Nonlinear effects of semantic progression on audience engagementInformation Processing & ManagementJingyi Wu, Shuhao Fu, Junying LiangSanta Fe Institute
2026-10-11Causal graph representation learning for robust out-of-distribution crop pest recognitionPattern RecognitionLI Guangke, Yue Teng, Chenghui Lin et al.Hefei University
2026-10-09Learning attention-enhanced relation-sensitive knowledge graph embeddings with variable receptive fields for link predictionInformation Processing & ManagementWanxin Zeng, Yuxin Ye, Zhezhe Xing et al.Jilin University; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University
2026-10-09Meta-contrastive learning with lightweight support-based query interaction for few-shot fine-grained visual classificationKnowledge-Based SystemsYuxiang Wang, Yazhou Wang, Peijia Yu et al.Guizhou University
2026-10-09A multimodal deep learning-driven model for assessing drawing composition qualityJournal Of Big DataYunhe Su, Xiaowen FengHongik University; Yonsei University
2026-10-09EEG-UDAF: A novel unsupervised domain adaptation framework leveraging dual-view frequency and spatio-temporal features for EEG-based emotion recognitionJournal Of Big DataYufei Chen, Gang Zhou, Pengwei Wang et al.Zhengzhou University
2026-10-08Cross-modal vision transformer with energy-system priors for interpretable operational pattern discoveryPattern RecognitionTianren Zhang, Liao Hui, Jimeng Zhang et al.East China University of Science and Technology; Shanghai Jiao Tong University; Chinese Academy of Sciences; …
2026-10-08Rank and data adaptive low-rank adaptation for 2-bit quantization-aware fine-tuning of ViTPattern RecognitionYuning Yang, Guowei Peng, Xiurui Xie et al.University of Electronic Science and Technology of China; Southwestern University of Finance and Economics
2026-10-06Anomaly detection in SAP S/4HANA using hybrid deep learning for multivariate time series dataJournal Of Big DataRam Reddy Jonnalagadda, B. P. Joshi, Kiran Kumar Reddy et al.Osmania University; Graphic Era Hill University; Graphic Era University; …
2026-10-06An attention-based multi-scale convolutional neural network for EEG emotional recognitionExpert Systems with ApplicationsWei Wang, Bo Zhang, ZhengTao Song et al.Southwest University; Tongji University
2026-10-05Receptive field enhanced Transformer with multi-attention mechanisms for single-image derainingKnowledge-Based SystemsJingyu Li, Zhenhai Zhang, Yilei Huang et al.Beijing Institute of Technology
2026-10-01MCA-DiTGBNet: Attention-Enabled Distributed Transformer-Based LightGBM for Anomaly Detection in IoT NetworkKnowledge-Based SystemsZeeshan Ahmed Mohammed, Mujahedullah Syed, Abdul Junaid Mohammed et al.
2026-10-01Layer-aware soft-biometric inference on OCT latent fingerprints using transformer-based modelsPattern RecognitionLingzhen Kong, Xiyuan Hu, Ning Zhang et al.Nanjing University of Science and Technology; Beijing University of Technology; Inner Mongolia Autonomous Region Meteorological Bureau; …
2026-10-01EMM-CVT: An Efficient Multi-Modal Vision Transformer for compressed video action recognitionPattern RecognitionHong Wen, Bing Li, Liang Qian et al.Beijing Jiaotong University; Nankai University
2026-10-01A global-local dual-branch network with Local Adaptive Sparse Attention for micro-expression recognitionPattern RecognitionJiawei LI, Yuanlun Xie, Zitong Yu et al.Guizhou University; Chengdu University; Dongguan University of Technology
2026-10-01A novel AF-CBA framework for accurate and interpretable EEG cognitive state classificationNeural Computing and ApplicationsBichitra Mandal, Subasish Mohapatra, Ramesh SahooIndian Institute of Technology Bhubaneswar; Odisha University of Agriculture and Technology
2026-10-01Network security monitoring based on multi-source heterogeneous data fusion and hierarchical attention mechanismNeural Computing and ApplicationsZheng Zhang, 楊啟瑞, Ling IyuEducation Department of Jiangxi Province; Jiangxi Vocational and Technical College of Communications; Capital Normal University; …
2026-10-01Alzheimer’s multimodal diagnosis: enhancing cross-attention between speech and textNeural Computing and ApplicationsKewen Ding, Azadeh Noori Hoshyar, Tanusri Bhattacharya et al.Federation University; Western Sydney University; Myer Foundation; …
2026-10-01SCTFormer: Sparse Convolution Transformer Hybrid Network with Dynamic Top-k Attention for Robust Low-light Enhancement and DetectionExpert Systems with ApplicationsJianpu Lin, Huiying Wang, Fangwei Lai et al.Fuzhou University
2026-10-01ARION: Attention-Optimized Transformer Inference on Encrypted DataExpert Systems with ApplicationsLinhan Yang, Jingwei Chen, Wangchen Dai et al.Sun Yat-sen University; Chongqing Institute of Green and Intelligent Technology; University of Chinese Academy of Sciences; …
2026-10-01SYNAPSE-X: a calibration-aware vision transformer framework for selective referral in multi-class brain tumor MRI classificationNeural Computing and ApplicationsSadaf Tabatabaee, Mohammad T. KhasawnehBinghamton University
2026-10-01ASGD-Net: An Intelligent Spectral Graph Diffusion Framework for Camouflaged Object Detection in Complex Visual EnvironmentsExpert Systems with ApplicationsAizaz Ali Shah, Kaleem Ullah, Adnan Hussain et al.Sejong University
2026-10-01DualFusion: Dual adaptive fusion for multi-view pedestrian detection via view reliability modeling and channel reweightingKnowledge-Based SystemsWeihang Kong, Yong Wang, Hui Li et al.Yanshan University; University of Exeter
2026-10-01Transformer-based lightweight model and category enhancement for road garbage segmentationNeural Computing and ApplicationsSuheng Peng, Jiacai Liao, Libo Cao et al.Hunan University; Changsha University of Science and Technology
2026-10-01Benchmarking vision transformers and convolutional networks for sugarcane leaf disease classification on a consolidated multi-source datasetNeural Computing and ApplicationsMiguel Franco-Hernández, Vladimir Mejía-Domínguez, Yakdiel Rodríguez-GalloUniversidad Don Bosco; University of El Salvador
2026-10-01Attention-guided deep transfer learning for 33-class bacterial species classification from microscopic imagesNeural Computing and ApplicationsNitya Ranjan Manihira, Prabira Kumar SethySambalpur University
2026-10-01Self-paced deep clustering: an attention model for unsupervised domain adaptive person re-identificationApplied IntelligenceQingyu Wang, Ruchun Jia, Bo Peng et al.Anhui Technical College of Industry and Economy; Sichuan University
2026-10-01FedHybrid-ViT: A Federated-Hybrid Framework for Generalizable Deepfake Detection on Heterogeneous DataMachine LearningAbdullah Mohammad Sakib, Md. Mohashin Hossain, Mahedy Hasan Foysal et al.International University of Business Agriculture and Technology; Dhaka International University; United International University
2026-10-01AdaEA++: Enhanced Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial TransferabilityInternational Journal of Computer VisionBin Chen, Jia-Li Yin, Bo‐Hao Chen et al.Minjiang University; Fujian Business University; Jiangsu Provincial Key Laboratory of Network and Information Security; …
2026-10-01TextSummNet: extractive filtering model and advanced transformer based model for abstractive summarizationNeural Computing and ApplicationsRenugadevi Somu, R. NandaKumar, A. PharthaSarathy et al.Anna University, Chennai
2026-10-01Detection of causality in sentences: a comparative study of ensemble and non-ensemble approaches across embedding paradigms with explainable AIJournal Of Big DataC. H. Rohith, K. B. Sivachandra, Neethu Mohan et al.Amrita Vishwa Vidyapeetham; Universitat Pompeu Fabra
2026-10-01Dynamic embedding and implicit relation-aware self-attention for TKGCApplied IntelligenceShuang Liu, Xiaohui Sun, Peng Chen et al.Dalian University; Dalian Minzu University; Dalian Neusoft University of Information; …
2026-10-01Parallelizable memory recurrent unitsNeural NetworksFlorent De Geeter, Gaspard Lambrechts, Damien Ernst et al.University of Liège
2026-09-30A systematic evaluation of classical, transformer-based, and foundation models for photovoltaic power time-series forecastingNeural Computing and ApplicationsDiaa Salman, Imad Alzeer, Wahib IsayedAl-Quds University
2026-09-30Dynamic modality activation for RGB-T tracking with prompt-guided enhancement and hierarchical fusionPattern RecognitionBin Fan, Feng Gao, Hongmin LiuUniversity of Science and Technology Beijing