Transformer-Based Discovery of Dynamic Relationships in High-Frequency Financial Time Series

Anna Vogel; Michael Kraus

doi:10.54097/dg5nnh24

Authors

Anna Vogel
Michael Kraus

DOI:

https://doi.org/10.54097/dg5nnh24

Keywords:

High-Frequency Trading, Time Series Analysis, Transformer Architecture, Multi-Head Attention, Temporal Pattern Recognition, Market Microstructure

Abstract

High-frequency financial data presents unique challenges for relationship discovery due to its complex temporal dependencies, microstructure effects, and rapidly evolving market dynamics. This paper presents a novel transformer-based framework specifically designed to discover and analyze dynamic relationships in high-frequency financial time series. Our approach leverages multi-head self-attention mechanisms of transformers to capture both short-term microstructure patterns and long-term dependencies while maintaining interpretability for financial practitioners. We introduce adaptive temporal position encoding that accounts for irregular trading intervals and market microstructure effects, specialized multi-head attention architectures designed to capture different types of market relationships at various scales, and temporal pattern recognition techniques that can identify time-varying correlations, lead-lag effects, and volatility clustering patterns. The framework incorporates advanced attention mechanisms with temporal pattern extraction capabilities and hyperparameter optimization strategies specifically designed for high-frequency data characteristics. Experimental evaluation demonstrates superior performance in discovering meaningful relationships compared to traditional econometric methods and standard deep learning approaches, with validation loss improvements of up to 40% across different model configurations. Our results reveal previously unknown intraday relationship patterns, provide insights into microstructure-driven correlations, and demonstrate the framework's ability to adapt to changing market regimes through attention weight analysis.

Downloads

Download data is not yet available.

References

[1] Addy, W. A., Ajayi-Nifise, A. O., Bello, B. G., Tula, S. T., Odeyemi, O., & Falaiye, T. (2024). Machine learning in financial markets: A critical review of algorithmic trading and risk management. International Journal of Science and Research Archive, 11(1), 1853-1862.

[2] Flas, M. (2025). Predicting Extreme Price Movements in Technology Stocks: A Study of High-Frequency Trading Dynamic.

[3] Dutta, C., Karpman, K., Basu, S., & Ravishanker, N. (2023). Review of statistical approaches for modeling high-frequency trading data. Sankhya B, 85(Suppl 1), 1-48.

[4] Chopra, R. (2025). Essentials of Time Series Econometrics. Educohack Press.

[5] Kamatala, S., Jonnalagadda, A. K., & Naayini, P. (2025). Transformers beyond nlp: Expanding horizons in machine learning. Yes it was accepted by IRE Journals, 8(7).

[6] Hasan, F., & Huang, H. (2023). MALS-Net: A multi-head attention-based LSTM sequence-to-sequence network for socio-temporal interaction modelling and trajectory prediction. Sensors, 23(1), 530.

[7] Nie, Y., Kong, Y., Dong, X., Mulvey, J. M., Poor, H. V., Wen, Q., & Zohren, S. (2024). A survey of large language models for financial applications: Progress, prospects and challenges. arXiv preprint arXiv:2406.11903.

[8] Zheng, W., & Liu, W. (2025). Symmetry-Aware Transformers for Asymmetric Causal Discovery in Financial Time Series. Symmetry.

[9] Balaji, K. (2025). Revolutionizing High-Frequency Trading: The Impacts of Financial Technology and Data Science Innovations. Machine Learning and Modeling Techniques in Financial Data Science, 103-124.

[10] Rasyid, A. (2024). The Art and Science of Financial Econometrics: Applications, Challenges, and Future Directions. Advances in Economics & Financial Studies, 2(1), 44-52.

[11] Muhammad, A., Aliyu, J. N., Adetunji, A. L., Adesugba, A. K., Mike, M. E. E., & Abdulmalik, M. (2023). Fractal geometry in high-frequency trading: Modeling market microstructure and price dynamics. Saudi J Econ Fin, 7(11), 484-488.

[12] Chang, P., Pienaar, E., & Gebbie, T. (2021). The Epps effect under alternative sampling schemes. Physica A: Statistical Mechanics and its Applications, 583, 126329.

[13] Pattanayak, A. M., Swetapadma, A., & Sahoo, B. (2024). Exploring different dynamics of recurrent neural network methods for stock market prediction-a comparative study. Applied Artificial Intelligence, 38(1), 2371706.

[14] Junior, E. H. (2025). High-frequency data and machine learning. In Impacts of Innovation and Cognition in Management (pp. 183-204). IGI Global.

[15] Mehra, A., & Singh, S. P. (2024). Event-driven architectures for real-time error resolution in high-frequency trading systems. International Journal of Research in Modern Engineering and Emerging Technology, 12(12), 671.

[16] Chogii, R. (2021). Market Information Risk, Trading Activity, Organizational Characteristics and Price Discovery for Stocks Listed at the Nairobi Securities Exchange (Doctoral dissertation, University of Nairobi).

[17] Ji, E., Wang, Y., Xing, S., & Jin, J. (2025). Hierarchical Reinforcement Learning for Energy-Efficient API Traffic Optimization in Large-Scale Advertising Systems. IEEE Access.

[18] Jin, J., Xing, S., Ji, E., & Liu, W. (2025). XGate: Explainable Reinforcement Learning for Transparent and Trustworthy API Traffic Management in IoT Sensor Networks. Sensors (Basel, Switzerland), 25(7), 2183.

[19] Wang, J., Liu, J., Zheng, W., & Ge, Y. (2025). Temporal Heterogeneous Graph Contrastive Learning for Fraud Detection in Credit Card Transactions. IEEE Access.

[20] Mai, N. T., Cao, W., & Liu, W. (2025). Interpretable Knowledge Tracing via Transformer-Bayesian Hybrid Networks: Learning Temporal Dependencies and Causal Structures in Educational Data. Applied Sciences, 15(17), 9605.

[21] Cao, W., Mai, N. T., & Liu, W. (2025). Adaptive knowledge assessment via symmetric hierarchical Bayesian neural networks with graph symmetry-aware concept dependencies. Symmetry, 17(8), 1332.

[22] Mai, N. T., Cao, W., & Wang, Y. (2025). The global belonging support framework: Enhancing equity and access for international graduate students. Journal of International Students, 15(9), 141-160.

[23] Tan, Y., Wu, B., Cao, J., & Jiang, B. (2025). LLaMA-UTP: Knowledge-Guided Expert Mixture for Analyzing Uncertain Tax Positions. IEEE Access.

[24] Cao, J., Zheng, W., Ge, Y., & Wang, J. (2025). DriftShield: Autonomous fraud detection via actor-critic reinforcement learning with dynamic feature reweighting. IEEE Open Journal of the Computer Society.

[25] Chen, S., Liu, Y., Zhang, Q., Shao, Z., & Wang, Z. (2025). Multi-Distance Spatial-Temporal Graph Neural Network for Anomaly Detection in Blockchain Transactions. Advanced Intelligent Systems, 2400898.

[26] Zhang, X., Chen, S., Shao, Z., Niu, Y., & Fan, L. (2024). Enhanced Lithographic Hotspot Detection via Multi-Task Deep Learning with Synthetic Pattern Generation. IEEE Open Journal of the Computer Society.

[27] Zhang, Q., Chen, S., & Liu, W. (2025). Balanced Knowledge Transfer in MTTL-ClinicalBERT: A Symmetrical Multi-Task Learning Framework for Clinical Text Classification. Symmetry, 17(6), 823.

[28] Shao, Z., Wang, X., Ji, E., Chen, S., & Wang, J. (2025). GNN-EADD: Graph Neural Network-based E-commerce Anomaly Detection via Dual-stage Learning. IEEE Access.

[29] Li, P., Ren, S., Zhang, Q., Wang, X., & Liu, Y. (2024). Think4SCND: Reinforcement Learning with Thinking Model for Dynamic Supply Chain Network Design. IEEE Access.

[30] Liu, Y., Ren, S., Wang, X., & Zhou, M. (2024). Temporal logical attention network for log-based anomaly detection in distributed systems. Sensors, 24(24), 7949.

[31] Ren, S., Jin, J., Niu, G., & Liu, Y. (2025). ARCS: Adaptive Reinforcement Learning Framework for Automated Cybersecurity Incident Response Strategy Optimization. Applied Sciences, 15(2), 951.