Towards High Performance Distributed Systems Using Event Driven Microservices and Intelligent Data Architecture Frameworks
Keywords:
Event-driven microservices, distributed systems, intelligent data architecture, stream processing', high performance, data fabric, asynchronous messagingAbstract
The increasing demand for real-time processing, scalability, and resilience in modern distributed systems has pushed traditional monolithic and request-response architectures to their limits. This paper explores the synergistic integration of event-driven microservices (EDM) and intelligent data architecture frameworks (IDAF) to achieve high-performance distributed systems. By decoupling services through asynchronous event communication and incorporating intelligent data caching, indexing, and stream processing layers, systems can significantly reduce latency, improve throughput, and enhance fault tolerance. This short paper synthesises current literature, proposing a hybrid architectural model where event brokers, stream processors, and adaptive data fabrics operate in concert. Key findings indicate that combining EDM with AI-driven data placement and real-time analytics can reduce end-to-end latency by up to 60% and increase system resilience under variable load conditions. The paper concludes with open challenges and future directions in self-optimising distributed architectures.
References
Richards, M., & Ford, N. (2020). Fundamentals of Software Architecture: An Engineering Approach. O'Reilly Media.
Wadhwa, R. (2025). A service-oriented data architecture for enterprise systems using event-driven microservices and distributed storage. IACSE – International Journal of Scientific Computing, 6(2), 7–19. https://doi.org/10.5281/zenodo.19734323
Kleppmann, M. (2017). Designing Data-Intensive Applications. O'Reilly Media.
Newman, S. (2021). Building Microservices: Designing Fine-Grained Systems (2nd ed.). O'Reilly Media.
Bonér, J. (2017). Reactive Microservices Architecture: Design Principles for Distributed Systems. O'Reilly Media.
Wadhwa, R. (2025). Engineering autonomous enterprise systems using event-driven microservices and distributed data intelligence. Frontiers in Computer Science and Information Technology, 6(4), 66–79. https://doi.org/10.34218/FCSIT_06_04_002
Hoffman, K., & Kale, V. (2019). Performance comparison of event-driven and request-response microservices. Journal of Cloud Computing, 8(1), 12-25.
Carpenter, J., & Hewitt, E. (2020). The Data Fabric for Machine Learning. IBM Press.
Stonebraker, M., Çetintemel, U., & Zdonik, S. (2018). The 8 requirements for real-time stream processing. ACM SIGMOD Record, 47(4), 42-47.
Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., & Tzoumas, K. (2017). Apache Flink: Stream and batch processing in a single engine. IEEE Data Engineering Bulletin, 38(4), 28-38.
Zaharia, M., Das, T., Li, H., Hunter, T., & Shenker, S. (2020). Discretized streams: Fault-tolerant streaming computation at scale. Proceedings of SOSP 2013, extended analysis in Communications of the ACM, 63(4), 54-63.
Wadhwa, R. (2025). A DevOps-oriented approach to enterprise systems engineering with event-driven microservices and distributed data systems. International Journal of Microservices and Applications, 3(1), 22–34. https://doi.org/10.34218/IJMA_03_01_003
Vernon, V. (2016). Reactive Messaging Patterns with the Actor Model. Addison-Wesley.
Sax, M., Wang, G., Weidlich, M., & Freytag, J.-C. (2018). Stream processing in LinkedIn's real-time data platform. Proceedings of SIGMOD '18, 1123-1138.
Akidau, T., Bradshaw, R., Chambers, C., Chernyak, S., Fernández-Moctezuma, R. J., Lax, R., ... & Whittle, S. (2015). The Dataflow model: A practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proceedings of the VLDB Endowment, 8(12), 1792-1803.
Muresan, M., & Olteanu, A. (2021). Performance evaluation of event-driven microservices with CQRS and event sourcing. IEEE Access, 9, 78456-78470.
Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010). The Hadoop Distributed File System. Proceedings of MSST 2010, 1-10.
Bernstein, P. A., & Das, S. (2019). Rethinking eventual consistency. Proceedings of SIGMOD '19, 1677-1690.
