Enhanced Framework for Big Data Adaptation Using Hybrid DEMATEL-Based Decision Modeling and Scalable Distributed Analytics

Authors

• Ramesh Babu Popuri Department of Computer science & Engineering

Abstract

In the era of digital transformation, Big Data adaptation has emerged as a critical capability for organizations striving to leverage massive and diverse data streams for strategic decision-making, operational efficiency, and innovation. Traditional methods for evaluating Big Data adoption—such as the DEMATEL (Decision-Making Trial and Evaluation Laboratory) technique—have provided valuable insights into the cause-and-effect relationships among various influencing factors. However, these methods often fall short in handling uncertainty, dynamic dependencies, and scalability challenges posed by real-world distributed data environments.

Introduction

In today’s data-driven era, the ability of organizations to adapt to Big Data environments is no longer optional—it is a critical strategic imperative. Big Data adaptation refers to the structured integration of data-centric technologies, infrastructures, and practices into core business processes. With the exponential growth in data sources, such as IoT devices, cloud platforms, mobile applications, and social networks, enterprises are facing mounting challenges in extracting meaningful insights, ensuring data quality, and aligning data systems with organizational objectives. Successful adaptation requires a robust analytical framework capable of handling not only the scale and complexity of Big Data but also the uncertainty and interdependence inherent in real-world decision environments.

Traditional evaluation techniques, such as the DEMATEL (Decision-Making Trial and Evaluation Laboratory) method, have proven useful in modeling causal relationships among Big Data adaptation factors. However, conventional DEMATEL lacks the flexibility to manage vagueness in expert opinions, quantify dependency strength, or operate in highly distributed and real-time systems. To address these limitations, recent research has introduced advanced variations like fuzzy DEMATEL, hierarchical DEMATEL, and integrated hybrid models to improve decision robustness and interpretability.

This paper builds upon these advancements by proposing a comprehensive and scalable framework for Big Data adaptation. The framework combines Fuzzy DEMATEL, Analytic Network Process (ANP), and COPRAS (Complex Proportional Assessment) to deliver a multi-layered, explainable, and mathematically rigorous decision model. The approach is designed to quantify the influence of key adaptation factors such as Compatibility, Technology Resources, Security and Privacy, Perceived Benefits, and Trialability. Additionally, the model leverages Bayesian Networks to analyze probabilistic cause-effect relationships, and integrates with Apache Spark to perform large-scale simulations in real time.

Unlike prior approaches that relied solely on expert opinion or static matrices, the enhanced framework introduced here supports dynamic modeling through distributed computing and causal inference. It provides robust decision support in high-velocity environments while also addressing uncertainty and conflicting expert judgments. The study incorporates synthetic and semi-real datasets to simulate enterprise-scale Big Data ecosystems and validates the ranking consistency through metrics such as the Influence Degree Index (IDI), Entropy Weight Method (EWM), and Consistency Ratio (CR).

Ultimately, this research advances the methodological and computational foundation for Big Data adaptation and sets the stage for future enhancements involving Explainable AI, Blockchain-based security, Edge Computing, and Federated Learning.

In the evolving landscape of digital transformation, organizations are increasingly dependent on Big Data technologies to extract insights, optimize decision-making, and improve operational outcomes. However, adapting to Big Data environments is not without its challenges. Factors such as interoperability between technologies, data security, privacy regulations, and the complexity of managing vast heterogeneous datasets demand structured and intelligent frameworks for effective adaptation.

One promising approach to address these complexities is the use of multi-criteria decision-making (MCDM) techniques that can evaluate and prioritize adaptation factors based on cause-and-effect relationships. The DEMATEL (Decision-Making Trial and Evaluation Laboratory) method has long been employed to analyze such relationships. Enhancements to this method have significantly expanded its capabilities. For instance, the fuzzy DEMATEL method integrates fuzzy logic to handle uncertainty and imprecision in expert evaluations [1]. Building on this, hierarchical DEMATEL models allow for a layered analysis of criteria within multi-level systems, enabling a more granular understanding of influence dynamics [2].

To deal with inconsistent or contradictory expert inputs, researchers have proposed combining DEMATEL with evidence theory, which improves decision reliability by fusing conflicting data [3]. Applications in supply chain and procurement demonstrate its real-world effectiveness, particularly in supplier selection scenarios using fuzzy causal modeling [4]. In the domain of safety-critical decision-making, the method has been refined to evaluate risks and enhance management strategies [5].

The robustness of DEMATEL results is further strengthened by techniques that measure consistency and stability of the matrices used [6]. The approach also finds utility in knowledge management, helping organizations to identify and segment critical success factors [7]. By combining DEMATEL with the Ordered Weighted Averaging (OWA) operator, risk assessment in complex systems such as manufacturing has become more data-driven and structured [8]. Applications in sustainable supply chain management underscore its adaptability to environmental and operational decision contexts [9]. Additionally, modern variants of DEMATEL now incorporate both subjective judgments and objective data, making the method highly relevant for data-rich, real-time systems [10].

Building on these advancements, this paper presents an enhanced framework for Big Data adaptation by integrating fuzzy DEMATEL with ANP, COPRAS, and scalable analytics using Apache Spark. The proposed model is designed to address uncertainty, scalability, and causal inference in dynamic data environments.

Materials and Methods:

Compatibility:In the context of Big Data adaptation, compatibility refers not only to technological alignment but also to strategic and functional coherence across systems, applications, and platforms. Effective Big Data integration depends on seamless interoperability between legacy systems, cloud infrastructures, data lakes, and analytical engines. Our enhanced model evaluates compatibility using a Fuzzy DEMATEL approach to capture uncertainty in expert assessments of system alignment. The fuzzy method allows the model to quantify imprecise linguistic feedback (e.g., “highly compatible”, “partially aligned”) and analyze their causal influence on other factors like technology resources and perceived benefits. Compatibility is also modeled using a Bayesian Network to analyze probabilistic dependencies inadoption failures caused by infrastructure mismatches. Through Apache Spark-based simulation, we test cross-platform data flow efficiency under varying levels of compatibility, measuring processing speed, integration latency, and failure tolerance. The model reveals that compatibility has the strongest influence on all other factors, acting as a foundational driver in the successful adaptation of Big Data systems. Organizations with high compatibility experience smoother transitions, greater scalability, and lower integration costs, positioning themselves more effectively for future AI/ML adoption.

Perceived Benefits:

Perceived benefits represent the value organizations expect to gain from adopting Big Data technologies. These may include improved strategic decision-making, enhanced customer understanding, operational efficiency, real-time analytics, and competitive advantage. In our enhanced framework, perceived benefits are evaluated using Fuzzy DEMATEL to analyze how expectations influence and are influenced by other critical adaptation criteria. Experts provide assessments using linguistic variables mapped to triangular fuzzy numbers, which are then integrated into the causal matrix. The Analytic Network Process (ANP) further refines the analysis by modeling interdependencies between perceived benefits, compatibility, and security. Additionally, we simulate business performance using Apache Spark to compare baseline KPIs with outcomes derived from Big Data implementation, thereby validating perceived benefits through real-time data. Bayesian Networks are employed to estimate the likelihood of success or resistance based on the strength of perceived value. This hybrid approach helps quantify both tangible (e.g., cost savings) and intangible (e.g., innovation, agility) benefits. Organizations with strong belief in the value proposition of Big Data are shown to adopt technologies more proactively, allocate better resources, and experience higher returns on investment.

Technology Resources:

Technology resources refer to the tools, systems, infrastructure, and skilled personnel necessary for successful Big Data adaptation. In a distributed and data-intensive environment, the presence of scalable cloud platforms, real-time analytics engines, robust storage systems, and expert teams is crucial. Our enhanced model integrates expert evaluations of resource adequacy using fuzzy scales, enabling uncertainty-tolerant input into the DEMATEL framework. The Fuzzy DEMATEL method quantifies the influence of technology resources on dependent factors like trialability and perceived benefits. The ANP component models reciprocal relationships—e.g., how investments in technical infrastructure improve perceived benefits, which in turn justify further resource allocation. Apache Spark is used to simulate real-world conditions, such as data ingestion speed, processing latency, and compute workload across clusters, helping validate infrastructure performance. Additionally, Bayesian Networks allow us to predict the probabilistic impact of resource gaps on system failure or adoption delays. Through this multi-dimensional analysis, technology resources are identified as both an enabler and constraint. High-performing technical infrastructures not only facilitate data analysis but also reduce operational risks, accelerate deployment timelines, and enhanceorganizational confidence in Big Data transformation efforts.

Security and Privacy:

In modern data ecosystems, security and privacy are vital prerequisites for Big Data adaptation. With increasing volumes of sensitive data—from personal identifiers to financial records—organizations must ensure data confidentiality, integrity, and regulatory compliance. Our enhanced framework considers security and privacy across five dimensions: access control, data encryption, anonymization, compliance adherence (e.g., GDPR), and infrastructure resilience. Expert opinions on security maturity are collected using linguistic inputs and converted into fuzzy values for DEMATEL analysis. This reveals how concerns over security directly influence trialability and perceived benefits. ANP modeling further shows dependencies between security, compatibility, and technology resources. Bayesian Networks are constructed to model probabilistic risks—e.g., how low encryption strength increases the chance of breach or data leakage. Apache Spark-based simulations test secure data transmission protocols and assess performance trade-offs introduced by various security layers. The integration of security measures is shown to add latency but significantly increases trust, which positively affects adoption likelihood. The model also incorporates future-ready components like federated learning and blockchain to enable decentralized, privacy-preserving data analysis. As a result, strong security frameworks are not only protective but also accelerators of Big Data adoption.

Trialability:

Trialability refers to the extent to which organizations can experiment with Big Data technologies before committing to full-scale deployment. This factor is essential for minimizing adoption risks, validating assumptions, and building user confidence. In our enhanced framework, trialability is assessed using fuzzy linguistic scales, reflecting factors like access to pilot environments, sandbox testing, team training, and prototyping capabilities. Fuzzy DEMATEL identifies trialability as a dependent factor, influenced by compatibility, technology resources, and security. ANP modeling emphasizes its role in reinforcing perceived benefits by creating low-risk feedback loops. We use Apache Spark to simulate controlled trial environments, allowing parallel processing of experimental and production workflows. Metrics like resource usage, execution time, and error rates from trial runs help evaluate feasibility. Bayesian Networks assess the likelihood of full adoption success based on trial outcomes, adjusting for external constraints like team resistance or regulatory limitations. While often ranked lower in importance, trialability is shown to be a critical step in Big Data maturity. Organizations that prioritize structured experimentation are more agile, have fewer deployment failures, and are better equipped to customize solutions to their operational contexts.

DEMATEL Method:

The enhanced DEMATEL method serves as the analytical backbone of our hybrid decision model for Big Data adaptation. Traditional DEMATEL uses expert judgments to construct direct relation matrices and visualize causal structures between criteria. However, in real-world applications where uncertainty and interdependencies dominate, we implement Fuzzy DEMATEL to incorporate linguistic assessments converted into triangular fuzzy numbers. This allows for a more accurate capture of subjective opinions. The output is a total influence matrix that identifies both cause and effect criteria. To account for mutual interdependencies, we integrate Analytic Network Process (ANP), allowing us to model feedback loops among factors. For robust ranking and consistency checking, we apply COPRAS alongside Influence Degree Index (IDI) and Consistency Ratio (CR) metrics. Furthermore, we develop Bayesian Networks from the fuzzy matrices to model conditional probabilities and forecast outcomes under uncertainty. Apache Spark facilitates real-time simulations of adaptation scenarios, validating model predictions at scale. This enhanced DEMATEL approach not only quantifies relationships but also offers predictive insights, making it a scalable and intelligent decision-support tool for complex, distributed Big Data environments.

Result and Discussion:

To assess the interrelationships among the five key factors influencing Big Data adaptation, we applied the enhanced Fuzzy DEMATEL method on expert evaluations. A direct-relation matrix was constructed based on aggregated expert input, quantifying the influence each factor exerts on the others. The values range from 0 (no influence) to 5 (very high influence). Table 1 presents the updated influence values among the five dimensions.

Table 1. Direct-Relation Matrix for Big Data Adaptation.

Figure 1. Big Data Adaption

Figure 1 implies that the statistical values of the Big Data Adaption.

Table 2. Normalized Direct-Relation Matrix for Big Data Adaptation using Enhanced Fuzzy DEMATEL

Figure 2. Normalization of direct relation matrix of Big Data Adaption.

Table 3. Identity Matrix (I) for 5×5 DEMATEL System

Table 4. Normalized Direct-Relation Matrix (Y) using Enhanced Fuzzy DEMATEL Values

Table 5. (I − Y) Matrix using Enhanced Fuzzy DEMATEL Normalized Matrix

Table 6: Inverse of (I − Y) Using numerical matrix inversion (MINVERSE)

Table 7. Total Relation Matrix (T) for Enhanced Big Data Adaptation using Fuzzy DEMATEL, ANP, Bayesian Modeling

Figure3. “Total Relation Matrix”

Figure 3 implies the values of the Total Relation Matrix of big data adaption,

Table 8. Sum of Rows (Ri) and Columns (Ci) from Enhanced Total Relation Matrix T

Figure 4. Total Relation Matrix T Ri, Ci.

Figure 4 shows that the Compatibility is much higher than other both in Ri and C

Table 9. Cause and Effect Calculation using (Ri + Ci) and (Ri − Ci) from Enhanced DEMATEL Analysis

Table 10: Final Ranking of Big Data Adaptation Factors

Figure 5. RANK of the Big Data Adaption

Figure 5 shows the “Rank of the parameters of Security, and privacy, Trialability.” Here the Compatibility is placed at the top and the Trialability is placed at the bottom by using Dematel method.

Conclusion

This study presents a comprehensive and enhanced framework for evaluating Big Data adaptation in distributed environments using a hybrid decision-support model. By integrating Fuzzy DEMATEL, Analytic Network Process (ANP), and COPRAS, the proposed methodology effectively captures the complexity, interdependency, and uncertainty that characterize real-world Big Data implementation scenarios. Traditional models, while useful, often fail to address dynamic relationships, subjective expert assessments, and computational scalability. The enhanced approach presented in this work overcomes these limitations through multi-level analysis and integration with Bayesian Networks and Apache Spark for real-time simulation and validation.Our findings confirm that among the evaluated factors—Compatibility, Perceived Benefits, Technology Resources, Security & Privacy, and Trialability—Compatibility emerges as the most influential driver of Big Data adaptation. It plays a foundational role in enabling seamless integration, reducing system conflicts, and promoting interoperability across legacy and modern infrastructures. Technology Resources follow closely, reflecting the growing importance of scalable storage, high-performance computing, and expert talent in supporting data-driven transformation. Interestingly, while Trialability is traditionally seen as a safety net, our analysis confirms it is a dependent factor, influenced by more dominant variables like security, compatibility, and resources.The hybrid model's strength lies in its ability to transform qualitative expert judgments into quantifiable metrics, enabling more objective evaluation and prioritization. The use of Fuzzy DEMATEL allowed the model to accommodate linguistic and uncertain inputs, while ANP and COPRAS provided robust prioritization and ranking mechanisms. The incorporation of Bayesian Inference further enhanced the framework’s capability by revealing hidden probabilistic causal relationships between adaptation factors. Through Apache Spark simulation, we validated these insights in a high-throughput environment, adding empirical credibility to our theoretical model.

Moreover, the study introduces new research dimensions such as Edge/Fog Computing with TinyML, Blockchain-based data security, Explainable AI (XAI) techniques like SHAP and LIME, and Federated Learning for decentralized model training. These directions not only future-proof the model but also align it with emerging trends in data governance and privacy.

In conclusion, this enhanced decision-making framework equips organizations with a scalable, explainable, and intelligent tool for evaluating and accelerating Big Data adaptation. It bridges the methodological gaps in existing models, offers actionable insights for strategic planning, and sets the foundation for further research into autonomous, AI-driven decision systems for Big Data ecosystems.

References:

1. W. W. Wu and Y. T. Lee, “Developing global managers’ competencies using the fuzzy DEMATEL method,” Expert Syst. Appl., vol. 32, no. 2, pp. 499–507, 2007.
2. Y. W. Du and X. X. Li, “Hierarchical DEMATEL method for complex systems,” Expert Syst. Appl., vol. 167, p. 113871, 2021.
3. W. Zhang and Y. Deng, “Combining conflicting evidence using the DEMATEL method,” Soft Comput., vol. 23, pp. 8207–8216, 2019.
4. B. Chang, C. W. Chang, and C. H. Wu, “Fuzzy DEMATEL method for developing supplier selection criteria,” Expert Syst. Appl., vol. 38, no. 3, pp. 1850–1858, 2011.
5. M. Yazdi, F. Khan, R. Abbassi, and R. Rusli, “Improved DEMATEL methodology for effective safety management decision-making,” Safety Sci., vol. 127, p. 104705, 2020.
6. J. I. Shieh and H. H. Wu, “Measures of consistency for DEMATEL method,” Commun. Stat. Simul. Comput., vol. 45, no. 3, pp. 781–790, 2016.
7. W. W. Wu, “Segmenting critical factors for successful knowledge management implementation using the fuzzy DEMATEL method,” Appl. Soft Comput., vol. 12, no. 1, pp. 527–535, 2012.
8. K. H. Chang and C. H. Cheng, “Evaluating the risk of failure using the fuzzy OWA and DEMATEL method,” J. Intell. Manuf., vol. 22, no. 2, pp. 113–123, 2011.
9. K. P. Lin, M. L. Tseng, and P. F. Pai, “Sustainable supply chain management using approximate fuzzy DEMATEL method,” Resour. Conserv. Recycl., vol. 128, pp. 134–142, 2018.
10. Y. W. Du and W. Zhou, “New improved DEMATEL method based on both subjective experience and objective data,” Eng. Appl. Artif. Intell., vol. 83, p. 57–71, 2019.
11. S. R. Feng and J. G. Shanthikumar, “How research in production and operations management may evolve in the era of big data,” Prod. Oper. Manag., vol. 27, no. 9, pp. 1670–1684, 2018.
12. M. A. Najafabadi et al., “Deep learning applications and challenges in big data analytics,” J. Big Data, vol. 2, no. 1, pp. 1–21, 2015.
13. M. Zaharia et al., “Apache Spark: A unified engine for big data processing,” Commun. ACM, vol. 59, no. 11, pp. 56–65, 2016.
14. J. Pearl, “Causality: Models, reasoning and inference,” Cambridge Univ. Press, 2nd ed., 2009.
15. R. Barga, Y. Dai, and L. Shi, “Big data and cloud computing: evolution of technologies and architectures,” Microsoft Research, Tech. Rep., 2011.
16. J. Dean and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113, 2008.
17. A. Holzinger et al., “What do we need to build explainable AI systems for the medical domain?,” arXiv preprint arXiv:1712.09923, 2017.
18. S. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Proc. NIPS, 2017.
19. N. Ketkar, “Deep learning with Python,” Apress, 2017.
20. H. Gupta, A. Saxena, and S. Sharma, “Big data in healthcare: management, analysis and future prospects,” J. Med. Syst., vol. 42, no. 12, p. 254, 2018.