Authors
Keywords
Abstract
This study presents a comprehensive performance evaluation of five prominent machine learning (ML) pipeline platforms—Kubeflow Pipelines, AWS SageMaker Pipelines, Azure ML Pipelines, Databricks MLflow Pipelines, and Google Vertex AI Pipelines—using the Additive Ratio Assessment (ARAS) method. The evaluation focuses on four critical parameters: model training speed, autoscaling efficiency, pipeline failure rate, and data transfer latency. By applying the ARAS methodology, the alternatives are ranked based on their overall utility and optimality. The results indicate that Google Vertex AI Pipelines outperform others in terms of combined efficiency and reliability, while AWS SageMaker Pipelines excel in latency and failure management. The findings provide valuable insights for organizations in selecting the most suitable ML pipeline framework for scalable and robust AI deployment. Research Significance: With the increasing adoption of artificial intelligence and machine learning in enterprise operations, the need for efficient, reliable, and scalable ML pipeline platforms has become paramount. This research addresses the gap in comparative, data-driven evaluations of such platforms by offering a structured decision-making approach. The findings empower stakeholders—data scientists, ML engineers, and decision-makers—to make informed platform selections based on objective performance data rather than subjective assessments. Methodology: ARAS The Additive Ratio Assessment (ARAS) method is employed to assess and rank the alternatives.
ARAS is a multi-criteria decision-making (MCDM) approach that normalizes performance data, assigns weights based on parameter importance, and computes optimality and utility degrees for each alternative. This enables a quantitative comparison of the ML platforms by accounting for both maximization and minimization objectives inherent in different evaluation parameters. Alternatives Evaluated: Kubeflow Pipelines, AWS SageMaker Pipelines, Azure ML Pipelines, Databricks MLflow Pipelines, Google Vertex AI Pipelines. Evaluation Parameters: Model Training Speed (Maximize), Autoscaling Efficiency (Maximize), Pipeline Failure Rate (Minimize), Data Transfer Latency (Minimize) These parameters represent a balanced view of performance, scalability, and reliability critical to modern ML operations. Result: The ARAS-based evaluation reveals that Google Vertex AI Pipelines achieve the highest optimality and utility scores, securing the top rank due to superior training speed and autoscaling capabilities. Databricks MLflow Pipelines follow closely in second place, showing competitive performance but higher failure and latency rates. Kubeflow Pipelines rank third with balanced metrics, while AWS SageMaker Pipelines and Azure ML Pipelines occupy the fourth and fifth positions respectively, each with specific strengths in reliability or consistency but falling short in overall optimization.
