Application of the One-Step Second-Derivative Method for Solving the Transient Distribution in Markov Chain
Received: 2 November 2024 | Revised: 13 December 2024 | Accepted: 1 January 2025 | Online: 3 April 2025
Corresponding author: Zeina Mueen
Abstract
Markov chains are an application of stochastic models in operation research, helping the analysis and optimization of processes with random events and transitions. The method that will be deployed to obtain the transient solution to a Markov chain problem is an important part of this process. The present paper introduces a novel Ordinary Differential Equation (ODE) approach to solve the Markov chain problem. The probability distribution of a continuous-time Markov chain with an infinitesimal generator at a given time is considered, which is a resulting solution of the Chapman-Kolmogorov differential equation. This study presents a one-step second-derivative method with better accuracy in solving the first-order Initial Value Problems (IVPs) compared to other approaches found in the literature, which is verified by the obtained solutions. The determination of the transient solutions for Markov chains is presented using the proposed method. The results show better accuracy in solving the transient distribution in Markov chains, which implies that there is an improved assurance in adopting this approach in future studies of the Markov chain modeling process for predicting future events based on the current state of a process. Future studies on Markov chain modeling could adopt the introduced method to predict future events based on the current state of a process.
Keywords:
transient distribution, Chapman-Kolmogorov, differential equation, numerical method, initial value problemDownloads
References
W. J. Stewart, Introduction to the Numerical Solution of Markov Chains, First Edition. Princeton, N.J: Princeton University Press, 1994.
M. Bilgen and N. Altın, "An Overview on Reliability Analysis and Evaluation Methods Applied to Smart Grids," Gazi University Journal of Science Part C: Design and Technology, vol. 9, no. 4, pp. 645–660, Dec. 2021.
A. S. Agbam and E. O. Udo, "Application of Markov Chain (MC) Model to the Stochastic Forecasting of Stocks Prices in Nigeria: The Case Study of Dangote Cement," International Journal of Applied Science and Mathematical Theory, vol. 06, no. 01, pp. 14–33, 2020.
L. A. F. Al-Ani and A. D. K. Alhiyali, "Using Markov Chains to Predict Productivity of Maize in Iraq for the Period (2019-2025)," Applied Economics and Finance, vol. 8, no. 5, pp. 1–9, Aug. 2021.
S. K. Hamza, D. A. Ahmed, and S. A. Hussein, "Forecasting the exchange rate of the Iraqi dinar against the US dollar using Markov chains," Periodicals of Engineering and Natural Sciences, vol. 08, no. 02, pp. 626–631, Jun. 2020.
W. A. S. Ashoura and M. T. Ghafil, "Using Markov Chains to Predict the Probability of Change in Cumulative Rates for Students of the College of Administration and Economics / University of Basrah," Journal of Al-Qadisiyah for Computer Science and Mathematics, vol. 11, no. 4, pp. 30–38, Dec. 2019.
A. S. Alsaqqar, B. H. Khudair, and R. K. Jbbar, "Condition Assessment and Rehabilitation for Trunk Sewer Deterioration based on Semi-Markov Model," Association of Arab Universities Journal of Engineering Sciences, vol. 25, no. 5, pp. 14–27, 2018.
A. O. Hamdin and M. M. F. Hussein, "An application of Wavelet Markov Chains Model to Study Earthquake Occurrence," Tikrit Journal of Administrative and Economic Sciences, vol. 19, no. 61, pp. 334–353, Mar. 2023.
O. Agboola and A. S. Nehad, "On the Application of Matrix Scaling and Powering Methods of Small State Spaces for Solving Transient Distribution in Markov Chain," Fudma Journal of Sciences, vol. 6, no. 1, pp. 135–140, Mar. 2022.
T. Freiheit, "A discrete state Markov chain model and decomposition solution method for the transient behaviour of buffered long serial transfer lines," Journal of Manufacturing Systems, vol. 68, pp. 493–507, Jun. 2023.
X. Wu, H. Yan, and L. Li, "Numerical Method for Reliability Analysis of Phased-Mission System Using Markov Chains," Communications in Statistics - Theory and Methods, vol. 41, no. 21, pp. 3960–3973, Nov. 2012.
S. O. Agboola and N. I. Badmus, "The Application of Runge-Kutta and Backward Differentiation Methods for Solving Transient Distribution in Markov Chain," Nigerian Journal of Mathematics and Applications, vol. 31, pp. 191–201, Jan. 2021.
V. Suñé and J. A. Carrasco, "Implicit ODE solvers with good local error control for the transient analysis of Markov models," Applied Mathematics and Computation, vol. 293, pp. 96–111, Jan. 2017.
R. B. Sidje, K. Burrage, and S. MacNamara, "Inexact Uniformization Method for Computing Transient Distributions of Markov Chains," SIAM Journal on Scientific Computing, vol. 29, no. 6, pp. 2562–2580, Jan. 2007.
N. C. Viswanath, "Transient study of Markov models with time-dependent transition rates," Operational Research, vol. 22, no. 3, pp. 2209–2243, Jul. 2022.
M. S. Raza, "Using Time-Inhomogeneity Markov Chain For Testing Kidney Diseases Departures: Apply Study For Razgari Hospital in Erbil-Iraq," Iraqi Journal of Statistical Sciences, vol. 20, no. 2, pp. 113–121, Dec. 2023.
J. G. Oghonyon, S. A. Okunuga, K. S. Eke, and O. A. Odetunmibi, "Block Milne’s Implementation For Solving Fourth Order Ordinary Differential Equations," Engineering, Technology & Applied Science Research, vol. 8, no. 3, pp. 2943–2948, Jun. 2018.
S. O. Agboola, O. O. Adebiyi, and D. A. Obaromi, "Application of Single Step Euler and Trapezoid Methods of Solving Transient Distribution in Markov Chain," Nigerian Journal of Pure and Applied Sciences, vol. 36, no. 02, pp. 4669–4678, 2023.
I. S. Qamber and A. Z. Keller, "A developed Transient-behaviour Method," in 11th Advances in Reliability Technology Symposium, P. Comer, Ed. Dordrecht: Springer Netherlands, 1990, pp. 75–85.
E. de Souza e Silva and H. R. Gail, "Transient Solutions for Markov Chains," in Computational Probability, W. K. Grassmann, Ed. Boston, MA: Springer US, 2000, pp. 43–79.
S. V. Dhople and A. D. Dominguez-Garcia, "A Parametric Uncertainty Analysis Method for Markov Reliability and Reward Models," IEEE Transactions on Reliability, vol. 61, no. 3, pp. 634–648, Sep. 2012.
A. Reibman and K. Trivedi, "Numerical transient analysis of markov models," Computers & Operations Research, vol. 15, no. 1, pp. 19–36, Jan. 1988.
A. P. A. van Moorsel and K. Wolter, "Numerical Solution of Non-Homogeneous Markov Processes through Uniformization," in 12th European Simulation Multiconference - Simulation - Past, Present and Future, Machester, United Kingdom, Jun. 1998, pp. 710–717.
J. D. Lambert, Numerical Methods for Ordinary Differential Systems: The Initial Value Problem, 1st edition. Chichester: Wiley, 1991.
I. S. M. Zawawi, Z. B. Ibrahim, and K. I. Othman, "Derivation of Diagonally Implicit Block Backward Differentiation Formulas for Solving Stiff Initial Value Problems," Mathematical Problems in Engineering - Wiley Online Library, Apr. 2015.
S. Yimer, A. Shiferaw, and S. Gebregiorgis, "Block Procedure for Solving Stiff First Order Initial Value Problems Using Chebyshev Polynomials," Ethiopian Journal of Education and Sciences, vol. 15, no. 2, pp. 33–44, Nov. 2020.
A. M. Badmus and K. A. Felix, "Some Implicit Hybrid Block Linear Multi-Step Methods for Solution of First Order Ordinary Differential Equations," Nigerian Defence Academy Journal of Military Science and Interdisciplinary Studies, vol. 1, no. 1, pp. 56–63, Aug. 2022.
O. C. Benjamin and A. B. Okisamen, "On The Derivation and Implementation of a Fifth-Stage Fourth- Order Runge-Kutta Formula for Solving Initial Value Problems in Ordinary Differential Equations," IOSR Journal of Mathematics, vol. 16, no. 04, pp. 29–39, Aug. 2020.
M. E. Ehiemua, G. U. Agbeboh, and J. O. Ataman, "On the Derivation and Implementation of a Four-Stage Harmonic Runge-Kutta Scheme for Solving Initial Value Problems in Ordinary Differential Equations," Abacus (Mathematics Science Series), vol. 47, no. 01, pp. 311–318, Dec. 2020.
Downloads
How to Cite
License
Copyright (c) 2025 Zeina Mueen
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
