Predictive methods for surgery duration

From HandWiki

Predictions of surgery duration (SD) are used to schedule planned/elective surgeries so that utilization rate of operating theatres be optimized (maximized subject to policy constraints). An example for a constraint is that a pre-specified tolerance for the percentage of postponed surgeries (due to non-available operating room (OR) or recovery room space) not be exceeded. The tight linkage between SD prediction and surgery scheduling is the reason that most often scientific research related to scheduling methods addresses also SD predictive methods and vice versa. Durations of surgeries are known to have large variability. Therefore, SD predictive methods attempt, on the one hand, to reduce variability (via stratification and covariates, as detailed later), and on the other employ best available methods to produce SD predictions. The more accurate the predictions, the better the scheduling of surgeries (in terms of the required OR utilization optimization). An SD predictive method would ideally deliver a predicted SD statistical distribution (specifying the distribution and estimating its parameters). Once SD distribution is completely specified, various desired types of information could be extracted thereof, for example, the most probable duration (mode), or the probability that SD does not exceed a certain threshold value. In less ambitious circumstance, the predictive method would at least predict some of the basic properties of the distribution, like location and scale parameters (mean, median, mode, standard deviation or coefficient of variation, CV). Certain desired percentiles of the distribution may also be the objective of estimation and prediction. Experts estimates, empirical histograms of the distribution (based on historical computer records), data mining and knowledge discovery techniques often replace the ideal objective of fully specifying SD theoretical distribution.

Reducing SD variability prior to prediction (as alluded to earlier) is commonly regarded as part and parcel of SD predictive method. Most probably, SD has, in addition to random variation, also a systematic component, namely, SD distribution may be affected by various related factors (like medical specialty, patient condition or age, professional experience and size of medical team, number of surgeries a surgeon has to perform in a shift, type of anesthetic administered). Accounting for these factors (via stratification or covariates) would diminish SD variability and enhance the accuracy of the predictive method. Incorporating expert estimates (like those of surgeons) in the predictive model may also contribute to diminish the uncertainty of data-based SD prediction. Often, statistically significant covariates (also related to as factors, predictors or explanatory variables) — are first identified (for example, via simple techniques like linear regression and knowledge discovery), and only later more advanced big-data techniques are employed, like Artificial Intelligence and Machine Learning, to produce the final prediction.

Literature reviews of studies addressing surgeries scheduling most often also address related SD predictive methods. Here are some examples (latest first).[1][2][3][4]

The rest of this entry review various perspectives associated with the process of producing SD predictions — SD statistical distributions, Methods to reduce SD variability (stratification and covariates), Predictive models and methods, and Surgery as a work-process. The latter addresses surgery characterization as a work-process (repetitive, semi-repetitive or memoryless) and its effect on SD distributional shape.

SD Statistical Distributions

Theoretical models

A most straightforward SD predictive method comprises specifying a set of existent statistical distributions, and based on available data and distribution-fitting criteria select the most fitting distribution. There is a large volume of comparative studies that attempt to select the most fitting models for SD distribution. Distributions most frequently addressed are the normal, the three-parameter lognormal, gamma (including the exponential) and Weibull. Less frequent "trial" distributions (for fitting purposes) are the loglogistic model, Burr, generalized gamma and the piecewise-constant hazard model. Attempts to presenting SD distribution as a mixture-distribution have also been reported (normal-normal, lognormal-lognormal and Weibull–Gamma mixtures). Occasionally, predictive methods are developed that are valid for a general SD distribution, or more advanced techniques, like Kernel Density Estimation (KDE), are used instead of the traditional methods (like distribution-fitting or regression-oriented methods). There is broad consensus that the three-parameter lognormal describes best most SD distributions. A new family of SD distributions, which includes the normal, lognormal and exponential as exact special cases, has recently been developed. Here are some examples (latest first).[5][6][7][8][9][10][11][12][13][14]

Using historical records to specify an empirical distribution

As an alternative to specifying a theoretical distribution as model for SD, one may use records to construct a histogram of available data, and use the related empirical distribution function (the cumulative plot) to estimate various required percentiles (like the median or the third quartile). Historical records/expert estimates may also be used to specify location and scale parameters, without specifying a model for SD distribution.

Data mining methods

These methods have recently gained traction as an alternative to specifying in-advance a theoretical model to describe SD distribution for all types of surgeries. Examples are detailed below ("Predictive models and methods").

Reducing SD variability (stratification and covariates)

To enhance SD prediction accuracy, two major approaches are pursued to reduce SD data variability: Stratification and covariates (incorporated in the predictive model). Covariates are often referred to in the literature also as factors, effects, explanatory variables or predictors.

Stratification

The term means that available data are divided (stratified) into subgroups, according to a criterion statistically shown to affect SD distribution. The predictive method then aims to produce SD prediction for specified subgroups, having SD with appreciably reduced variability. Examples for stratification criteria are medical specialty, Procedure Code systems, patient-severity condition or hospital/surgeon/technology (with resulting models related to as hospital-specific, surgeon-specific or technology-specific). Examples for implementation are Current Procedural Terminology (CPT) and ICD-9-CM Diagnosis and Procedure Codes (International Classification of Diseases, 9th Revision, Clinical Modification).[5][15][16][17]

Covariates (factors, effects, explanatory variables, predictors)

This approach to reduce variability incorporates covariates in the prediction model. The same predictive method may then be more generally applied, with covariates assuming different values for different levels of the factors shown to affect SD distribution (usually by affecting a location parameter, like the mean, and, more rarely, also a scale parameter, like the variance). A most basic method to incorporate covariates into a predictive method is to assume that SD distribution is lognormally distributed. The logged data (taking log of SD data) then represent a normally distributed population, allowing use of multiple- linear-regression to detect statistically significant factors. Other regression methods, which do not require data normality or are robust to its violation (generalized linear models, nonlinear regression) and artificial intelligence methods have also been used (references sorted chronologically, latest first).[14][18][19][20][21][22][23][24][25][26][27][28][29][30]

Predictive models and methods

Following is a representative (non-exhaustive) list of models and methods employed to produce SD predictions (in no particular order). These, or a mixture thereof, may be found in the sample of representative references below:

Linear regression (LR); Multivariate adaptive regression splines (MARS); Random forests (RF); Machine learning; Data mining (rough sets, neural networks); Knowledge discovery in databases (KDD); Data warehouse model (used to extract data from various, possibly non-interacting, databases); Kernel density estimation (KDE); Jackknife; Monte Carlo simulation.[31][32][33][34][35][2][36][37][38][39][40]

Surgery as work-process (repetitive, semi-repetitive, memoryless)

Surgery is a work process, and likewise it requires inputs to achieve the desired output, a recuperating post-surgery patient. Examples of work-process inputs, from Production Engineering, are the five M's — "money, manpower, materials, machinery, methods" (where "manpower" refers to the human element in general). Like all work-processes in industry and the services, surgeries also have a certain characteristic work-content, which may be unstable to various degrees (within the defined statistical population to which the prediction method aims). This generates a source for SD variability that affects SD distributional shape (from the normal distribution, for purely repetitive processes, to the exponential, for purely memoryless processes). Ignoring this source may confound its variability with that due to covariates (as detailed earlier). Therefore, as all work-processes may be partitioned into three types (repetitive, semi-repetitive, memoryless), surgeries may be similarly partitioned. A stochastic model that takes account of work-content instability has recently been developed, which delivers a family of distributions, with the normal/lognormal and exponential as exact special cases. This model was applied to construct a statistical process control scheme for SD.[5][41][42]

References

  1. Rahimi, Iman; Gandomi, Amir H. (2020-05-11). "A Comprehensive Review and Analysis of Operating Room and Surgery Scheduling" (in en). Archives of Computational Methods in Engineering 28 (3): 1667–1688. doi:10.1007/s11831-020-09432-2. ISSN 1886-1784. https://doi.org/10.1007/s11831-020-09432-2. 
  2. 2.0 2.1 Bellini, Valentina; Guzzon, Marco; Bigliardi, Barbara; Mordonini, Monica; Filippelli, Serena; Bignami, Elena (January 2020). "Artificial Intelligence: A New Tool in Operating Room Management. Role of Machine Learning Models in Operating Room Optimization" (in en). Journal of Medical Systems 44 (1): 20. doi:10.1007/s10916-019-1512-1. ISSN 0148-5598. PMID 31823034. http://link.springer.com/10.1007/s10916-019-1512-1. 
  3. Zhu, Shuwan; Fan, Wenjuan; Yang, Shanlin; Pei, Jun; Pardalos, Panos M. (2019-04-01). "Operating room planning and surgical case scheduling: a review of literature" (in en). Journal of Combinatorial Optimization 37 (3): 757–805. doi:10.1007/s10878-018-0322-6. ISSN 1573-2886. https://doi.org/10.1007/s10878-018-0322-6. 
  4. Cardoen, Brecht; Demeulemeester, Erik; Beliën, Jeroen (March 2010). "Operating room planning and scheduling: A literature review" (in en). European Journal of Operational Research 201 (3): 921–932. doi:10.1016/j.ejor.2009.04.011. https://linkinghub.elsevier.com/retrieve/pii/S0377221709002616. 
  5. 5.0 5.1 5.2 Shore, Haim (2020-04-02). "An explanatory bi-variate model for surgery-duration and its empirical validation" (in en). Communications in Statistics 6 (2): 142–166. doi:10.1080/23737484.2020.1740066. ISSN 2373-7484. https://www.tandfonline.com/doi/full/10.1080/23737484.2020.1740066. 
  6. Taaffe, Kevin; Pearce, Bryan; Ritchie, Gilbert (2021). "Using kernel density estimation to model surgical procedure duration" (in en). International Transactions in Operational Research 28 (1): 401–418. doi:10.1111/itor.12561. ISSN 1475-3995. 
  7. Joustra, Paul; Meester, Reinier; van Ophem, Hans (June 2013). "Can statisticians beat surgeons at the planning of operations?" (in en). Empirical Economics 44 (3): 1697–1718. doi:10.1007/s00181-012-0594-0. ISSN 0377-7332. http://link.springer.com/10.1007/s00181-012-0594-0. 
  8. Choi, Sangdo; Wilhelm, Wilbert E. (2012-04-01). "An analysis of sequencing surgeries with durations that follow the lognormal, gamma, or normal distribution". IIE Transactions on Healthcare Systems Engineering 2 (2): 156–171. doi:10.1080/19488300.2012.684272. ISSN 1948-8300. https://doi.org/10.1080/19488300.2012.684272. 
  9. Chakraborty, Santanu; Muthuraman, Kumar; Lawley, Mark (2010-02-26). "Sequential clinical scheduling with patient no-shows and general service time distributions" (in en). IIE Transactions 42 (5): 354–366. doi:10.1080/07408170903396459. ISSN 0740-817X. http://www.tandfonline.com/doi/abs/10.1080/07408170903396459. 
  10. Eijkemans, Marinus J. C.; van Houdenhoven, Mark; Nguyen, Tien; Boersma, Eric; Steyerberg, Ewout W.; Kazemier, Geert (2010-01-01). "Predicting the Unpredictable: A New Prediction Model for Operating Room Times Using Individual Characteristics and the Surgeon's Estimate". Anesthesiology 112 (1): 41–49. doi:10.1097/ALN.0b013e3181c294c2. ISSN 0003-3022. PMID 19952726. 
  11. Spangler, William E.; Strum, David P.; Vargas, Luis G.; May, Jerrold H. (2004-05-01). "Estimating Procedure Times for Surgeries by Determining Location Parameters for the Lognormal Model" (in en). Health Care Management Science 7 (2): 97–104. doi:10.1023/B:HCMS.0000020649.78458.98. ISSN 1572-9389. PMID 15152974. https://doi.org/10.1023/B:HCMS.0000020649.78458.98. 
  12. Strum, David P.; May, Jerrold H.; Sampson, Allan R.; Vargas, Luis G.; Spangler, William E. (2003-01-01). "Estimating Times of Surgeries with Two Component Procedures: Comparison of the Lognormal and Normal Models". Anesthesiology 98 (1): 232–240. doi:10.1097/00000542-200301000-00035. ISSN 0003-3022. PMID 12503002. https://doi.org/10.1097/00000542-200301000-00035. 
  13. May, Jerrold H.; Strum, David P.; Vargas, Luis G. (2000). "Fitting the Lognormal Distribution to Surgical Procedure Times*" (in en). Decision Sciences 31 (1): 129–148. doi:10.1111/j.1540-5915.2000.tb00927.x. ISSN 1540-5915. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1540-5915.2000.tb00927.x. 
  14. 14.0 14.1 Strum, David P.; May, Jerrold H.; Vargas, Luis G. (2000-04-01). "Modeling the Uncertainty of Surgical Procedure Times" (in en). Anesthesiology 92 (4): 1160–1167. doi:10.1097/00000542-200004000-00035. ISSN 0003-3022. PMID 10754637. https://pubs.asahq.org/anesthesiology/article/92/4/1160/413/Modeling-the-Uncertainty-of-Surgical-Procedure. 
  15. Dexter, Franklin; Dexter, Elisabeth U.; Ledolter, Johannes (April 2010). "Influence of Procedure Classification on Process Variability and Parameter Uncertainty of Surgical Case Durations" (in en). Anesthesia & Analgesia 110 (4): 1155–1163. doi:10.1213/ANE.0b013e3181d3e79d. ISSN 0003-2999. PMID 20357155. https://dx.doi.org/10.1213%2FANE.0b013e3181d3e79d. 
  16. Li, Ying; Zhang, Saijuan; Baugh, Reginald F.; Huang, Jianhua Z. (2009-11-30). "Predicting surgical case durations using ill-conditioned CPT code matrix". IIE Transactions 42 (2): 121–135. doi:10.1080/07408170903019168. ISSN 0740-817X. https://doi.org/10.1080/07408170903019168. 
  17. Stepaniak, Pieter S.; Heij, Christiaan; Mannaerts, Guido H. H.; de Quelerij, Marcel; de Vries, Guus (October 2009). "Modeling Procedure and Surgical Times for Current Procedural Terminology-Anesthesia-Surgeon Combinations and Evaluation in Terms of Case-Duration Prediction and Operating Room Efficiency: A Multicenter Study" (in en-US). Anesthesia & Analgesia 109 (4): 1232–1245. doi:10.1213/ANE.0b013e3181b5de07. ISSN 0003-2999. PMID 19762753. https://dx.doi.org/10.1213%2FANE.0b013e3181b5de07. 
  18. Wang, Jin; Cabrera, Javier; Tsui, Kwok-Leung; Guo, Hainan; Bakker, Monique; Kostis, John B. (2020-02-10). "Clinical and Nonclinical Effects on Operative Duration: Evidence from a Database on Thoracic Surgery" (in en). Journal of Healthcare Engineering 2020: 1–8. doi:10.1155/2020/3582796. PMID 32104558. 
  19. Parker, Sarah Henrickson; Lei, Xue; Fitzgibbons, Shimae; Metzger, Thomas; Safford, Shawn; Kaplan, Seth (November 2020). "The Impact of Surgical Team Familiarity on Length of Procedure and Length of Stay: Inconsistent Relationships Across Procedures, Team Members, and Sites" (in en). World Journal of Surgery 44 (11): 3658–3667. doi:10.1007/s00268-020-05657-1. ISSN 0364-2313. PMID 32661690. http://link.springer.com/10.1007/s00268-020-05657-1. 
  20. Powezka, Katarzyna; Normahani, Pasha; Standfield, Nigel J.; Jaffer, Usman (2020-03-01). "A novel team Familiarity Score for operating teams is a predictor of length of a procedure: A retrospective Bayesian analysis" (in English). Journal of Vascular Surgery 71 (3): 959–966. doi:10.1016/j.jvs.2019.03.085. ISSN 0741-5214. PMID 31401113. https://www.jvascsurg.org/article/S0741-5214(19)31325-4/abstract. 
  21. Wang, Jin; Cabrera, Javier; Tsui, Kwok-Leung; Guo, Hainan; Bakker, Monique; Kostis, John B. (2020-02-10). "Clinical and Nonclinical Effects on Operative Duration: Evidence from a Database on Thoracic Surgery" (in en). Journal of Healthcare Engineering 2020: 1–8. doi:10.1155/2020/3582796. ISSN 2040-2295. PMID 32104558. 
  22. Tan, K. W.; Nguyen, F. N. H. L.; Ang, B. Y.; Gan, J.; Lam, S. W. (August 2019). "Data-Driven Surgical Duration Prediction Model for Surgery Scheduling: A Case-Study for a Practice-Feasible Model in a Public Hospital". 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE). pp. 275–280. doi:10.1109/COASE.2019.8843299. ISBN 978-1-7281-0356-3. https://ieeexplore.ieee.org/document/8843299. 
  23. van Eijk, Ruben P. A.; van Veen-Berkx, Elizabeth; Kazemier, Geert; Eijkemans, Marinus J. C. (August 2016). "Effect of Individual Surgeons and Anesthesiologists on Operating Room Time" (in en). Anesthesia & Analgesia 123 (2): 445–451. doi:10.1213/ANE.0000000000001430. ISSN 0003-2999. PMID 27308953. https://dx.doi.org/10.1213%2FANE.0000000000001430. 
  24. Kayış, Enis; Khaniyev, Taghi T.; Suermondt, Jaap; Sylvester, Karl (2015-09-01). "A robust estimation model for surgery durations with temporal, operational, and surgery team effects" (in en). Health Care Management Science 18 (3): 222–233. doi:10.1007/s10729-014-9309-8. ISSN 1572-9389. PMID 25501470. https://doi.org/10.1007/s10729-014-9309-8. 
  25. Gillespie, Brigid M.; Chaboyer, Wendy; Fairweather, Nicole (2012-01-01). "Factors that influence the expected length of operation: results of a prospective study" (in en). BMJ Quality & Safety 21 (1): 3–12. doi:10.1136/bmjqs-2011-000169. ISSN 2044-5415. PMID 22003174. https://qualitysafety.bmj.com/content/21/1/3. 
  26. Kayis, Enis; Wang, Haiyan; Patel, Meghna; Gonzalez, Tere; Jain, Shelen; Ramamurthi, R. J.; Santos, Cipriano; Singhal, Sharad et al. (2012). "Improving prediction of surgery duration using operational and temporal factors". AMIA ... Annual Symposium Proceedings. AMIA Symposium 2012: 456–462. ISSN 1942-597X. PMID 23304316. 
  27. Palmer, Phillip B.; O'Connell, Dennis G. (September 2009). "Regression analysis for prediction: understanding the process". Cardiopulmonary Physical Therapy Journal 20 (3): 23–26. doi:10.1097/01823246-200920030-00004. ISSN 2374-8907. PMID 20467520. 
  28. Dexter, Franklin; Dexter, Elisabeth U.; Masursky, Danielle; Nussmeier, Nancy A. (April 2008). "Systematic Review of General Thoracic Surgery Articles to Identify Predictors of Operating Room Case Durations" (in en-US). Anesthesia & Analgesia 106 (4): 1232–1241. doi:10.1213/ane.0b013e318164f0d5. ISSN 0003-2999. PMID 18349199. https://dx.doi.org/10.1213%2Fane.0b013e318164f0d5. 
  29. Ballantyne, Garth H.; Ewing, Douglas; Capella, Rafael F.; Capella, Joseph F.; Davis, Dan; Schmidt, Hans J.; Wasielewski, Annette; Davies, Richard J. (2005-02-01). "The Learning Curve Measured by Operating Times for Laparoscopic and Open Gastric Bypass: Roles of Surgeon's Experience, Institutional Experience, Body Mass Index and Fellowship Training" (in en). Obesity Surgery 15 (2): 172–182. doi:10.1381/0960892053268507. ISSN 1708-0428. PMID 15810124. https://doi.org/10.1381/0960892053268507. 
  30. Strum, David P.; Sampson, Allan R.; May, Jerrold H.; Vargas, Luis G. (2000-05-01). "Surgeon and Type of Anesthesia Predict Variability in Surgical Procedure Times". Anesthesiology 92 (5): 1454–1466. doi:10.1097/00000542-200005000-00036. ISSN 0003-3022. PMID 10781292. https://doi.org/10.1097/00000542-200005000-00036. 
  31. Soh, K. W.; Walker, C.; O’Sullivan, M.; Wallace, J. (2020-01-02). "An Evaluation of the Hybrid Model for Predicting Surgery Duration" (in en). Journal of Medical Systems 44 (2): 42. doi:10.1007/s10916-019-1501-4. ISSN 1573-689X. PMID 31897758. https://doi.org/10.1007/s10916-019-1501-4. 
  32. Soh, K. W.; Walker, C.; O’Sullivan, M.; Wallace, J. (2020-10-01). "Comparison of Jackknife and Hybrid-Boost Model Averaging to Predict Surgery Durations: A Case Study" (in en). SN Computer Science 1 (6): 316. doi:10.1007/s42979-020-00339-0. ISSN 2661-8907. 
  33. Jiao, York; Sharma, Anshuman; Ben Abdallah, Arbi; Maddox, Thomas M.; Kannampallil, Thomas (2020-12-09). "Probabilistic forecasting of surgical case duration using machine learning: model development and validation". Journal of the American Medical Informatics Association 27 (12): 1885–1893. doi:10.1093/jamia/ocaa140. ISSN 1527-974X. PMID 33031543. 
  34. Rozario, Natasha; Rozario, Duncan (2020-11-12). "Can machine learning optimize the efficiency of the operating room in the era of COVID-19?". Canadian Journal of Surgery 63 (6): E527–E529. doi:10.1503/cjs.016520. PMID 33180692. PMC 7747850. http://canjsurg.ca/63-6-e527/. 
  35. Bartek, Matthew A.; Saxena, Rajeev C.; Solomon, Stuart; Fong, Christine T.; Behara, Lakshmana D.; Venigandla, Ravitheja; Velagapudi, Kalyani; Lang, John D. et al. (October 2019). "Improving Operating Room Efficiency: Machine Learning Approach to Predict Case-Time Duration" (in en). Journal of the American College of Surgeons 229 (4): 346–354.e3. doi:10.1016/j.jamcollsurg.2019.05.029. PMID 31310851. 
  36. Tuwatananurak, Justin P.; Zadeh, Shayan; Xu, Xinling; Vacanti, Joshua A.; Fulton, William R.; Ehrenfeld, Jesse M.; Urman, Richard D. (2019-01-17). "Machine Learning Can Improve Estimation of Surgical Case Duration: A Pilot Study" (in en). Journal of Medical Systems 43 (3): 44. doi:10.1007/s10916-019-1160-5. ISSN 1573-689X. PMID 30656433. https://doi.org/10.1007/s10916-019-1160-5. 
  37. Zhao, Beiqun; Waterman, Ruth S.; Urman, Richard D.; Gabriel, Rodney A. (2019-01-05). "A Machine Learning Approach to Predicting Case Duration for Robot-Assisted Surgery" (in en). Journal of Medical Systems 43 (2): 32. doi:10.1007/s10916-018-1151-y. ISSN 1573-689X. PMID 30612192. https://doi.org/10.1007/s10916-018-1151-y. 
  38. Hosseini, N.; Sir, M. Y.; Jankowski, C. J.; Pasupathy, K. S. (2015). "Surgical Duration Estimation via Data Mining and Predictive Modeling: A Case Study". AMIA ... Annual Symposium Proceedings. AMIA Symposium 2015: 640–648. ISSN 1942-597X. PMID 26958199. 
  39. ShahabiKargar, Zahra; Khanna, Sankalp; Good, Norm; Sattar, Abdul; Lind, James; O’Dwyer, John (2014). Pham, Duc-Nghia; Park, Seong-Bae. eds. "Predicting Procedure Duration to Improve Scheduling of Elective Surgery" (in en). PRICAI 2014: Trends in Artificial Intelligence. Lecture Notes in Computer Science (Cham: Springer International Publishing) 8862: 998–1009. doi:10.1007/978-3-319-13560-1_86. ISBN 978-3-319-13560-1. https://link.springer.com/chapter/10.1007%2F978-3-319-13560-1_86. 
  40. Combes, C.; Meskens, N.; Rivat, C.; Vandamme, J.-P. (2008-03-01). "Using a KDD process to forecast the duration of surgery" (in en). International Journal of Production Economics 112 (1): 279–293. doi:10.1016/j.ijpe.2006.12.068. ISSN 0925-5273. https://www.sciencedirect.com/science/article/abs/pii/S0925527307001466. 
  41. Shore, Haim (2020). "SPC scheme to monitor surgery duration" (in en). Quality and Reliability Engineering International 37 (4): 1561–1577. doi:10.1002/qre.2813. ISSN 1099-1638. https://onlinelibrary.wiley.com/doi/abs/10.1002/qre.2813. 
  42. Shore, Haim (2021-12-13). "Estimating operating room utilisation rate for differently distributed surgery times". International Journal of Production Research 61 (2): 447–461. doi:10.1080/00207543.2021.2009141. ISSN 0020-7543. https://doi.org/10.1080/00207543.2021.2009141.