How to Dynamically Schedule Multiple Knowledge-Intensive Projects Under Uncertainty: An Approximate Dynamic Programming Approach

Knowledge-intensive projects in research and development (R&D), software, and high-technology sectors are delivered by knowledge workers who each command several skills. When many such projects compete for a shared pool of multi-skilled workers, and neither project arrival times nor durations are known in advance, deciding who works on what at each moment becomes a sequential decision problem. Therefore, we propose the multi-skilled, multi-project dynamic scheduling problem with random project arrivals and uncertain durations and formulate it as a Markov decision process (MDP) that minimizes the total human resource cost. Since the cost-to-go function is computationally intractable, we develop a rollout-based approximate dynamic programming (ADP) algorithm that approximates it via Monte Carlo simulation embedded with a randomized base policy and restricts the action space to ten representative allocation policies. On benchmarks extended from the Project Scheduling Problem Library (PSPLIB), the proposed policy lowers the average total cost by 3.1% to 16.7% relative to Q-learning on medium- and large-scale instances while completing more projects with shorter delays; on small-scale instances Q-learning attains a lower nominal cost, revealing a trade-off among cost, completion rate, and delay.