This paper examines the optimization of production involving a tandem two-machine system producing a single part type, with each machine being subject to random breakdowns and repairs. An analytical model is formulated with a view to solving an optimal stochastic production problem of the system with machines having up-downtime non-exponential distributions. The model developed is obtained by using a dynamic programming approach and a semi-Markov process. The control problem aims to find the production rates needed by the machines to meet the demand rate, through a minimization of the inventory/shortage cost. Using the Bellman principle, the optimality conditions obtained satisfy the Hamilton-Jacobi-Bellman equation, which depends on time and system states, and ultimately, leads to a feedback control. Consequently, the new model enables us to improve the coefficient of variation (CVup/down) to be less than one while it is equal to one in Markov model. Heuristics methods are used to involve the problem because of the difficulty of the analytical model using several states, and to show what control law should be used in each system state (i.e., including Kanban, feedback and CONWIP control). Numerical methods are used to solve the optimality conditions and to show how a machine should produce.