In this book, dynamical systems subject to some random perturbations which can be controlled are studied in order to optimize some performance criteria. A stochastic control problem in both finite and infinite time interval with terminal state constraint is formulated. The formulated stochastic control problem is solved using the dynamic programming principle for continuous Markov processes and also the maximum principle using the Hamilton-Jacobi-Bellman (HJB) equations.