Stochastic optimization algorithm provides a robust and efficient approach for solving complex real world problems. Backtracking Search Optimization Algorithm (BSA) is a new stochastic evolutionary algorithm and the aim of this paper is to introduce a hybrid approach combining the BSA and Quadratic approximation (QA), called HBSAfor solving unconstrained non-linear, non-differentiable optimization problems. For the validity of the proposed method the results are compared with five state-of-the-art particle swarm optimization (PSO) variant approaches in terms of the numerical result of the solutions. The sensitivity analysis of the BSA control parameter (F) is also performed.