In the complex and dynamic stock market landscape, investors seek to optimize returns while minimizing risks associated with price volatility. Various innovative approaches have been proposed to achieve high profits by considering historical trends and social factors. Despite advancements, accurately predicting market dynamics remains a persistent challenge. This study introduces a novel deep reinforcement learning (DRL) architecture to forecast stock market returns effectively. Unlike traditional approaches requiring manual feature engineering, the proposed model leverages convolutional neural networks (CNNs) to directly process daily stock prices and financial indicators. The model addresses overfitting and data scarcity issues during training by replacing conventional Q-tables with convolutional layers. The optimization process minimizes the sum of squared errors, enhancing prediction accuracy. Experimental evaluations demonstrate the model's robustness, achieving a 67% improvement in directional accuracy over the buy-and-hold strategy across short-term and long-term horizons. These findings underscore the model’s adaptability and effectiveness in navigating complex market environments, offering a significant advancement in financial forecasting.