At the edge of the network, the exponential increase of data produced along with the growing concerns over data privacy coming from regulations and society have all together triggered the rise of Federated Learning (FL) as the main approach in distributed machine learning (DML). Fed learning allows the model training to be performed on decentralized devices or data silos even without the raw data being transferred. Hence, FL is completely in line with the objectives of the privacy-preserving techniques. In this paper, we carry out a scientometric analysis on the 200 most cited papers, which are the first 200 papers at the intersection of "Distributed Machine Learning," "Federated Learning," and "Privacy-Preserving" published between 2020 and 2024, and the Scopus database is where they are indexed. The literature of publication trends, prominent authors and works, the thematic clusters, and research fronts that are changing are all systematically examined in this study; hence, the intellectual landscape of this fast developing field is mapped out. Our findings point to the existence of certain streams of research such as the algorithms with differential privacy being the mainstay, secure aggregation methods through the use of homomorphic encryption and multi-party computation, blockchain-based FL systems which ensure security and trust, and resource-efficient FL that supports IoT and edge computing. The results also show an area that is nearly enjoying a complete transformation as a result of the overpowering need to address the triad of model quality, data protection, and system efficiency. The review not only encourages researchers, and practitioners but also helps the policymakers by providing the current trend to which the key challenges can be identified and the future directions in privacy-preserving distributed intelligence anticipated.