Non-orthogonal multiple access (NOMA) is one of the promising radio access techniques for resource allocation improvement in the (5th) generation of cellular networks. Compared to orthogonal multiple access techniques, NOMA offers extra benefits, including greater spectrum efficiency which is provided through multiplexing users in the transmission power domain while using the same spectrum resources non-orthogonally. Even though NOMA uses Successive Interference Cancellation to repeal the interference among users, user grouping has shown to have a substantial impact on its performance. This performance improvement can appear in different parameters such as system capacity, data rate, or power consumption. In this paper, we propose a novel user grouping scheme for sum-rate maximization which increases the sum rate by approximately 12–25% in comparison with random user grouping and two other authenticated recent works. In addition to being matrix-based and having a polynomial time complexity, the proposed method is also able to cope with users experiencing different channel gains and powers in different sub-bands. Moreover, the proposed scheme is scalable and can be used for any number of users and sub-bands.