A common approach for detecting rare variants (RVs) associated with complex human diseases is to perform a gene-based or a region-based test of association. However, including all the RVs within a gene-based test might reduce its power since most RVs are not associated with the outcome of interest. As a shift to this paradigm, we propose to add a variable selection step to choose the RVs that compose the gene-based test statistic as a way to enhance the power of the test. We propose a Bayes Factor (BF) test statistic derived from the generalized linear model and its conjugate prior where functional annotation at the RV level can easily be integrated and the extent in prior belief can also be accommodated. A key component of our approach is the selection of the important RVs within a gene, which is performed through a novel scalable birth-death MCMC algorithm. Through simulation studies, we show that the proposed BF outperformed competing approaches both in terms of gene ranking and power to detect gene-based associations. The power of BF was improved by the use of functional annotation but interestingly, even when no annotation was included, substantial power gain was obtained from the variable selection procedure. Our application to a large whole-exome sequencing data set comparing 1,658 individuals with lung cancer to 1,492 healthy controls was able to identify new genes associated with lung cancer and pointed towards interesting cancer-related pathways.