We propose that rational down-selection criteria for fusion space propulsion should be based on the goals for NASA's future missions, and in particular, on performance goals. Specifically, if the ultimate long-range performance for a certain fusion concept for a particular mission cannot exceed that expected for an economically and environmentally viable fission-propulsion system, which is obviously based on a more mature technology than the fusion system, NASA should not spend the time and resources required to develop that fusion system. We also propose consideration of inherent physical constraints for each space-propulsion concept, because the physical constraints can limit a concept's ultimate performance. Such constraints can thus make a concept subject to down-selection even though there are currently large uncertainties in a particular system's ultimate performance, projected cost of development, or even 'proof-of-principle' status. One way to impose such goal-oriented criteria is to require all viable fusion concepts for a given mission to have an alpha (i.e., a ratio of dry mass to jet power) less than a maximum that corresponds to the performance of the fission systems. Specifically, using a Mars roundtrip as an example, we discuss how physical limitations in target gain and nozzle physics can preclude a concept achieving the required alpha. This goal-oriented approach for down-selection based on physical constraints can help NASA know up front where to wisely spend its R&D funds.