Computer vision approximates human vision using computers. Two subsets are explored in this work: image segmentation and visual tracking. Segmentation involves partitioning an image into logical parts, and tracking analyzes objects as they change over time.The presented research explores a key hypothesis: localizing analysis of visual information can improve the accuracy of segmentation and tracking results. Accordingly, a new class of segmentation techniques based on localized analysis is developed and explored.Next, these techniques are applied to two challenging problems: neuron bundle segmentation in diffusion tensor imagery (DTI) and plaque detection in computed tomography angiography (CTA) imagery.Experiments demonstrate that local analysis is well suited for these medical imaging tasks.Finally, a visual tracking algorithm is shown that uses temporal localization to track objects that change drastically over time.