Recent US government initiatives have made available a large number of Electronic Health Records (EHRs). These EHRs contain valuable information which can be used in Clinical Decision Support (CDS). So, Information Extraction (IE) from EHRs is a very promising research area. In this thesis, I focus on two tasks namely Mention Detection and Coreference Resolution. A lot of domain knowledge is available regarding clinical narratives. There are also several tools like SpecialistLexicalTools, MetaMap, etc. which help in analyzing clinical narratives. I integrate the domain knowledge and features derived from these tools in the local statistical models. Clinical narratives have a very special format which gives several interconnections between the tasks of mention detection and coreference resolution. A joint formulation for these two tasks has been presented in this thesis. Along with this, there is also a discussion regarding joint formulation for finding the mention types together. Soft constraints have been used while formulating the inference tasks. Softening the constraints is helpful because it allows the constraints to be violated during inference. Joint formulation is based on the fact that only local models are learned in the training phase. Inconsistencies in the decisions based on local models are resolved during the global inference step. I report the best results, to date, on end-to-end coreference resolution. The joint formulation presented in this thesis is very general and would benefit other information extraction tasks as well. I have made the systems described in this thesis publicly available for research use.