Information-theoretic Results on Communication Problems with Feed-forwardand Feedback.

Information Theory;Feed-forward;Source Coding;Feedback;Electrical Engineering;Engineering;Electrical Engineering: Systems

Venkataramanan, RamjiTeneketzis, Demosthenis ;

University of Michigan

关键词: Information Theory;    Feed-forward;    Source Coding;    Feedback;    Electrical Engineering;    Engineering;    Electrical Engineering: Systems;   

Others  :  https://deepblue.lib.umich.edu/bitstream/handle/2027.42/61696/rvenkata_1.pdf?sequence=1&isAllowed=y

瑞士|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

As networked communication systems become increasingly sophisticated, understanding information flow in networks is a problem of central importance.Multi-user information theory attempts to understand various interactions in a networkby studyingsmall building blocks such as distributed compression, multiple access, broadcast etc. In this thesis, we investigate two problems in network communication from an information-theoretic perspective:feed-forward in sources and feedback in channels. The two problems are closely related in the sense that there is a dynamic aspect to either the decoder or the encoder in each of them.Feedback in channels has been extensively studied, but the problem of source coding with feed-forward is recent.In source coding with feed-forward, to reconstruct each source sample, the decoder has knowledge of some past source samples in addition to the codebook index. This extra information can presumably help the decoder produce a better reconstruction. We obtain the optimal rate-distortion function and characterize the error exponent for this problem.The relevant information quantity is directed information, which captures the causal flow of information from one random sequence to another.Directed information was recently used to characterize the capacity of channels with feedback. We provide an interpretation of directed information that helps understand why it arises in the context of feedback. This interpretation is used to obtain the feedback capacity of channels with side-information available with delay at both the encoder and decoder.It turns out that feed-forward/feedback does not improve the performance limit for memoryless sources/channels. Hence we consider general sources and channels with memory. Consequently, the rate-distortion function and capacityare multi-letter expressions that cannot be computed easily in general. We present an approach that can used to compute these expressions for a wide class of sources, channels and distortion measures.Finally, we investigate the effect of feed-forward in multiple descriptions coding.Weobtain a computable `single-letter;; achievable rate region. In contrast to the point-to-point case, even for i.i.d sources, the obtained rate-region is strictly better than the best known region without feed-forward.