Information theory, superprofs.com/blog/gate/information-theory/

Challenging the status quo

26 November 2015

Dr Raymond Wai-Ho Yeung (Croucher Senior Research Fellowship 2000) is Choh-Ming Li Professor of Information Engineering and Co-Director of the Institute of Network Coding at the Chinese University of Hong Kong. 

Yeung is a theoretician at heart. He is motivated by the task of coming up with, and solving, fundamental problems. This means that typically his first work on a certain subject has very few references and is simple to describe. In his own words, “If the question is too complicated, the problem is very specific and not fundamental.” However, Yeung does offer advice (to his students) on the perils of following a similar path:

“Don’t expect to be under the spotlight the whole time… It may be more than ten years before the importance of the research is recognised. If a piece of work takes ten years to be recognised, it may as well take 20 years, or it may just as well be forgotten forever.”

Yeung can say this from personal experience. Before the year 2000, despite being very much the same researcher as he is now, Yeung’s citation numbers were relatively low. In the year 2000, a turning point in his career took place. This turning point, marked by the publication of a paper which launched a new research field: network coding, was not based on any sudden discovery; it was a combination of things that came together, many of which cannot be controlled by the researcher. In fact, in 1995, Yeung and his fellow researchers had published a paper along the same lines, yet it did not receive much attention.

Yeung credits the validation he received in the year 2000, from the academic and scientific community, to the fact that in this particular paper, he and his colleagues had been able to present an explicit example showing the advantage of network coding over routing, a clean solution of the problem posed, and coined an appealing name: “network coding”.

What is network coding?

To understand what network coding is, it is important to gain an elementary understanding of how computer networks operate. Computer networks work similarly to a postal system: data is packed into packets, very much like the parcels you would use to send gifts; each parcel requires an address, it then goes to the nearest post office and is sorted - eventually ending up at its labeled destination. Comparably, data is put into packets and each packet has an address at which it should end up.

For Yeung, network coding began as a research topic in Information Theory. Information Theory was established by the publication of Claude Shannon’s classic paper “A Theory of Communication” in the Bell System Technical Journal in July and October 1948. It is concerned with the fundamentals of information transactions, that include: how much information can be reliably communicated through a channel; what the limit of data compressions is, etc.

In the past, the flow of data in network communication and cryptography had been thought of as a commodity flow - comparable to the flow of parcels through a postal system. The travel of lots of data packets was compared to a superhighway, with trucks travelling from point to point, stopping at various junctions.

In the early nineties, Yeung became suspicious of this fundamental idea; that the flow of data packets could be so closely compared to the travel of commodities. Through research, he discovered that information in a network does not behave like a commodity. To explain: when you post a parcel, you can expect that it will arrive at its destination having been unopened. However, information in a network does not behave this way. It is possible to program the network so that the ‘post office’ can open the parcel, reshuffle and move the information into different parcels.

To illustrate this, imagine two cars arriving at a junction− in commodity travel, one would be required to yield; in networks, the two cars (data packets) are able to merge because data is not a physical entity. This is the main difference between commodity travel and network communication− and is now possible to take advantage of this to send more information by employing coding within the network: hence, “network coding”.

Although network coding began as a research topic in Information Theory, it quickly became clear how important it would be to many different research fields - among them: computer networks, wireless communication, data storage, cryptography, and quantum computing. The field of Information Theory has been through many up and downs. Indeed, Yeung recalls a panel discussion among leading researchers, at its 50th anniversary in 1998, as to whether Information Theory is dying or already dead. However, almost twenty years have passed since this discouraging discussion, and research going into areas of Information Theory, such as network coding, is thriving.


Professor Raymond Wai-Ho Yeung received his BS (1984), MEng (1985) and PhD (1988) degrees in electrical engineering from Cornell University. Yeung worked AT&T Bell Laboratories (1988-1991), before joining the Chinese University of Hong Kong as a Lecturer at the Department of Information Engineering. In 2000, Yeung received a Croucher Senior Research Fellowship. As cofounder of network coding, he has been serving as co-director of the Institute of network coding since 2010. He is author of the books “A First Course in Information Theory” (Kluwer Academic/Plenum Publishers, 2002) and “Information Theory and Network Coding” (Springer 2008), both of which have been adopted by over 100 institutions around the world. In spring 2014, he gave the first ever “massive online open course on information theory”, reaching over 25,000 students.


To view Raymond Wai-Ho Yeung’s personal Croucher profile, please click here