Objectives
There are many types of mechanisms for data delivery. An important consideration of how to deal with data delivery is to consider the type of data and how it affects delivery requirements. One of the more interesting types of data to study is audio (and video) data. Of course, the type of application also has an impact. A p2p client downloading a song file is still just downloading data. But applications that do Voice-over-IP or streaming media are much more interesting. In these applications, the multimedia data does not require 100% reliable delivery; some data can be lost and the audio will still be intelligible and/or the video will still be watchable.
The goal of this assignment is to study the impact of network loss on multimedia, but using audio as a simplified case study. There are also ties between this homework assignment and the performance evaluation theme we have been discussing.
Somewhat surprisingly, this assignment does not require data to actually be transmitted over a network. It is more of a simulation. It still requires network-based coding, just not socket programming.
Assignment Background
First, "real-time" traffic means traffic is needed by a certain deadline. There are two types of real-time: loose and strict. "Loose" means that the traffic needs to be there as quickly as possible (e.g., game traffic). "Strict" means if it is not there by a specific deadline, bad things happen (e.g., data that indicates an operator has initiated a breaking procedure on, say, a train). While there is quite a bit of loose real-time traffic in the Internet (but not so much strict real-time traffic), TCP is still often used for this data even though TCP can delay packets due to in-order delivery and reliability requirements.
Of the services offered by TCP (connections, in-order delivery, reliability, ports), what is particular unnecessary is that for many types of real-time traffic, reliability is not needed. Audio data is one example of data where delay is more important than reliability. (We will talk more in class about why real-time voice applications use TCP instead of UDP.) Worse, because TCP delivers data in order, it can delay delivery of data to the application layer when it might have been better to deliver the data out of order and let the application handle the ordering.
The simplified figure below shows a typical architecture over which audio packets might be transmitted. Handling real-time audio traffic in such an environment is non-trivial for two main reasons. First, the data being transmitted is assumed to be real-time; therefore, minimizing delay is crucial. Second, the network can introduce a certain amount of jitter (i.e., variations in delivery time) that must be compensated for at the receiver.
If a packet is delayed long enough, it does not reach the receiver by the time it needs to be played out. For example, assume you are listening to a song being streamed over the Internet. If jitter is so bad that data is delayed by an hour (yes, an exaggeration, but it makes the point), when it does arrive at the receiver, the data will be useless. The receiver will have
had to skip over the missing data and move on. In essence, even though the data eventually arrives, it will have arrived too late, and to the client, it will be like it was never received at all.
For streaming audio, and for this assignment, you are to assume that if jitter causes a packet to arrive after its playback time, then the packet is lost. Since it is almost certain at least some audio data will be lost, the receiver must have some policy for handling playout when some packets arrive after their playback time. You will experiment with some of these different policies.
Packets can also be lost because they arrive at the receiver too quickly. If the receiver's buffer fills up, packets will be dropped because there is no place to store them. The simplest ways of solving this problems are for the sender to transmit data to the receiver only when it is needed or to receive feedback from the receiver that its buffer is full. For your assignment, you can assume an infinite buffer and will not have to deal with this problem. (If you are doing a streaming-based project, you will have to deal with this problem.
Assignment Details
As mentioned above, in this assignment, you will not have to actually transmit packets over a network. Instead, you will simulate their delivery over a network. The "simulation" needed for this assignment is quite simple. You only have to determine whether a packet is lost or if a packet of data is delayed enough that it misses its playback time. Ultimately, it is really about whether the data is received before it needs to be played or not.
One way to simulate the network is to create one procedure to transmit packets, and one procedure to receive packets. You can pass the packets between the transmitting procedure and the receiving procedure any way you want. One way might be to pass a C pointer to a data structure.
Since the focus of this assignment is to study the effects of loss on audio data, the code you write should be straightforward. Again, since you will not actually be sending data over the network, it really is a process of opening a file containing audio data, reading a chunk of data from the file, determining if the data was lost during transmission over the simulated network, and then writing the data was read from the file (if the data is determined to have been successfully delivered) or write something else (e.g., silence if that's the policy--see below for detail on the alternatives).
The main procedure in your code should be simple. It only has to open the devices, and then start a loop to send and receive packets until there is no more data in the source file. The main procedure in your code might look like the following:
main()
{
open_audio_source_file(); open_audio_dest_file(); while not EOF {
read_sample(); /* Read audio sample from a file */
loss = calculate_loss_probability(); /* Packet loss or jitter probabiliy */ if (loss < threshold) {
store_sample(); /* Store the received sample */
}
else
store_alternate_data(); /* See details for alternatives */
}
}
The idea of your simulation is to take as input a file (theoretically) to be streamed over a network from a server to a client and to create an output file that represents the simulated result of what was received at the client. You will not actually play the output as the simulation executes; instead, you will store the resulting audio packets into a "receive file." Once the output file is created, you can listen to it using any standard audio player; analyze the results; and write your report for this assignment.
Remember, because of the way you create the output file (either store the packet that is read if was determined to have been successfully received or store an alternative), it will be the same size and duration as the original file, but some (or much) of the file will be changed to reflect lost or late data.
Audio Format
There are many audio encoding formats available. To make this assignment less complex, you will use "mu-law" encoding, a type of audio encoding that can be configured not to use compression. Samples are taken 8000 times a second (8 kHz) and each sample creates a value 0-255 (8 bits). Therefore, the rate at which data is generated by the encoder and consumed by the decoder is 64 kbps (or 8 kBps). The fact that this encoding has a constant bit rate will make this assignment a whole lot easier. It will be most straightforward to read bytes from the file as unsigned integers and then changing the values as necessary.
You can google for .au files to download. I've also encoded some songs and attached them.
Experiments
The real goal of this assignment is to experiment with several of the parameters that affect perceived audio quality. You will look specifically at the effect of packet size (ranging from small to medium to large packets), loss probability (ranging from very low to medium to high packet loss), and what to do if data is lost or late. Furthermore, you should test the effects of changing these parameters on different types of audio sources (i.e., voice versus
music). The parameters you should experiment with include (but are not limited to) the following:
Test the Effect of Packet Size. Each audio packet will contain some number of sample values (i.e., data). This data can then be grouped into logical "packets" for (simulated) transmission. A packet can be small with only a few bytes of data or large with 1500 bytes or more (in your simulation you can assume no fragmentation). For different packet loss values (see below), what impact does the size of a packet have on the perceived quality of the audio?
Test the Effect of Packet Loss. What happens as the network starts to lose packets? How is audio quality affected? Describe what happens when the percentage of lost packets increases from zero percent to greater than fifty percent. When does the loss become noticeable? When does the audio become unintelligible? How does the size of the packet affect the sound quality during lossy playback? (NOTE: The measurement of audio quality can be quite subjective. As part of your report, describe how you are assessing audio )
Policy for Playing Unavailable Packets. When a packet is lost, or does not arrive in time to be played, the receiver must have a policy for deciding what to do. In the case of your simulator, if a packet is lost, you must still write some data to the file. Three policies include playing silence (writing all 0s), replaying the last sample (just the one sample value) for the duration of the missed packet, or playing the entire last packet received. For this experiment, implement all three solutions, and describe the difference in quality between them. How is the sound quality different for the three solutions as the loss rate and packet size varies? Describe in some detail other policies that can be used to reduce the effects of lost
Deliverables
For this project, you will turn in two files. The first is the source code for your simulator (called "-simulator.X" where "" is your last name and "X" is the normal extension for the programming language you are using). For this assignment, you can use use python 3. Note: turn in one and only one source code file. (I realize this will take a bit more work for some programming languages, but it makes grading easier.)
The second deliverable is a report describing what tests your performed (be sure to fully describe what the parameters were for each test), what results you obtained, an analysis of the results, and conclusions. Results should be shown in tables or graphs and any factors should be tested over a sufficiently wide range of values as to allow a comprehensive analysis. The report should be submitted in PDF and called "-report.pdf" where "" is your last name. Also make sure your name is in the body of the report.
Note: even though you are writing code, the more important deliverable is the report. So be thorough.
Assignment Turnin
Your turnin should be submitted as two files: "-simulator.X" and -report.pdf". The report should be PDF.
Extra Commentary:
While you are writing to the destination/output file, you need to make sure that the header of the .au file that you are reading from is copied before you start reading the rest of the packets. The header should not be dropped. This is necessary for being able to play the output audio file.
Report sections
Abstract
Intro/Background
Methodology -- Put your program logic/pseudo code, the libraries, programming language and other experimental details that you used.
Results -- A table with the Mean Opinion Scores for the different output files that you obtained along with the values of the parameters that you chose. You may also choose to put some remarks beside the scores about the audio
Discussion -- Put your assessment of the results in terms of the effect (how much and why) that the different parameters (packet size, loss rate, loss mitigation policy) had on the audio
Mention other different policies that may be used to reduce the effect of lost packets (existing techniques or you may suggest your own ideas)
Conclusion
Mean Opinion Score is a "human-judged" assessment of the overall quality of the audio. It's how you perceive how good the audio is. For a definite structure of the scoring criteria you may refer to the chart that I have attached below.
EXTRA INFO ABOUT PROJECT
8 bits (0 - 255) is enough to cover range of hearing
Read a packet (1st parameter is packet size, experiment with it)
Loss rate (for example 30% of packets) use rand gen
Loss mitigation techniques(fill in missing sections in audio due to loss)
Option 1: fill in with silence
Fill in with last sample of the last packet (packet has many samples) for the duration of the empty section
Fill in with last entire packet for the duration of the empty section
Write to output file
In real life they usually don’t repeat the words, most play an audio to indicate the call is still live for example
Score you determine based on
Understandable (clarity of speech, etc)
Enjoyable (music context but subjective)
No breaks in audio
No glitches
etc…
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
