Streaming media technology foundation

1. The basis of streaming

At present, there are mainly two schemes for downloading and streaming multimedia information such as audio / video transmission on the network. A / V files are generally large, so the required storage capacity is also large; at the same time, due to network bandwidth limitations, downloading often takes several minutes or even hours, so this processing method is also delayed. When streaming, the time-based media such as sound, image or animation is continuously and real-time transmitted from the audio and video server to the user's computer. The user does not have to wait for the entire file to be downloaded, but only after a few seconds or tens of seconds of startup delay Available for viewing. When sound-based time-based media is played on the client, the rest of the file will continue to download from the server in the background. Streaming not only shortens the startup delay by a factor of ten or a hundred, but also does not require much cache capacity. Streaming avoids the disadvantage that users must wait for the entire file to be downloaded from the Internet before viewing.

Streaming media refers to continuous time-based media that uses streaming technology in the Internet / Intranet, such as audio, video, or multimedia files. Streaming media does not download the entire file before playing, only the beginning part is stored in the memory, the data stream of streaming media is transmitted at any time and played at any time, but there is some delay at the beginning. The key technology for streaming media is streaming.

Streaming is widely defined, and now mainly refers to the general term for the technology of transmitting media (such as video and audio) through the network. Its specific meaning is to transmit video programs to PC through the Internet. There are two methods to achieve streaming: Realtime streaming and progressive streaming. Generally speaking, if the video is a real-time broadcast, or a streaming media server is used, or a real-time protocol such as RTSP is applied, it is real-time streaming. If an HTTP server is used, the files are sent via sequential streaming. Which transmission method to use depends on your needs. Of course, streaming files also support full download to the hard disk before playing.

Sequential streaming

Sequential streaming is sequential downloading. While downloading files, users can watch the remedial media. At a given moment, users can only watch the part that has been downloaded, and cannot jump to the first part that has not been downloaded. Unlike real-time streaming, it adjusts to the speed of the user ’s connection during transmission. Since a standard HTTP server can send this form of file and does not require other special protocols, it is often referred to as HTTP streaming. Sequential streaming is more suitable for high-quality short clips, such as the opening, ending and commercials. Since the part of the file viewed before playback is losslessly downloaded, this method guarantees the final quality of the movie playback. This means that users must experience a delay before watching, especially for slower connections.

For streaming short clips through a modem, sequential streaming appears to be practical. It allows video clips to be created at a higher data rate than modems. Despite the delay, after all, it allows you to post higher-quality video clips.

Sequential streaming files are placed on a standard HTTP or FTP server, which is easy to manage and basically has nothing to do with firewalls. Sequential streaming is not suitable for long clips and videos with random access requirements, such as lectures, speeches and presentations. It also does not support live broadcasting. Strictly speaking, it is an on-demand technology.

Real-time streaming

Real-time streaming refers to ensuring that the bandwidth of the media signal matches the network connection so that the media can be viewed in real time. Real-time streaming is different from HTTP streaming, which requires a dedicated streaming server and transmission protocol.

Real-time streaming is always transmitted in real time, which is especially suitable for live events. It also supports random access. Users can fast forward or rewind to view the front or back content. Theoretically, the real-time stream does not need to stop as soon as it is played, but in practice, periodic pauses may occur.

Real-time streaming must match the connection bandwidth, which means that the image quality is poor when connected at modem speed. Moreover, because the information lost due to errors is ignored, the video quality is poor when the network is crowded or there is a problem. To ensure video quality, sequential streaming may be better. Real-time streaming requires specific servers, such as QuickTime Streaming Server, RealServer, and Windows Media Server. These servers allow you to have more control over media delivery, so system setup and management are more complicated than standard HTTP servers. Real-time streaming also requires special network protocols, such as RTSP (Realtime Streaming Protocol) or MMS (Microsoft Media Server). These protocols sometimes have problems when there is a firewall, which prevents users from seeing real-time content in some locations.

Second, the principle of streaming media technology

The implementation of streaming requires caching. Because the Internet performs intermittent asynchronous transmission based on packet transmission, for a real-time A / V source or stored A / V file, they will be decomposed into many packets during transmission. Since the network is dynamically changing, each packet is selected The routes may not be the same, so the time delay to reach the client will not be equal, and even the first data packet may arrive later. To this end, a buffer system is used to compensate for the effects of delay and jitter, and to ensure that the order of data packets is correct, so that media data can be continuously output without causing pauses in playback due to temporary network congestion. Usually the required capacity of the cache is not large, because the cache uses a circular linked list structure to store data: by discarding the content that has been played, the stream can reuse the free cache space to cache the content that has not been played later. Generally, the realization of streaming transmission requires a suitable transmission protocol. Because TCP requires more overhead, it is not suitable for transmitting real-time data. In the implementation scheme of streaming transmission, HTTP / TCP is generally used to transmit control information, and RTP / UDP is used to transmit real-time sound data.

The process of streaming transmission is generally this: after the user selects a certain streaming service, HTTP / TCP is used to exchange control information between the web browser and the web server, so as to retrieve the real-time data to be transmitted from the original information; then the client The Web browser on the machine starts the A / VHelper program and uses HTTP to retrieve the relevant parameters from the Web server to initialize the Helper program. These parameters may include directory information, A / V data encoding type, or server address related to A / V retrieval.

The A / VHelper program and the A / V server run real-time flow control protocol (RTSP) to exchange control information required for A / V transmission. Similar to the functions provided by CD players or VCRs, RTSP provides methods for manipulating playback, fast forward, fast reverse, pause, and record commands. The A / V server uses the RTP / UDP protocol to transmit A / V data to the A / V client program (the client program is generally considered to be equivalent to the Helper program). Once the A / V data reaches the client, the A / V client program can play Output.

It should be noted that in streaming, two different communication protocols RTP / UDP and RTSP / TCP are used to establish contact with the A / V server, in order to be able to redirect the output of the server to a different The destination address of the client where the program is located. The realization of streaming generally requires a dedicated server and player, the basic principle of which is shown in the figure.

3. Smart Streaming Technology (SureStream)

Today, 28.8Kbps modems are the basic rate for Internet connections. Cable modems, ADSL, DSS, ISDN, etc. are developing rapidly. Content providers have to either restrict the quality of published media or limit the number of connected people. According to the statistics of the RealNetwork site, for the 28.8Kbps modem, the actual traffic is 10bps to 26Kbps, which has a bell-shaped distribution, with a peak at 20Kbps. This means that if the content provider chooses a fixed rate of 20Kbps, a large number of users will not get a good quality signal, and may stop the media stream and cause the client to buffer again until it receives enough data.

One solution is that the server reduces the data sent to the client to prevent rebuffering. In RealSystem 5.0, this method is called "video stream thinning". The limitation of this method is that the RealVideo file is designed for a data rate, and the result can be expanded to a lower rate by extracting internal frames, resulting in lower quality. The further away from the original data rate, the worse the quality. Another solution is to create multiple files according to different connection rates, and the server sends the corresponding file according to the user connection. This method brings difficulties in production and management. Moreover, the user connection is dynamic and the server cannot coordinate in real time. . Intelligent streaming technology overcomes bandwidth coordination and stream thinning in two ways. First, establish a coding framework that allows multiple streams of different rates to be encoded at the same time and merge into the same file; second, use a complex client / server mechanism to detect bandwidth changes.

In response to differences in software, device, and data transmission speeds, users browse audio and video content with different bandwidths. In order to meet customer requirements, Progressive Networks encodes and records media data at different rates and saves them in a single file. This file is called an intelligent streaming file, that is, an extensible streaming file is created. When the client sends a request, it transmits its bandwidth capacity to the server, and the media server transmits the corresponding part of the smart stream file to the user according to the client's bandwidth. In this way, users can see the most likely high-quality transmission, the production staff only needs to compress it once, the administrator only needs to maintain a single file, and the media server automatically switches according to the resulting bandwidth. Smart Streaming delivers high-quality media and guarantees reliability by describing the changing bandwidth characteristics of the Internet in the real world, and provides a solution to content authorization in a mixed connection environment. The streaming media is implemented as follows:

Create a file for all connection rate environments
Transfer media at different rates in a mixed environment
According to network changes, seamlessly switch to other rates
Key frame priority, audio is more important than some frame data
Backwards compatible with older versions of RealPlayer

Intelligent flow in RealSystem G2 is the realization of the so-called adaptive flow management (ASM) API. ASM describes the type of streaming data, assists intelligent decision-making, and determines which type of data packet is sent. The file format and broadcast plug-in define ASM rules. In the simplest form, assign predefined attributes and average bandwidth to packet groups. For the advanced form, the ASM rules allow the plug-in to change the packet transmission based on changes in network conditions. Each ASM rule can have a demonstration with defined conditions. For example, the demonstration defines that the customer bandwidth is 5,000 to 15,000 Kbps, and the packet loss is less than 2.5%. If this condition describes the customer's current network connection, the customer subscribes to this rule. The attributes defined in the rules help RealServer to effectively transmit data packets. If the network conditions change, customers subscribe to a different rule.