IEEE MultiMedia

PrePrint: Multimedia Presentation for Computer Games and Web 3.0

The HTML standard is an old and text-based format that is designed for narrowband networks. New enhancements to HTML, as well as new formats, are being evaluated. We wish to present a multimedia presentation system, format, and use cases for a wide range of context uses, including Web 3.0 and computer games. The system is compact and enables high performance 2D and 3D graphics. Use cases include in-game heads-up displays, remote playing of games, and 3D authoring for Web 3.0.

PrePrint: Web-based Music Lecture Database Framework with Aligned MIDI Score and Real Performance Audio

This paper presents a framework for authoring, storing, retrieving, and presenting music lectures on the Web. For a synchronized presentation between score and recorded performance audio, we propose a dynamic programming-based algorithm for MIDI-to-Wave alignment to explore the temporal relations between MIDI and the corresponding performance recording. With rapid advances in music transcription technology, it had become more possible to align MIDI and wave in a symbolic domain. However, transcription errors usually occur when transcribing polyphonic music or multi-instruments music because the complex harmonic of different instruments. The proposed alignment algorithm works in the symbolic domain even if many transcription errors have occurred. The aligned MIDI and wave can be attached to many kinds of teaching materials. With a synchronized presentation, learners can read music scores and get instructional information when listening to certain sections of music pieces. We built an evaluation system for doing a subjective evaluation. The percentage of bars which were regarded as aligned perfectly and aligned within acceptable limits is 97.08%. The questionnaire in the evaluation system also reported positive opinions from both engineers and musicians.

PrePrint: The Case for VM-based Cloudlets in Mobile Computing

Resource poverty is a fundamental constraint that severely limits the class of applications that can be run on mobile devices. We present a vision of mobile computing that breaks free of this fundamental constraint. In this vision, mobile users seamlessly utilize nearby computers to obtain the resource benefits of cloud computing without incurring WAN delays and jitter. Rather than relying on a distant "cloud," a mobile user instantiates a "cloudlet" on nearby infrastructure and uses it via a wireless LAN. Crisp interactive response for immersive applications that augment human cognition is then much easier to achieve because of the proximity of the cloudlet. We confirm that a critical aspect of this vision, namely rapid customization of cloudlet infrastructure, is achievable through dynamic VM synthesis.

PrePrint: An MPEG-7 Compatible Video Retrieval System with Integrated Support for Complex Multimodal Queries

We present BilVideo-7, an MPEG-7 compatible, video indexing and retrieval system that supports complex multimodal queries in a unified framework. An MPEG-7 profile is developed to represent the videos by decomposing them into Shots, Keyframes, Still Regions and Moving Regions. The MPEG-7 compatible XML representations of videos according to this profile are obtained by the MPEG-7 compatible video feature extraction and annotation tool of BilVideo-7, and stored in a native XML database. Users can formulate text-based semantic, color, texture, shape, location, motion and spatio-temporal queries on an intuitive, easy-to-use Visual Query Interface, whose Composite Query Interface can be used to specify very complex queries containing any type and number of video segments with their descriptors. The multi-threaded Query Processing Server parses incoming queries into subqueries and executes each subquery in a separate thread. Then, it fuses subquery results in a bottom-up manner to obtain the final query result. The whole system is unique in that it provides very powerful querying capabilities with a wide range of descriptors and multimodal query processing in an MPEG-7 compatible interoperable environment. We present sample queries to demonstrate the capabilities of the system.

PrePrint: Layer 5 Session Handoff: Mobility Management for Video Streaming Sessions among Heterogeneous Networks and Devices

In the past decade, mature video processing and Internet technologies have made video services over IP feasible and increasingly popular, e.g., video-on-demand (VoD) and IPTV. Considering mobile life over heterogeneous networks, a user can exploit any device to enjoy/share pictures, videos and music at anytime and anywhere, and he may change his currently used device or attached network to another one. To manage video streaming sessions among heterogeneous networks and devices, we developed a Ubiquitous Multimedia Service Platform for Mobile Life over the Digital Home Environment (UMS-MLDH) in a 3-year (2006-2009) National Telecommunication Project (NTP). Many distinct-domainsˇ core techniques are devised and well integrated in UMS-MLDH, including session mobility management, session signaling and communication and adaptive layered video streaming over a 3-tier server-proxy-client network architecture. In our experiments, the system integration, implementation and merits of UMS-MLDH are exhibited over a hybrid Home-Vehicular-Office network.

PrePrint: Cooperative Beam-forming in "Smart Dust": Getting rid of Multi-hop Communications

Bridging the digital with the physical world is made feasible through a proposed system architecture utilizing recent developments in cooperative beam-forming, RFIC technology and spread-spectrum techniques. In the proposed system, nodes will have to be activated only when they are triggered to send their own data to the fusion centre, without burdening the network with MAC layer collisions and energy-consuming routing algorithms, thus guaranteeing long lifetime cycles. Due to the simplicity of the node architecture that is used to implement the proposed system, the cost, size and energy consumption of the nodes can finally justify the use of the term ‘smart dust’.

IEEE MultiMedia - July-September 2009 (Vol. 16, No. 3)

IEEE MultiMedia

PrePrint: Mobile Application Profiling for Connected Smartphones

The Smartphone application market is expected to continue to grow at even higher rates and keeping high levels of quality will be a key factor for successful growth. The community of mobile developers is attracting members from other environments with mobile version initiatives for languages like Python or Ruby. But developing mobile applications requires facing mobile specific constraints and issues. With current development environments and profiling tools, it is highly difficult to deal with mobile communication issues: the main problems appear when using actual devices in real operations instead of emulators. We introduce a tool that aims to fill that gap by providing developers with the means to analyze mobile data communications. The key novelty of the profiling tool introduced in this paper is the correlation of traffic information, radio access technology measurements and location data, which helps third party developers test and evaluate their mobile applications in the field.