Publikationen


Suche nach „[K.] [Brunnström]“ hat 15 Publikationen gefunden
Suchergebnis als PDF
    DigitalAngewandte Informatik

    Zeitschriftenartikel

    K. Brunnström, Marcus Barkowsky

    Statistical quality of experience analysis: on planning the sample size and statistical significance testing

    Journal of Electronic Imaging, vol. 27, no. 5

    2018

    DOI: 10.1117/1.JEI.27.5.053013

    Abstract anzeigen

    This paper analyzes how an experimenter can balance errors in subjective video quality tests between the statistical power of finding an effect if it is there and not claiming that an effect is there if the effect is not there, i.e., balancing Type I and Type II errors. The risk of committing Type I errors increases with the number of comparisons that are performed in statistical tests. We will show that when controlling for this and at the same time keeping the power of the experiment at a reasonably high level, it is unlikely that the number of test subjects that are normally used and recommended by the International Telecommunication Union (ITU), i.e., 15 is sufficient but the number used by the Video Quality Experts Group (VQEG), i.e., 24 is more likely to be sufficient. Examples will also be given for the influence of Type I error on the statistical significance of comparing objective metrics by correlation. We also present a comparison between parametric and nonparametric statistics. The comparison targets the question whether we would reach different conclusions on the statistical difference between the video quality ratings of different video clips in a subjective test, based on the comparison between the student T-test and the Mann‐Whitney U-test. We found that there was hardly a difference when few comparisons are compensated for, i.e., then almost the same conclusions are reached. When the number of comparisons is increased, then larger and larger differences between the two methods are revealed. In these cases, the parametric T-test gives clearly more significant cases, than the nonparametric test, which makes it more important to investigate whether the assumptions are met for performing a certain test.

    DigitalAngewandte Informatik

    Zeitschriftenartikel

    Marcus Barkowsky, E. Masala, G. van Wallendael, K. Brunnström, N. Staelens, P. Le Callet

    Objective Video Quality Assessment ‐- Towards Large Scale Video Database Enhanced Model Development

    IEICE Transactions on Communications, vol. E-98b, no. 1, pp. 2-11

    2015

    Abstract anzeigen

    The current development of video quality assessment algorithms suffers from the lack of available video sequences for training, verification and validation to determine and enhance the algorithm's application scope. The Joint Effort Group of the Video Quality Experts Group (VQEG-JEG) is currently driving efforts towards the creation of large scale, reproducible, and easy to use databases. These databases will contain bitstreams of recent video encoders (H.264, H.265), packet loss impairment patterns and impaired bitstreams, pre-parsed bitstream information into files in XML syntax, and well-known objective video quality measurement outputs. The database is continuously updated and enlarged using reproducible processing chains. Currently, more than 70,000 sequences are available for statistical analysis of video quality measurement algorithms. New research questions are posed as the database is designed to verify and validate models on a very large scale, testing and validating various scopes of applications, while subjective assessment has to be limited to a comparably small subset of the database. Special focus is given on the principles guiding the database development, and some results are given to illustrate the practical usefulness of such a database with respect to the detailed new research questions.

    DigitalAngewandte Informatik

    Zeitschriftenartikel

    Marcus Barkowsky, I. Sedano, K. Brunnström, M. Leszczuk, N. Staelens

    Hybrid video quality prediction: reviewing video quality measurement for widening application scope

    Multimedia Tools and Applications, vol. 74, pp. 323-343

    2014

    DOI: 10.1007/s11042-014-1978-2

    Abstract anzeigen

    A tremendous number of objective video quality measurement algorithms have been developed during the last two decades. Most of them either measure a very limited aspect of the perceived video quality or they measure broad ranges of quality with limited prediction accuracy. This paper lists several perceptual artifacts that may be computationally measured in an isolated algorithm and some of the modeling approaches that have been proposed to predict the resulting quality from those algorithms. These algorithms usually have a very limited application scope but have been verified carefully. The paper continues with a review of some standardized and well-known video quality measurement algorithms that are meant for a wide range of applications, thus have a larger scope. Their individual artifacts prediction accuracy is usually lower but some of them were validated to perform sufficiently well for standardization. Several difficulties and shortcomings in developing a general purpose model with high prediction performance are identified such as a common objective quality scale or the behavior of individual indicators when confronted with stimuli that are out of their prediction scope. The paper concludes with a systematic framework approach to tackle the development of a hybrid video quality measurement in a joint research collaboration.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    Marcus Barkowsky, K. Brunnström, T. Ebrahimi, L. Karam, P. Lebreton, P. Le Callet, A. Perkis, A. Raake, M. Subedar, K. Wang, L. Xing, J. You

    Subjective and Objective Visual Quality Assessment in the Context of Stereoscopic 3D-TV

    3D-TV System with Depth-Image-Based Rendering

    2013

    ISBN: 978-1-4419-9963-4

    DOI: 10.1007/978-1-4419-9964-1_14

    DigitalAngewandte Informatik

    Zeitschriftenartikel

    K. Brunnström, I. Ananth, C. Hedberg, K. Wang, B. Andrén, Marcus Barkowsky

    36.4: Comparison between Different Rating Scales for 3D TV

    SID Symposium Digest of Technical Papers, vol. 44, no. 1, pp. 509-512

    2013

    DOI: 10.1002/j.2168-0159.2013.tb06256.x

    Abstract anzeigen

    Stereoscopic 3D viewing experience has been studied quite intensively recently, but still the subjective test methods have not yet been settled. It has become clear that the 3D viewing experience cannot easily be described by just one scale. This paper describes a study where three different rating scales (Quality, Discomfort and Presence) are compared in a subjective test, combined with two viewing distances. The results shows that in a stereoscopic 3D video quality test targeting mainly coding distortions one scale such as video quality could be sufficient.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    K. Wang, K. Brunnström, Marcus Barkowsky, M. Urvoy, M. Sjöström, P. Le Callet, S. Tourancheau, B. Andrén

    Stereoscopic 3D video coding quality evaluation with 2D objective metrics

    Proceedings of SPIE Vol. 8648: Stereoscopic Displays and Applications XXIV

    2013

    Abstract anzeigen

    The 3D video quality is of highest importance for the adoption of a new technology from a user’s point of view. In this paper we evaluated the impact of coding artefacts on stereoscopic 3D video quality by making use of several existing full reference 2D objective metrics. We analyzed the performance of objective metrics by comparing to the results of subjective experiment. The results show that pixel based Visual Information Fidelity metrics fits subjective data the best. The 2D stereoscopic video quality seems to have dominant impact on the coding artefacts impaired stereoscopic videos.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    Marcus Barkowsky, J. Li, T. Han, S. Youn, J. Ok, C. Lee, I. Vijai Ananth, K. Wang, K. Brunnström, P. Le Callet

    Towards standardized 3DTV QoE assessment: Cross-lab study on display technology and viewing environment parameters

    Proceedings of SPIE Vol. 8648: Stereoscopic Displays and Applications XXIV;

    2013

    Abstract anzeigen

    Subjective assessment of Quality of Experience in stereoscopic 3D requires new guidelines for the environmental setup as existing standards such as ITU-R BT.500 may no longer be appropriate. A first step is to perform cross-lab experiments in different viewing conditions on the same video sequences. Three international labs performed Absolute Category Rating studies on a freely available video database containing degradations that are mainly related to video quality degradations. Different conditions have been used in the labs: Passive polarized displays, active shutter displays, differences in viewing distance, the number of parallel viewers, and the voting device. Implicit variations were introduced due to the three different languages in Sweden, South Korea, and France. Although the obtained Mean Opinion Scores are comparable, slight differences occur in function of the video degradations and the viewing distance. An analysis on the statistical differences obtained between the MOS of the video sequences revealed that obtaining an equivalent number of differences may require more observers in some viewing conditions. It was also seen that the alignment of the meaning of the attributes used in Absolute Category Rating in different languages may be beneficial. Statistical analysis was performed showing influence of the viewing distance on votes and MOS results.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    Marcus Barkowsky, N. Staelens, L. Janowski, Y. Koudota, M. Leszczuk, M. Urvoy, P. Hummelbrunner, I. Sedano, K. Brunnström

    Subjective experiment dataset for joint development of hybrid video quality measurement algorithms

    QoEMCS 2012 ‐ Third Workshop on Quality of Experience for Multimedia Content Sharing, Berlin

    2012

    Abstract anzeigen

    The application area of an objective measurement algorithm for video quality is always limited by the scope of the video datasets that were used during its development and training. This is particularly true for measurements which rely solely on information available at the decoder side, for example hybrid models that analyze the bitstream and the decoded video. This paper proposes a framework which enables researchers to train, test and validate their algorithms on a large database of video sequences in such a way that the ‐ often limited ‐ scope of their development can be taken into consideration. A freely available video database for the development of hybrid models is described containing the network bitstreams, parsed information from these bitstreams for easy access, the decoded video sequences, and subjectively evaluated quality scores.

    DigitalAngewandte Informatik

    Zeitschriftenartikel

    K. Wang, Marcus Barkowsky, K. Brunnström, M. Sjöström, R. Cousseau, P. Le Callet

    Perceived 3D TV Transmission Quality Assessment: Multi-Laboratory Results Using Absolute Category Rating on Quality of Experience Scale

    IEEE Transactions on Broadcasting, vol. 58, no. 4, pp. 544-557

    2012

    Abstract anzeigen

    Inspired by the rapidly increasing popularity of 3D movies, there is an industrial push for 3DTV services to the home. One important factor for the success and acceptance by the viewers is a positive quality of experience (QoE) of the new service when delivered. The questions of how to efficiently deliver 3DTV service to the home, and how to evaluate the visual quality perceived by end users are a recent research focus. We have investigated users\textquoteright experience of stereoscopic 3D video quality by preparing two subjective assessment datasets. The first dataset aimed at the evaluation of efficient transmission in the transmission error free case, while the second focused on error concealment. A total of three subjective assessments, two for first dataset and one for the second, were performed using the Absolute Category Rating with Hidden unimpaired Reference video (ACR-HR) method. The experimental setup allows to show that the ACR-HR subjective method provides repeatable results across labs and across conditions for video quality. It was also verified that MVC is more efficient than H.264 simulcast coding. Furthermore it was discovered that based on the same level of quality of experience spatial down-sampling may lead to better bitrate efficiency while temporal down-sampling is not acceptable. When network impairments occur, traditional error 2D concealment methods need to be reinvestigated as they were outperformed by displaying the same view for both eyes (switching to 2D presentation).

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    S. Tourancheau, K. Wang, J. Bulat, R. Cousseau, L. Janowski, K. Brunnström, Marcus Barkowsky

    Reproducibility of crosstalk measurements on active glasses 3D LCD displays based on temporal characterization

    Proceedings of SPIE Vol. 8288: Stereoscopic Displays and Applications XXIII

    2012

    ISBN: 9780819489357

    Abstract anzeigen

    Crosstalk is one of the main display-related perceptual factors degrading image quality and causing visual dis-comfort on 3D-displays. It causes visual artifacts such as ghosting eects, blurring, and lack of color delitywhich are considerably annoying and can lead to diculties to fuse stereoscopic images. On stereoscopic LCDwith shutter-glasses, crosstalk is mainly due to dynamic temporal aspects: imprecise target luminance (highlydependent on the combination of left-view and right-view pixel color values in disparity regions) and synchro-nization issues between shutter-glasses and LCD. These dierent factors inuence largely the reproducibilityof crosstalk measurements across laboratories and need to be evaluated in several dierent locations involvingsimilar and diering conditions. In this paper we propose a fast and reproducible measurement procedure forcrosstalk based on high-frequency temporal measurements of both display and shutter responses. It permitsto fully characterize crosstalk for any right/left color combination and at any spatial position on the screen.Such a reliable objective crosstalk measurement method at several spatial positions is considered a mandatoryprerequisite for evaluating the perceptual inuence of crosstalk in further subjective studies.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    K. Brunnström, I. Sedano, K. Wang, Marcus Barkowsky, M. Kihl, P. Le Callet, Patrick, M. Sjöström, A. Aurelius

    2D No-Reference Video Quality Model Development and 3D Video Transmission Quality

    Sixth International Workshop on Video Processing and Quality Metrics for Consumer Electronics - VPQM 2012 (Jan 2012; Scottsdale, AZ, USA)

    2012

    Abstract anzeigen

    This presentation will target two different topics in video quality assessment. First, we discuss 2D no-reference video quality model development. Further, we discuss how to find suitable quality for 3D video transmission. No-reference metrics are the only practical option for monitoring of 2D video quality in live networks. In order to decrease the development time, it might be possible to use full-reference metrics for this purpose. In this work, we have evaluated six full-reference objective metrics in three different databases. We show statistically that VQM performs the best. Further, we use these results to develop a lightweight no-reference model. We have also investigated users' experience of stereoscopic 3D video quality by performing the rating of two subjective assessment datasets, targeting in one dataset efficient transmission in the transmission error free case and error concealment in the other. Among other results, it was shown that, based on the same level of quality of experience, spatial down-sampling may lead to better bitrate efficiency while temporal down-sampling will be worse. When network impairments occur, traditional error 2D concealment methods need to be reinvestigated as they were outperformed switching to 2D presentation.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    N. Staelens, I. Sedano, Marcus Barkowsky, L. Janowski, K. Brunnström, P. Le Callet

    Standardized Toolchain And Model Development For Video Quality Assessment ‐ The Mission Of The Joint Effort Group In Vqeg

    Proceedings of 2011 Third International Workshop on Quality of Multimedia Experience (QoMEX), Mechelen, Belgique

    2011

    Abstract anzeigen

    Since 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group.

    DigitalAngewandte Informatik

    Zeitschriftenartikel

    Marcus Barkowsky, S. Tourancheau, K. Brunnström, K. Wang, B. Andrén

    55.3: Crosstalk Measurements of Shutter Glasses 3D Displays

    SID Symposium Digest of Technical Papers, vol. 42, no. 1, pp. 812-815

    2011

    Abstract anzeigen

    Crosstalk is probably one of the main perceptual factors contributing to perceived image quality and visual comfort. The Video Quality Experts Group (VQEG) within their 3D video quality project is specifying a practical measurement procedure that will produce consistent results across laboratories. This paper is part of that effort. Two different method of measuring cross talk on shutter glasses stereo displays have been studied. One is based on time average luminance measurements and the other on temporal measurements. The results show that crosstalk is roughly 0.5% but that there are differences in the crosstalk between the two eyes in the shutter glasses.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    K. Wang, Marcus Barkowsky, R. Cousseau, K. Brunnström, R. Olsson, P. Le Callet, M. Sjöström

    Subjective evaluation of HDTV stereoscopic videos in IPTV scenarios using absolute category rating

    Proceedings of SPIE Vol. 7863: Stereoscopic Displays and Applications XXII

    2011

    Abstract anzeigen

    Broadcasting of high definition (HD) stereobased 3D (S3D) TV are planned, or has already begun, in Europe, the US, and Japan. Specific data processing operations such as compression and temporal and spatial resampling are commonly used tools for saving network bandwidth when IPTV is the distribution form, as this results in more efficient recording and transmission of 3DTV signals, however at the same time it inevitably brings quality degradations to the processed video. This paper investigated observers quality judgments of state of the art video coding schemes (simulcast H.264/AVC or H.264/MVC), with or without added temporal and spatial resolution reduction of S3D videos, by subjective experiments using the Absolute Category Rating method (ACR) method. The results showed that a certain spatial resolution reduction working together with high quality video compressing was the most bandwidth efficient way of processing video data when the required video quality is to be judged as "good" quality. As the subjective experiment was performed in two different laboratories in two different countries in parallel, a detailed analysis of the interlab differences was performed.

    DigitalAngewandte Informatik

    Beitrag (Sammelband oder Tagungsband)

    Marcus Barkowsky, K. Wang, R. Cousseau, K. Brunnström, R. Olsson, P. Le Callet

    Subjective Quality Assessment of Error Concealment Strategies for 3DTV in the presence of asymmetric Transmission Errors

    2010 18th International Packet Video Workshop

    2010

    Abstract anzeigen

    The transmission of 3DTV sequences over packet based networks may result in degradations of the video quality due to packet loss. In the conventional 2D case, several different strategies are known for extrapolating the missing information and thus concealing the error. In 3D however, the residual error after concealment of one view might leads to binocular rivalry with the correctly received second view. In this paper, three simple alternatives are presented: frame freezing, a reduced playback speed, and displaying only a single view for both eyes, thus effectively switching to 2D presentation. In a subjective experiment the performance in terms of quality of experience of the three methods is evaluated for different packet loss scenarios. Error-free encoded videos at different bit rates have been included as anchor conditions. The subjective experiment method contains special precautions for measuring the Quality of Experience (QoE) for 3D content and also contains an indicator for visual discomfort. The results indicate that switching to 2D is currently the best choice but difficulties with visual discomfort should be expected even for this method.