So far, the methods of estimating the agreement, which have been corrected on the possibility of conducting evaluations of the free response, have necessitated a simplification of the data in order to highlight the negative results. One option is to analyze the data at a patient`s level by deeming a patient to be “positive” when at least one lesion is detected, but this results in a significant loss of information. Another approach is to divide the X-ray into areas of interest. Each region of interest is then evaluated by all advisors. Because negative ratings are explicitly reported, the number of regions considered negative by all advisors is known and default statistics can be calculated. This approach reduces the loss of information compared to a single dichotomous assessment per patient, but the areas of interest must be small and numerous enough to maintain clinical relevance. In a diagnostic study, for example, Mohamed et al  defined 68 areas of interest per patient. In general, limiting a free response paradigm to a finite number of evaluations (patients or regions) results in a loss of information and may lead to an overestimation of the agreement, as differences of opinion are ignored below the level of granularity chosen. Kundel HL, Polansky M. Measure of the observation agreement. radiology. 2003;228:303–8.
A good agreement between advisors is a desirable property of any diagnostic method. The agreement is generally assessed by Kappa`s statistics , which quantify the extent to which the agreement observed between the councillors exceeds the agreement solely on the basis of chance. The evaluation of Kappa`s statistics requires that the number of positive (or abnormal) and negative (or normal) evaluations be known to all advisors. This is not the case when advisors report only positive results and do not report the number of negative results. This situation can be seen as the paradigm of free reaction . This is a common situation in imaging techniques where the raters generally report positive results, but do not list all the negative observations for a given patient. Kappa will only address its maximum theoretical value of 1 if the two observers distribute codes in the same way, i.e. if the corresponding totals are the same. Everything else is less than a perfect match.
Nevertheless, the maximum value Kappa could achieve helps, as uneven distributions help interpret the actual value received from Kappa. The equation for the maximum is:  If patients can contribute to more than one observation, the data are grouped together. Yang et al  proposed a Kappa statistic based on the usual formula (po-pe)/(1-Pe), pô being a weighted average of cluster concordance (patients) and pe is derived from weighted investment weighted averages of the ratings of each rat. In this approach, Kappa has the same estimate for aggregated data as for cluster ignorance. Therefore, Basic Table 2 × 2 is also appropriate for estimating the agreement for aggregated data.