The 21st RISIS Research Seminar will take place on 9th November from 12.30 to 2.00 pm (CET) and will focus on a presentation entitled Reliability of innovation measures by Mika Naumanen, VTT Technical Research Centre. Carter Bloch, University of Aarhus is involved as a discussant.
The Community Innovation Surveys are the main data source for measuring innovation in Europe. The aggregated CIS data tables cover the basic information of the enterprise, product and process innovation, innovation activity and expenditure, effects of innovation, innovation cooperation, public funding of innovation, source of information for innovation patents, etc. Similarly, national statistical institutions collect information on the R&D activities in their respective countries.
The study examines the reliability of innovation measures by comparing the responses company representatives have given to the Community Innovation Surveys and national R&D surveys starting from the year 2000. The work analyses the stability and relatedness of the answers over time as well as their correlation to the company characteristics, in particular to the share of R&D and ICT personnel in the company.
Four general classes of reliability estimates are examined, each of which estimates reliability in a different way, that is, Inter-rater reliability assesses the degree to which different raters/observers give consistent estimates of the same phenomenon; test-retest reliability assesses the consistency of a measure from one time to another; parallel-forms reliability assessing the consistency of the results of two tests constructed in the same way from the same content domain, and internal consistency reliability assesses the consistency of results across items within a test. The researchers wish that their study will improve the applicability of the innovation data by noting the survey data that seem to be reliable and the ones where caution might be needed.
