STELLA II - Infrastructures for Living Labs

The STELLA project provides an innovative technology and methodology infrastructure that allows information providers to evaluate their information systems with the actual users of their web platforms. The underlying principle of the Living Labs allows STELLA to interpose itself between the communication of the users with the systems and thus not to evaluate in an artificial and static laboratory environment (as usual in so-called TREC studies), but to pick up the users there, where they satisfy their actual information needs. By incorporating the Living Lab principle, the systems can be evaluated iteratively and continuously, thus taking a big step towards a "Continues Evaluation", as is already common practice with large commercial web platforms (such as Amazon). After a successful test run of the platform at the two specialist information providers GESIS and ZB MED and the implementation of an evaluation campaign in the context of the international conference CLEF, a special focus will be placed on the continuation and dissemination of the evaluation infrastructure in project phase II. To this end, the project will integrate further external providers from the German-language specialist information landscape, into whose systems STELLA will be implemented. In order to make it as easy as possible for the partners to get started, the STELLA II team will take on the role of a technical and methodological consultant and will also carry out consulting and development work on site. In order to anchor the Living Labs not only technically but also methodologically further into the professional information landscape, special attention will be paid to the principle of reproducibility and comparability of the experiments.