A Recommendation Strategy based on Similarity in Big Measurement Repositories

Abstract

The data stream processing architecture is an Apache Storm-based processing strategy focused on the Measurement and Evaluation (M&E) projects. Projects are defined using an M&E framework which allows previously establishing the entity under monitoring jointly with their associated concepts. This approach is able to guide the processing and decision making based on data coming from multiple concurrent projects supported by the semantics of each tag defined into each project scope. The strategy incorporates active behaviour, which implies it is possible to provide recommendations or instruct courses of actions given a situation based on previous experiences. However, it is possible there exist situations for which there are no specific knowledge or previous experiences (e.g. in front of new situations), being no possible to provide suggestions. Thus, the idea is to detect entities under monitoring by similarity, establishing a score that allows providing experiences and knowledge by analogy. In this way, when there is no experience available for a given entity, other experiences and knowledge could be retrieved from third entities by analogy, being able to provide approximated suggestions. Here, advances of the research line are introduced in terms of the establishment of structural and behavioural coefficients, the effect of load shedding techniques in the data stream processing, the perspectives of measurement and evaluation projects based in a project-common definition language jointly with its effect on the reliability of the measurement and evaluation system.