INES – National Institute of Science and Technology for Software Engineering

Towards an integrated infra-structure for developing applications related to Big Data

Analytics focuses on manipulating data to find patterns and information useful for some purpose. This task has a high value in data processing and uses several areas, theories and underlying techniques such as statistics, programming, operational search, data mining, optimization, visualization, etc.

In this context, many auxiliary techniques are used by analytics for defining, applying and adapting other existing techniques. This originates costs related to problem definition, use of frameworks, application and evaluation of existing techniques, architectural and programmatic knowledge, issues related to high volumes of data, etc. Therefore, integrating tools, techniques and frameworks is highly desirable to reduce application costs during software development to deal with Big Data.

This project is aimed at producing a prototype (framework) that integrates methodologies and tools into the development process of applications related to Big Data. This contributes to the approximation of Software Engineering and Big Data areas, as the first deals with development process and the second essentially focuses on information extraction and processing of data (scalability). The idea is to provide a way of handling Big Data in software development, providing some abstraction level for developers, so that they can use analytics (and frameworks) without worrying about low-level details of their implementation.