Big Data Persistence and Analytics Architectures¶
The need for levels of availability and scalability beyond those supported by relational databases has led to the emergence of a new generation of purpose-specific databases grouped under the term NoSQL. In general, NoSQL databases are designed with horizontal scalability as a primary concern and deliver increased availability and fault-tolerance at a cost of temporary inconsistency and reduced durability of data. To balance the requirements for data consistency and availability, organisations increasingly migrate towards hybrid data persistence architectures comprising both relational and NoSQL databases. The consensus is that this trend will only become stronger in the future; critical data will continue to be stored in ACID (predominately relational) databases while non-critical data will be progressively migrated to high-availability NoSQL databases.
TYPHON is a European Commission H2020 project (2018-2020, €4.4M), of which I am the Technical Director, which aims to provide a methodology and an integrated technical offering for designing, developing, querying and evolving scalable architectures for persistence, analytics and monitoring of large volumes of hybrid (relational, graph-based, document-based, natural language etc.) data.
TYPHON brings together research partners with a long track record of conducting internationally-leading research on software modelling, domain-specific languages, text mining and data migration, and of delivering research results in the form of robust and widely-used open-source software, industrial partners active in the automotive, earth observation, banking, and motorway operation domains, an industrial advisory board of world-class experts in the fields of databases, business intelligence and analytics, and large-scale data management, and a global consortium including more than 400 organisations from all sectors of IT.