WP8 – Advanced Statistics and Data Analysis

Lattice Field Theory has set a high standard both for high performance computing and data analytics. Our field plays an important role in addressing current challenges related to the volume and variability of big data sets and their analysis. The solutions developed have a wide applicability outside of the lattice community, for the benefit of sectors of science where computational methods are just emerging and of the wider industry relying on data analytics. This emphasis on data has generated new interest in artificial intelligence and machine learning techniques.

In this respect, the Dense QCD project will be dedicated to the study of supervised and unsupervised machine learning technique applications in the investigation of strongly interacting matter at finite chemical potential. The objective of this effort is to unveil new structures in the data that may reveal symmetries of exotic phases at high density.

Novel machine learning techniques will be studied in collaboration with industrial partners, in order to favor a fast transfer of knowledge. Concerning more structural aspects, one important issue affecting large datasets that span across various applications (including Lattice Gauge Theory) is the necessity to minimize data movement. At the same time, with the increase in adoption of HPC techniques, it is crucial to decrease the learning curve for novel adopters. In order to address these needs (common in academic and industrial contexts), our target is to develop a system built on an open source web-based platform of cloud computing equipped with a system of containers to execute pre-packaged applications on computational platforms. In addition, the web interface will be extended to allow processing of distributed data.

Besides applications across other work packages (e.g. in the analysis of large configuration sets), we expect these tools to find natural applications in industrial contexts.