XR for Heritage at Risk

Overview & Core Objectives

This use case focuses on leveraging Extended Reality (XR) technologies to safeguard cultural heritage located in regions threatened by active conflict or environmental hazards. Led by Pixelated Realities in collaboration with consortium partners (IN2, ARCTUR, FBK, UNIVPM, RDF), the primary objective is to empower non-expert museum professionals, volunteers, and emergency responders to rapidly preserve, document, and share digital heritage datasets under severe time and resource constraints.

Poster: XR Scenario for Heritage at Risk: Testing of AR app for museums

Key Technological Components

• Mobile AR Capture Application & 3D Reconstruction MVP: An open-source, offline-first Android MAUI application tailored for field photogrammetry and Gaussian Splatting. It features automated metadata generation and offline capture guidance, allowing field personnel to document artifacts confidently without internet connectivity. A companion 3D Reconstruction Service runs on a secure online server or locally via a Docker image when connectivity becomes available.
• Ukrainian Heritage 3D Data Aggregation Pipeline: A pipeline integrating Zenodo, Repox 3D, and the Share3D Dashboard to streamline uploading models and documentation into national registries and Europeana. The pipeline was piloted using challenging movable artifacts (e.g., glass, porcelain, gemstone jewelry, lacquered icons) and immovable heritage (e.g., the Kuialnyk necropolis Cross collection) from Ukrainian institutions.
• Interoperability & National Viewer Integration: The first-ever integration of a Gaussian Splatting 3D viewer directly into the Registry of the Museum Fund of Ukraine (RMFU), establishing a standardized Middleware Protocol for 3D Viewers within the Europeana Data Space.

Reconstruction results for training in mobile photogrammetry with AR app developed for Scenario

Testing, Deployment & Key Findings

Field testing was conducted through hands-on practical training sessions and public presentations, combining photogrammetry basics with live testing of the mobile app.

• Participants: Engaged around 23 museum staff, directors, and field personnel directly (30 participants in the core training session), reaching over 692 Cultural Heritage Institutions (CHIs) indirectly across national networks.
• User Feedback: Participants validated the critical need for offline functionality, simplified field workflows, and secure data pipelines during emergency conditions.

Views from user training - on-site, online and hybrid.

Wider Cultural & Strategic Impact

• Data Sovereignty & Security: The local Docker container deployment ensures sensitive digitised data remains hosted in-country without foreign server dependencies.
• National-Level Adoption: The Ministry of Culture of Ukraine engaged directly with the project, expressing intent to adopt Version 2 of the app and apply the Middleware Protocol to future RMFU infrastructure.
• "Hidden Evacuation" & Diplomacy: Cross-border partnerships (such as exhibiting digitised porcelain collections from Odesa in Bielsko-Biała, Poland) demonstrated how digital twins can safeguard endangered heritage abroad while catalyzing international academic research.

3D asset on the RMFU platform. A landing page for a Europeana aggregated object from OMMPC named after O. Bleschunov