From Smart 3D Capture to Seamless XR Delivery.

AI for 3D

XRculture explores new approaches to 3D digitisation by applying artificial intelligence techniques to the processing and reconstruction of cultural heritage objects. The focus is on objects that present challenges to traditional methods - such as transparent, reflective or textureless surfaces commonly found in museum collections.

Defining requirements for AI-driven 3D digitisation and XR optimisation

This phase laid the groundwork for the AI-driven 3D digitisation workflow, including a comprehensive metadata assessment to determine the necessary types of AI-based 3D models and ensure alignment with digital cultural heritage standards.

Key achievements

• Complete definition and validation of requirements for AI-based 3D content creation and improvement
• AI-based generation of new 3D cultural heritage models
• Improving 3D models from museums and digital collections
• Optimisation techniques for preparing 3D content for XR applications

AI-based generation of new 3D cultural heritage models

In this phase we explored existing AI-based 3D reconstruction tools, with a focus on NeRF-based and Gaussian Splatting techniques. We carried out multiple digitisation campaigns across Italy, Ukraine and Croatia, identifying in total 611 new models.

The campaigns targeted a wide range of archaeological and museum artefacts, including complex and non-collaborative materials (like glass or reflective surfaces), in line with the technical objectives of the project. Digitisation campaigns were carried out in:

• Bleschunov - Museum of Private Collections, Odessa (Ukraine) - Pixelated Realities
• Archaeological Museum, Pula (Croatia) - Inception
• Italian National Archaeological Museum of Adria (Italy) - Inception
• National Archaeological Museum of Verona (Italy) - Pixelated Realities
• National Archaeological Museum of Marche (Italy) - Università Politecnica delle Marche
• National Archaeological Museums of Lomellina (Italy) - FBK
• National Archaeological Museums of Ravenna (Italy) - Inception
• National Archaeological Museum of Aquileia (Italy) - FBK

Datasets describing all newly captured 3D content have been published on Zenodo, ensuring immediate availability, versioning, and traceability of the datasets.

Improving existing 3D models from museums and digital collections

The focus of this phase was to enhance the quality of the existing 3D models using AI pipelines. A total of 652 models were consolidated and prepared, with a mitigation strategy implemented to address availability constraints for legacy museum objects, including the creation of scaled heritage content.

A first set of over 200 museum-related 3D models was effectively collected and made available from partner repositories. These included content from archaeological museums in Italy: Adria (34 models), Ecolano (94 models), and Taranto (18 models), as well as the Bleschunov private collection in Odessa, Ukraine (50 models). Additional datasets were consolidated from UNIVPM archives, including models from Portugal (6) and Syria (10).

The consortium also expanded the scope of eligible content beyond complex museum artefacts to include architectural and building-scale heritage. In this context over 300 building-scale 3D models were identified and sourced from the archives of the Italian Ministry of Culture.

Overall approximately 500 existing 3D models have been identified and prepared for improvement through AI-based pipelines.

Parallel to the activities, synchronisation was carried out with the 3DBigDataSpace project which focuses on 3D model enrichment and optimisation, particularly in the creation of 3D model derivatives optimised for low-end use, enabling deployment in web platforms, mobile XR applications, and spatial visualizations.

Content optimisation techniques for the 3D-to-XR pipeline

During this phase we combined two key components to improve the interactivity and accessibility of 3D models in XR environments:

• Definition and validation of optimization strategies for XR use cases, including polygon reduction, level-of-detail management, texture compression, and format compatibility (e.g. gITF-based outputs).
• Preparatory work on the automation of 3D-to-2D video generation for holographic and spatial visualization, with ARCTUR’s HPC infrastructure being configured to support scalable and repeatable processing workflows.

These ongoing activities are being developed in close coordination with the improvement of existing 3D content, running in parallel to ensure that optimization and improvement pipelines converge towards consistent, reusable, and XR-ready outputs, particularly for low-end and mobile XR deployments.

XRculture Middleware Protocol

The focus within WP3 has been on developing the XRculture Middleware Protocol, which is now available.

The XRculture Middleware Protocol enables viewers, services and storage solutions to semantically define and explain their individual capabilities and functionalities. This allows any central HUB to ‘understand’ specific capabilities of individual fully distributed components.

Any solution requiring the handling of 3D model data can connect to one or more HUBs to visualise, process or search 3D models. This covers everything from simple websites that want to use the best possible solution for visualising a 3D model to applications that want to apply complex processes to improve the quality of or generate 3D models. Updates to available software will not require websites or solutions to update; new software can simply announce itself to one or more hubs using the protocol.

Several implementations of the protocol have been developed alongside the protocol itself. A large number of services are already compatible with the protocol, including online 3D viewers (both open-source and closed-source commercial versions), converters between different open 3D formats, remeshing tools for improving quality and/or reducing mesh size, a thumbnail generator, and an open-source service that converts a set of 2D images into a 3D model using photogrammetry or AI NeRF technology.

Several HUBs are public and their source code is available on GitHub, enabling anyone to build and host their own HUB. This allows all content to make use of this dynamic area of 3D models and services based on these models, with clear separation between functionality and responsibility through the use of the standardised XRculture Middleware Protocol.

Share3D Dashboard: Supporting the Publishing Pipeline for 3D Content

Share3D Dashboard is a standards-based platform designed to support the structured documentation, publication, and visualization of 3D cultural heritage (CH) assets, with a particular emphasis on interoperability within European aggregation infrastructures, such as Europeana.

The initial version of the system, developed in the Share3D project (2018-2020) and integrated into the CARARE aggregation pipeline. It provided a metadata authoring environment for 3D models sourced from online repositories, including Sketchfab and MyMiniFactory. This version utilized structured metadata forms enriched with controlled vocabularies (e.g. Art & Architecture Thesaurus), and supported serialization to the Europeana Data Model (EDM), enabling the seamless ingestion of 3D resources into Europeana’s publishing workflow.

Building on this operational experience, the platform is being re-envisioned within the XRculture project. The new implementation adopts a modern technology stack and enhanced interoperability mechanisms. Integration with external repositories such as Sketchfab and MyMiniFactory is extended through the OAuth 2.0 authentication protocol, while additional integrations - such as Zenodo and Repox - are being explored.

Zenodo has recently been adopted as a key input source for 3D models due to its flexibility in storing diverse file types. However, its limited structured metadata capabilities are addressed within the Share3D Dashboard through manual ingestion workflows. These allow users to ingest 3D models via direct file links, enabling external hosting while leveraging the platform for metadata enrichment and publication.

The metadata schema implemented in the platform aligns with the latest EDM guidelines for 3D content. It is organized into core categories such as Identity, Place, Time, Copyright, Relations and Model Creation. Additional elements capture the intended usage, the relationship between the digital model and its physical counterpart, as well as model typology and technical characteristics (e.g., point, polygon, and vertex counts). For technical metadata extraction, the platform integrates external services, including the Metadata Extractor service of the Repox 3D ecosystem, enabling automated metadata generation on a best-effort basis.

For visualization, the Share3D Dashboard dynamically selects the most appropriate viewer based on the source of the 3D model. When an embeddable viewer is available from the source platform, it is used by default. Otherwise, alternative solutions - such as the Repox 3D Viewer - are employed to ensure consistent visualization across all records.

Access to the platform is provided through a moderated registration process. Approved users are required to sign a Data Exchange Agreement (DEA) before publishing content to Europeana via the Share3D Dashboard, ensuring compliance with data sharing and licensing requirements.