A Collaborative Environment for Assisted 3D Reconstruction of Cuneiform Tablets

Introduction

The aim of this project has been to improve the worldwide collaborative workflow that allows cuneiform experts and enthusiasts to work on the reconstruction of 3D cuneiform tablets via an interactive 3D interface.

The project involves the cataloguing of 3D cuneiform tablets in a structure that allows efficient network transmissions and algorithms for automatic/assisted reconstructions. The project contributes toward the recovery of more than three millennia of written history in one of the most important regions of the ancient world. The results can also be extended to a range of different fields that require collaborative effort.

Grant Code

The Leverhulme Trust's logo

RCEJ15249 - Leverhulme Grant - F000 94 BP - i3D Cuneiform Reconstruction

Researchers

Drs Eugene Ch'ng, Sandra I Woolley and Erlend Gehlken
in collaboration with Andrew Lewis, Tim Collins and Luis Hernandez-Munoz

Background

Cuneiform script is one of mankind’s earliest systems of writing (3,300BC). The invention of writing was an intellectual breakthrough that made recording information possible. Cuneiform was written with a reed stylus on small “tablets” made out of clay.

Cuneiform was a medium of international communication across the Ancient Middle East and was even used at the beginning of the Christian era. Current collections of thousands of inscribed fragments from complete tablets are distributed in museums worldwide. Pieces of one tablet may be in different museums due to illicit and unprofessional excavations. The largest collections are found in the British Museum, the Louvre (Paris), the Vorderasiatische Museum (Berlin) and the Iraq Museum in Baghdad. Smaller sets of collections in Britain are housed in The Ashmolean Museum of Oxford and also in The Birmingham Museum and Art Gallery. Heidelberg University houses a collection of about 3,000 unpublished fragments which belong to a single Late Babylonian temple archive from the time of Nebuchadnezzar.

Cuneiform tablets are still being discovered although the current political instability of Mesopotamia (modern-day Iraq and the adjoining countries) do not allow regular excavations at present.

Cuneiform collections are catalogued by excavation or museum numbers with a short description of the tablets and with accompanying photographs or hand-drawn copies. In some cases, large collections of tablets numbering thousands remained un-catalogued even after a century of storage.

The ancient texts on the tablets depict content encompassing religious, literary, scientific and everyday life documentation of their age and include encyclopaedias, dictionaries, political texts, letters, administrative documents and school tablets.

The reconstruction of these fragments into a readable form can provide researchers valuable insights into the otherwise partial data related to the social, political, scientific and historical aspects of these ancient cultures. As time has progressed, however, little progress has been made in this reconstruction while at the same time antiquities of the region have been lost and damaged. It is of utmost importance that fragments are digitised, reconstructed, and read as long as it is still possible.

The System

The system interface was defined via a formal participatory design process and informed by usability testing feedback. Implementations have been made on desktop computers, on a 3D monitor and on touchscreen systems from tablets to large displays.
The resources created to assist in the reconstruction process include manipulation, joining and annotating tools.
In addition, the project has successfully joined fragments automatically using a novel matching algorithm with measure of fit metrics which dramatically reduce false positive match reports.

The system interface defined by participatory design and usability testing feedback

CARE system interface

 

The Touchscreen Interface

 

The Desktop Interface

 

Interactive Displays

Interactive displays

 

Automated Reconstruction of Fragmented Cuneiform Tablets

Automated Matching of Fragments

Part of the research into the virtual reconstruction of cuneiform tablets has focused on automated methods for finding and evaluating matches. These methods can be used within the collaborative environment as an aid to users (e.g. providing a ‘snap to fit’ function and reporting metrics of goodness of fit) and also in a standalone application attempting to find matches autonomously by iterating through a database of scans.

In both cases, the matching algorithm works by iteratively finding the optimal relative orientation of the two fragments under consideration in three-dimensional space. The cost function used for this optimisation relates to the distances between pairs of points uniformly distributed on the surfaces of the join. These distances are calculated using hardware acceleration provided by the computer’s Graphical Processing Unit (GPU). In this way, a potential matching geometry can be evaluated within a few milliseconds for fragment pairs consisting of tens of thousands of vertices and a complete optimisation achieved within a matter of seconds.

A virtual cuneiform tablet under reconstruction by the automated algorithm. The depth map calculation performed by the GPU is illustrated. (left) Depth map of the top face of the first fragment, (centre) depth map of the bottom face of the second fragment, (right) joined fragments and their summed depth map (i.e. the distance between the joined surfaces). The uniform, near zero, depth values of the summed depth map indicates a good quality match.

The first fragmented cuneiform tablet to be automatically joined. The fragments, W 18349 and Wy 777, are part of a letter written in the Neo-Babylonian empire (626-539 BC, around the time of King Nebuchadnezzar) and were found in the ancient city of Uruk (near modern-day Basra, Iraq). The fragments were scanned at the Heidelberg Graduate School of Mathematical and Computational Methods for the Sciences at the Interdisciplinary Center for Scientific Computing, Heidelberg University and are published with the kind permission of the German Archaeological Institute, Berlin.

Automated Ranking of Matches

All joins discovered by an automated system must be validated by a human user. A common problem with similar automated matching algorithms for related applications in the past has been the large number of false positive matches generated. In this work, we have focused on both reducing the number of false positives and on attempting to prioritise the matches that are found so that users will be presented with the most significant and promising candidates first.

Metrics used to rank the matches include the cost function used by the iterative algorithm (the closeness of fit), the degree of interlocking (the tightness of fit), the area of the joint surfaces and the surface continuity. Initial experiments have shown some of these metrics to be more useful than others but all are still under consideration pending a larger study with a larger database of fragment scans.

Assessing tightness of fit.The matched fragment pair is translated by a small sideways displacement. The resulting vertical offset required to prevent intersection of the fragment volumes gives a measure of interlocking or tightness of fit. In case (i), the vertical offset is large indicating a tighter fit than in case (ii).

Assessing significance of match by joint area.In order to prioritise matches with equally well fitting joins, the size of the joint area is a useful metric. In case (i), a relatively small number of distance samples are less than the threshold, ε, suggesting a less significant join than case (ii).

Assessing likely significance of a match by the surface continuity.In most cases, the inscribed surfaces of the completed table are roughly continuous as in case (iii). Cases (i) and (ii) illustrate cases where either the surface or the surface gradient exhibits a discontinuity suggesting that these may not be genuine matches.

People

Dr Eugene Ch’ng (Principal Investigator)

Dr Eugene Ch’ng managed the research as Innovations Director at the IBM Visual and Spatial Technology Centre and the Heritage and Cultural Learning Hub, the University of Birmingham. He is also a visiting professor at the Centre for Creative Content and Digital Innovation, University of Malaya. Dr. Ch’ng has formal education in a wide variety of fields (Fine Arts, Graphic Design, Interior Architecture, Computer Science and, Electronic, Electrical and Computer Engineering).

His research has an overarching theme in Complex Systems Science research for studying and reconstructing physical, social and virtual landscapes. He specialises in advanced interactive Systems, enhanced Virtual Environments, Agent-based Modelling and Multi-Agent Systems requiring large computing clusters for processing of agent-interaction and computer graphics rendering. The fusion of 3D visualisation and complex systems modelling is a unique strength that is applicable to a wide variety of disciplines. His research focuses on a number of complex and interactive systems related to the reconstruction of terrestrial, social, political and virtual landscapes. He also applies cutting-edge technology for facilitating user experience and learning using emerging hardware and information computation in digital heritage.

Dr Ch’ng has over 40 peer-reviewed scholarly publications and is actively involved in editorial boards, technical and program committees in international journals and conferences in his field. Dr Ch’ng is a member of the IEEE Computer Society.

Dr Sandra I Woolley (Co-Investigator)

Sandra Woolley is a senior member of the Institute of Electrical and Electronics Engineers (IEEE) and lectures at the School of Electronic, Electrical and Systems Engineering at The University of Birmingham, U.K. She trained as a graduate engineer with Lucas Aerospace, U.K. and received a PhD degree in electronic engineering from The University of Manchester before working as a researcher at the National Institute of Standards and Technology (NIST), Maryland, U.S.A. She has lectured students in a variety of subjects, including microcontrollers, multimedia and computer networking.

She has previously researched digital imaging and data storage, and her current research interests include aspects of e-health and, in particular, new applications in rehabilitation and assistive technology. She also works on multidisciplinary research in cultural heritage and has enjoyed contributing to two Leverhulme funded projects related to the digital heritage of cuneiform tablets.

Dr Erlend Gehlken (Co-Investigator)

Dr Erlend Gehlken is a physicist and Assyriologist. Erlend studied at the University of Heidelberg where he received a diploma in theoretical physics and a PhD. in Assyriology. In 2003, Erlend achieved Habilitation at the University of Marburg, and has since lectured in Assyriology at the Universities of Heidelberg, Frankfurt, Marburg, and Birmingham (UK). Erlend is currently associated with the University of Frankfurt/Main, and is editor of a Babylonian temple archive from the ancient city of Uruk.

Dr Tim Collins (Collaborator)

Tim Collins is an academic from the School of Electronic, Electrical and Systems Engineering at The University of Birmingham. His PhD was in the field of active sonar and he supervises research in applied signal processing for applications such as spatial audio, underwater acoustics and image processing. His interest in the project relates to the signal processing aspects of automated 3D cuneiform reconstruction. This processing has made use of hardware acceleration via the OpenGL graphics library as well as parallel processing with general-purpose GPU algorithms in order to manipulate and match the scanned 3D fragments.

Andrew Lewis (PhD researcher)

Andrew Lewis is a Leverhulme Trust funded PhD researcher in the The Heritage and Cultural Learning Hub (do.collaboration) at the University of Birmingham. His work explores reconstruction of cuneiform tablets in physical and virtual environments. With a background in web design, coding, and journalism, Andrew's skill base also includes 3D scanning, additive manufacturing, and mechatronics. His work has been featured internationally in publications as diverse as O'Reilly Make magazine and The Guardian and Observer.

Luis Hernandez-Munoz (Research Assistant)

Luis Hernandez-Munoz obtained his PhD from the School of Electronic, Electric and Systems Engineering (EESE) at the University of Birmingham, UK. Luis joined the multidisciplinary Leverhulme Trust funded project team to assist with experimental research related to the digital reconstruction of cuneiform tablets. His PhD research was related to pervasive computing for healthcare in support of people suffering from anaphylactic allergies. Luis also contributed as a research fellow to BraveHealth and the Next-TELL European projects. He obtained a 5-year degree in Electrical and Electronic Engineering with honours at the National Autonomous University of Mexico (UNAM) and he has worked in industry as a certified metrologist. His current research involves software design and programming for mobile devices, human-centred design, usability, effectiveness and acceptance studies, and the design and development of embedded systems.

Publications

Major publications

  • E. Ch’ng, S. I. Woolley, E. Gehlken, A. Lewis, L. H. Munoz, and T. Collins, “A Collaborative Environment for Assisted 3D Reconstruction of Cuneiform Tablets,” in 20th International Conference on Virtual Systems and Multimedia (VSMM), 2014.
  • T. Collins, E. Ch’ng, E. Gehlken, S. I. Woolley, A. Lewis and L. H. Munoz, “Computer-Assisted Reconstruction of Virtual Fragmented Cuneiform Tablets,” in 20th International Conference on Virtual Systems and Multimedia (VSMM), 2014.
  • E. Gehlken, S. I. Woolley, T. Collins, and E. Ch’ng, “Automated Joining of Cuneiform Tablet Fragments,” Nouvelles Assyriologiques Brèves et Utilitaires [NABU], awaiting publication, 2014.
  • T. Collins, S. I. Woolley, E. Ch’ng, E. Gehlken, A. Lewis and L. Hernandez Munoz, Automated Reconstruction of Virtual Fragmented CuneiformTablets
  • A. Lewis, S. Woolley, E. Ch'ng, E. Gehlken, “Observed Methods of Cuneiform Tablet Reconstruction in Virtual and Real World Environments”, Journal of Archaeological Science, (in press)
  • A. Lewis and E. Ch’ng, “A Photogrammetric Analysis of Cuneiform Tablets for the Purpose of Digital Reconstruction.” International Journal of Heritage in the Digital Era, vol. 1, no. 1, pp. 49–53, 2012
  • E. Ch’ng, A. Lewis, R. E. Gehlken, and S. I. Woolley, “A Theoretical Framework for Stigmergetic Reconstruction of Ancient Text,” in Visual Heritage in the Digital Age. Springer, 2013, pp. 43–65.

Other publications

  • T. N. Arvanitis, T. R. Davis, A. Livingstone, J. Pinilla-Dutoit, and S. I.Woolley, “The Digital Classification of Ancient Near Eastern Cuneiform Data.” British Archaeological Review, BAR International Series, p. 1075, 2002.
  • S. I. Woolley, T. R. Davis, N. J. Flowers, J. Pinilla-Dutoit, A. Livingstone, and T. N. Arvanitis, “Communicating cuneiform: The Evolution of a Multimedia Cuneiform Database,” The Journal of Visible Language, Special Edition on Research in Communication Design, vol. 36, no. 3, pp. 308–324, 2002.
  • S. I. Woolley, N. J. Flowers, T. N. Arvanitis, A. Livingstone, T. R. Davis, and J. Ellison, “3D Capture, Representation and Manipulation of Cuneiform Tablets,” Proc SPIE (Three Dimensional Image Capture and Applications IV), vol. 4298 (0277), pp. 103–110, 2001.

Acknowledgements

This research has been supported by a research grant from The Leverhulme Trust (F000 94 BP). The multidisciplinary support of the University of Birmingham’s Digital Humanities Hub is also gratefully acknowledged.

For data acquisition with the 3D scanner of the Heidelberg Graduate School of Mathematical and Computational Methods for the Sciences at the Interdisciplinary Center for Scientific Computing, Heidelberg University, we would like to thank Sonja Speck, Hubert Mara and Susanne Krömker.

The scanned tablets are published with the kind permission of the German Archaeological Institute, Berlin.

The pre-processing computations described in this paper were performed using the University of Birmingham’s BlueBEAR HPC service, which provides a High Performance Computing service to the University’s research community.

For assistance and the firing of the laboratory fabricated test tablets, we would like to thank The Potters Barn, Sandbach, Cheshire, UK.