Social:National Digital Preservation Program

Short description: Indian digital preservation initiative

Keeping the foresight of rapidly changing technologies and rampant digital obsolescence, in 2008, the R & D in IT Group, Ministry of Electronics and Information Technology, Government of India envisaged to evolve Indian digital preservation initiative. In order to learn from the experience of developed nations, during March 24–25, 2009, an Indo-US Workshop on International Trends in Digital Preservation was organized by C-DAC, Pune with sponsorship from Indo-US Science & Technology Forum, which lead to more constructive developments towards formulation of the national program.

National Study Report on Digital Preservation Requirements of India

During April 2010, Ministry of Electronics and Information Technology, Government of India entrusted the responsibility of preparing National Study Report on Digital Preservation Requirements of India with Human-Centred Design & Computing Group, C-DAC, Pune, which was already active in the thematic area of heritage computing. The objective of this project was to present a comprehensive study of current situation in India versus the international trends of digital preservation along with the recommendations for undertaking the National Digital Preservation Program by involving all stakeholder organizations.

Technical experts from around 24 organizations representing diverse domains such as e-governance, government and state archives, audio, video and film archives, cultural heritage repositories, health, science and education, insurance and banking, law, etc. were included in the national expert group. Major institutions represented in the expert group were Centre for Development of Advance Computing (C-DAC), National Informatics Centre (NIC), Unique Identity Program, National Archives of India, National Film Archive of India, Indira Gandhi National Centre for the Arts, Information and Broadcasting (Doordarshan and All India Radio), National Remote Sensing Center (NRSC) / ISRO, Controller of Certifying Authorities (CCA), National e-Governance Division (NeGD), Life Insurance Corporation, Reserve Bank of India (RBI), National Institute of Oceanography (NIO), Indian Institute of Public Administration, Defense Scientific Information & Documentation Centre (DSIDC) and several other organizations. The expert group members were asked to submit position papers highlighting the short term and long-term plans for digital preservation with respect to their domain. The study report was presented before Government of India in two volumes as under -

• Volume –I Recommendations for National Digital Preservation Program of India
• Volume-II Position Papers by the National Expert Group Members

The report included an overview of international digital preservation projects, study of legal imperatives (Information Technology ACT 2000/2008), study of technical challenges and standards, consolidated recommendations given by the national expert group for the National Digital Preservation Program.

One of the key recommendations given in this report was to harmonize Public Records Act, Right to Information Act, Indian Evidence Act, Copyright Act and other related Acts with the Information Technology Act in order to address the digital preservation needs. The foresight of this recommendation has proved right, as in 2018, the Indian judiciary has initiated the drafting of electronic evidence rules to be introduced under the Indian Evidence Act. In this context, the Joint Committee of High Court Judges visited C-DAC, Pune on 10 March 2018 to examine the technical aspects of the proposed electronic evidence rules in terms of extraction, encryption, preservation, retrieval and authentication of e-evidence in the court of law.

Centre of Excellence for Digital Preservation

As recommended in the national study report, during April 2011, Centre of Excellence for Digital Preservation was launched as the flagship project under the National Digital Preservation Program, funded by Ministry of Electronics and Information Technology, Government of India. The project was awarded to Human-Centred Design & Computing Group, C-DAC Pune, India. The objectives of Centre of Excellence were as under:

• Conduct research and development in digital preservation to produce the required tools, technologies, guidelines and best practices.
• Develop the pilot digital preservation repositories and provide help in nurturing the network of Trustworthy Digital Repositories (National Digital Preservation Infrastructure) as a long-term goal
• Define the digital preservation standards by involving the experts from stakeholder organizations, consolidate and disseminate the digital preservation best practices generated through various projects under National Digital Preservation Program, being the nodal point for pan-India digital preservation initiatives.
• Provide inputs to Ministry of Electronics & Information Technology in the formation of National Digital Preservation Policy
• Spread awareness about the potential threats and risks due to digital obsolescence and the digital preservation best practices.

The major outcomes of this project are briefly summarised hereafter.

Digital Preservation Standard and Guidelines

Digital preservation standard and guidelines are developed in order to help local data intensive projects in preparing for highly demanding standards such as ISO 16363 for Audit and Certification of Trusted Digital Repositories. The standard is duly notified by Ministry of Electronics and Information Technology, Government of India Vide Notification No. 1(2)/2010-EG-II dated December 13, 2013 for all e-governance applications in India.

• e-Governance standard for Preservation Information Documentation (eGOV-PID) of Electronic Records

The eGOV-PID provides standard metadata dictionary and schema for automatically capturing the preservation metadata in terms of cataloging information, enclosure information, provenance information, fixity information, representation information, digital signature information and access rights information immediately after an electronic record is produced by e-governance system. It helps in producing an acceptable Submission Information Package (SIP) for an Open Archival Information System (OAIS) ISO 14721:2012.

• Best practices and guidelines for Production of Preservable e-Records

Best practices and guidelines introduce 5 distinct steps of e-record management namely e-record creation, e-record capturing, e-record keeping, e-record transfer to trusted digital repository and e-record preservation which need to be adopted in all e-governance projects. It also specifies the open source and standard based file formats for the production of e-records. The guidelines incorporate the Electronic Records Management practice as per the ISO/TR 15489-1 and 2 Information and Documentation - Records Management.

Digital Preservation Tools and Solutions

However, it is difficult to implement the digital preservation standard due to unavailability required tools and solutions. Therefore, the standard and guidelines are supported with a variety of digital preservation tools and solutions which can be given to the memory institutions and records creating organizations for long term preservation. The project team at C-DAC Pune has developed a software framework for digital archiving named as DIGITĀLAYA (डिजिटालय in Hindi language) which is customizable for various domains, data types and application contexts such as

• E-records management & archival (a variety of born digital records produced by organizations on day-to-day basis)
• Large volume of e-governance records
• Audiovisual archives
• Digital libraries / document archives

DIGITĀLAYA (डिजिटालय) is designed and developed as per the CCSDS Open Archival Information System (OAIS) Reference Model, ISO 14721: 2012.

A number of digital preservation tools are developed to help in processing the digital data

• e-SANGRAHAN (ई-संग्रहण): E-acquisition tool
• e-RUPĀNTAR (ई-रूपांतर): Pre-archival data processing tool
• DATĀNTAR (डेटांतर): E-records extraction tool
• SUCHI SAMEKAN (सूची समेकन): Metadata importing and aggregation tool
• META-PARIVARTAN (मेटा-परिवर्तन): Any to any metadata conversion tool
• DATA HASTĀNTAR (डेटा-हस्तांतर): Data encryption and transfer tool
• PDF/A converter tool

All the archival systems and digital preservation tools are developed in such a way that they enable in producing evidences / reports as required for the audit and certification of trustworthy digital repositories.

Pilot Digital Repositories

In order to test and demonstrate the effectiveness of digital preservation tools, various pilot digital repositories were developed in collaboration with domain institutions such as Indira Gandhi National Centre for Arts; New Delhi; National Archives of India, New Delhi; Stamps and registration Department, Hyderabad; and e-District. C-DAC Noida developed the pilot digital repository for e-Court in collaboration with district courts of Delhi using e-Goshwara: e-Court Solution.

The pilot digital repositories were selected from different domains with following objectives:

• Understand different data sets in terms of metadata, digital objects, file formats, authenticity, access control and requirements of designated users
• Identify opportunities for development of tools and solutions in order to address domain specific requirements
• Involve the stakeholders in digital preservation process
• Generate proof of concept by deploying the solutions in the domain institutions

ISO 16363 Certified Trusted Digital Repository

As a part of the pilot digital repositories, National Cultural Audiovisual Archive (NCAA) at IGNCA, New Delhi is established using DIGITĀLAYA (डिजिटालय). NCAA manages around 2 Petabytes of rare cultural audiovisual data. During June 2017, Primary Trustworthy Digital Repository Authorization Body (PTAB), UK got accredited by National Accreditation Board for Certification Bodies (NABCB), New Delhi, India. PTAB was involved to audit National Cultural Audiovisual Archive. Both NCAA and C-DAC teams worked together during the audit process. Finally, NCAA has been awarded the certified status as Trusted Digital Repository on 27 November 2017, as per ISO 16363. It happens to be the first Certified Trusted Digital Repository (Certificate No. PTAB-TDRMS 0001) as per ISO 16363 in India and world.

Capacity Building for Audit and Certification

The High Level 3-day Training Course on ISO 16363 for Auditors and Managers of Digital Repositories was conducted during 11–13 January 2017 at India Habitat Centre, New Delhi, India. This training was organized as per the deliverable of Centre of Excellence for Digital Preservation by C-DAC Pune in collaboration with Primary Trustworthy Digital Repository Authorization Body (PTAB), UK.

This initiative was helpful in formally introducing the ISO 16363 and ISO 16919 through the National Accreditation Board for Certification Bodies (NABCB) for the audit and certification of Indian digital repositories. The first batch of potential technical auditors was trained which included 27 Participants from various stakeholder organisations. Apart from this, numerous digital preservation and DIGITĀLAYA (डिजिटालय) training sessions were organised for the staff of NAI, IGNCA and the 21 partner institutions contributing in NCAA project.

Contribution to UNESCO Standard Setting Instrument on Preservation of Digital Heritage

The Principal Investigator of Centre of Excellence for Digital Preservation, Dr. Dinesh Katre represented India in the UNESCO International Experts Consultative Meet on Preservation and Access during June 25–26, 2014 at Warsaw, Poland, which drafted the Standard Setting Instrument for the protection and preservation of the digital heritage. General Conference of UNESCO at its 38th session on 1 and 2 July 2015 unanimously adopted the Recommendation Safeguarding the Memory of the World –Preservation of, Access to, Documentary Heritage in the Digital Era (38 C/Resolutions – Annex V).

Based on the experience gained from this project, Government of India is considering to create a national policy on digital preservation which will be instrumental in establishing national digital preservation infrastructure. The digital preservation initiative stands at the crux where it is crucial to fill up the gap between the Digital India and the challenges posed by rampant technological obsolescence, to make it a truly sustainable vision.

See also

• National Digital Information Infrastructure and Preservation Program (NDIIPP), USA
• Internet Archive, USA
• Wayback Machine
• Internet Memory Foundation
• Digital Curation Centre, UK
• Digital Preservation Coalition (DPC), UK
• Trustworthy Repositories Audit & Certification
• Big Data