Skip to main content
SearchLoginLogin or Signup

Series as the Starting Point: Unveiling the Potential of Series in Digital Preservation

Published onSep 05, 2024
Series as the Starting Point: Unveiling the Potential of Series in Digital Preservation
·

Abstract – To tackle the challenges faced by the expanding digitization of the government, the document management cluster of The Agency for Facility Operations of the Flemish Government, uses the concept of series: “a grouping of the same kind of administrative information created or received by the same governing body within the same process”. Because of its characteristics a series can play an important role in the world of digital preservation.

A series is used to describe information and to provide the relevant context. Additionally, it is used to establish certain management rules on inventory and contextualization, acquisition and disposal, access and consultation, preservation and destruction. The biggest advantage gained from this concept, is that rules on appraisal are not imposed upon file or record level, but upon series level. As a result, information is already given a retention period and appraised at a high level.

This paper starts with an insight into what a series is and how it functions in the Series Register. The paper then elaborates on four different ways in which series can be implemented: in a digital repository, in SharePoint, in a folder structure and finally to raise awareness on preservation in the workplace.

KeywordsSeries, register, metadata, appraisal

This paper was submitted for the iPRES2024 conference on March 17, 2024 and reviewed by Jaye Weatherburn, João Andrade, Bonface Odhiambo and Susanne van den Eijkel. The paper was accepted with reviewer suggestions on May 6, 2024 by co-chairs Heather Moulaison-Sandy (University of Missouri), Jean-Yves Le Meur (CERN) and Julie M. Birkholz (Ghent University & KBR) on behalf of the iPRES2024 Program Committee.

Regulatory framework

The definition of the concept ‘series’

In the Decree on Archives of July 9th, 2010 the concept ‘series’ was introduced under the name ‘Categories of government records’ [1]. The explanatory memorandum explaining the content of the decree (Flemish law) described that the uniform way of governance and preservation was the unifying characteristic of a category of administrative documents. In 2018 the Flemish legislator decided to simplify, optimize and align the Decree on Archives with the entire Flemish administrative law which resulted in the Decree on Governance of December 7th, 2018 [2]. This decree brought about the harmonization of several different existing regulations (e.g.: freedom of information, reusage of information…). In comparison to the old Decree on Archives this new one made a solid link between a series and the concept of categories of administrative documents. The explanatory memorandum explaining the content of the decree included the following: “A category of administrative documents, also known as a ‘series’ or ‘sequence’, has to be interpreted as a grouping of the same kind of administrative information created or received by the same governing body within the same process [3].

Figure 1 - Series - file/folder - item(s) hierarchy [4]

Each series consists of at least one file. Each file in turn contains one or more documents. (figure 1) [4] Examples of categories of administrative documents are subsidy application files for the improvement of reading skills, environmental permit request files, environmental impact reports, complaint handling files”.

The central register

Both the Decree on Archives of July 9th, 2010 and the Decree on Governance of December 7th, 2018 required that all governing bodies, as mentioned in article III. 79 of the Decree on Governance, have to publicize all their categories of administrative documents (i.e. series). This publication has to be made in the central register which is owned and managed by the Flemish Government, more specifically by The Agency for Facility Operations (since 2019). The regulator has the following objectives with this central register:

  • Develop it to be a central system disclosing all information from the Flemish Government.

  • Stimulate accessibility to government information in order that the public will gain more insight into the information owned by the government.

  • Publicize approved series.

The committees on selection

Since the Decree on Archives of July 9th, 2010 the authority to evaluate and approve series resides in one of the committees on selection. Each level of government has its own committee on selection. (e.g.: Flemish Government, municipalities, provinces…). Working on the level of a committee on selection has the advantage that expertise is centralized and subjectivity while evaluating series is excluded as much as possible. The choice of one authorized institution offers above all the advantage that government-wide agreements on retention levels can be made and consistency in retention periods can be pursued.

The task of the committees on selection is to establish and / or approve selection rules for all categories of administrative documents (aka series). A selection rule must be drawn up for each series. A distinction must be made between selection rules for series that arise from supportive tasks and management tasks - the so-called general selection rules - and selection rules for series that result from core tasks - the so-called specific selection rules.

These selection rules must contain the following information:

  • a meaningful name and description;

  • a motivated administrative and / or legal retention period;

  • a motivated appraisal choice at the end of the retention period: saving, destroying or all possibilities that lie between them (e.g.: saving a sample of the files within a series).

These selection rules are immediately published in the central register upon approval by the selection committee. 

The Series Register

Context

To comply with the legal obligation to publicize series in a register, the Flemish Government gave The Agency for Facility Operations the task to embed the series register within the services related to Digital Archive Flanders [5]. The Agency then developed a registry that included at least the following high-level functions:

  • Create and submit a series to the authorised committee on selection

  • Review a series

  • Close or follow up a series

  • Copy a published series

The web-based application ‘Serieregister’ went live in January 2020 [6], [7]. Since then, the Series Register has been available to all authorities in Flanders. Due to this, Flanders has a centrally managed application in which all series from all governments in Flanders will eventually be included.

In the beginning, the majority of the series in the register originated from institutions within the Flemish government which had already listed their series in Excel files called ‘information management plans”. The series from these Excel files were migrated to the Series Register.

The largest amount of growth in published series was achieved in 2023 when the majority of the 300 local governments joined. They eagerly used the functionality to copy published series from other local governments. As a result, the total amount of published series doubled to 17.000. This upward trend continues into 2024 with the record number of newly published series each quarter being already broken in the first quarter. (figure 1)

Figure 1 - evolution of the total amount of published series since 2021

Content of the series register

The content of the application is best explained on the basis of the high-level functionalities mentioned earlier.

Creating and submitting a series to the authorised committee on selection

A user can create a series in the Series Register by filling in the mandatory metadata fields. If these are filled in correctly, the user will have the opportunity to send the series to the authorised committee for evaluation.

A series is evaluated by the authorised committee on selection whose members also work in the application [8]. The evaluation process by the committee is built into the application. If the evaluation is positive, the series is published in the Series Register. Once published, each series in the register is given a unique ID. This ID makes it possible to link or refer to a series in other systems and applications.

Publication of a series in the Series Register is needed to be able to execute destruction of its files in a legally correct manner. However, this is only possible if the appraisal metadata prescribes 'destruction'.

Reviewing a series

Every series that is published in the Series Register has a validity period of five years because an organization can undergo many changes within this period. These changes impact the existing processes, as a result of which the series can also change. Therefore, a re-evaluation is required at least once every five years. The user evaluates the metadata of the series and makes changes where necessary. The user then resubmits the series to the authorised committee. When approved the reviewed version of the series is published but the previous expired version is preserved to build a version history of the series.

Closing or following up a series

Processes are sometimes terminated. The Series Register provides the option to close the series that is linked to the terminated process. Closing a series results in an end date being given to the series. This indicates that there are no new files related to the series created after the end date. If a process ends, but is immediately followed by another, it is possible to link the series that result from these processes to each other. The application links these series with a ‘follow-up-relation’ so that a user can understand the full context of the new series.

Copying a published series

Since all governments mentioned in art. III. 79 of the Decree on Governance have to publish all their series, everyone will be able to consult the series of other governments. This only refers to the consultation of metadata of a series and not to the underlying files and documents.

This gives governments the option to copy published series from other governments if there is a similar process and series within their organization. The copying functionality is especially useful for municipalities because the processes, and therefore the series, are mostly identical across all municipalities. There are a lot of smaller municipalities that do not employ an archivist or an information manager who can work in the Series Register full-time. By copying a published series, they only need to check the metadata to make sure it applies to their municipality. Since this is a published series, the metadata has already been approved by the committee when the original series was submitted. It therefore makes it more likely to be approved by the committee when submitting a copied series.

Metadata of a series

The following metadata fields must be entered when creating a series:

  • Name and description

  • Depositing institution. This is the institution creating the series

  • Institution creating the files within the series

  • The process and / or type of service which the series is linked to

  • Legal and / administrative retention period. This metadata determines how long the files of the series are preserved after closing

  • Appraisal choice at the end of the retention period

  • Institution responsible for the retention of the files within the series

  • Public access to the files and the documents. This metadata determines whether or not the public can have access to the files of the series

  • Reusage of information. This metadata determines whether or not the public can reuse the information within the files for other purposes than the purpose for which the files were created.

  • Information classification level [9]. This metadata determines whether or not employees within the depositing institution have access to the files of the series and it determines the technical and organisational security measures to be taken.

The following metadata fields are optional when creating a series:

  • Dating of the series. The starting and end date indicate the period during which the files within the series were created

  • Types of personal data that appear in the files of the series

  • Keywords

  • Remarks

Figure 2 - Example of a series in the Series Register. The title appears clearly at the top. Below that, descriptive metadata are listed first, followed by appraisal metadata (not shown in the figure). In the blue left-hand bar, the user has the option to navigate to different lists such as the list of tasks, the list of series in draft and all published series. The plus button at the very top of this bar allows the user to create a new series or upload series in bulk. At the bottom right is a button to export the series in a CSV format which can be viewed in Microsoft Office Excel.

The metadata of a newly published series are displayed on a summary page and can be accessed by any user of the Series Register (figure 2). This is essential as it allows another user to copy this series for their own use.

The published series can also be accessed by the public in the archives portal [10]. Only the metadata on the institution responsible for the retention of the files within the series, the public access to the files and the documents, the reusage of information and the information classification level and types of personal data are not shown in the archives portal.

Using series in a digital repository

In addition to the Series Register, another application was built to store all government data. This digital repository, called the E-depot, went live in March 2020 [11], [12]. It was built as a Software-As-A-Service application (SaaS) for the browser built by an external vendor. Like the series register, the E-depot gets an update twice a year where new functionalities become available for the uses or existing functionalities are modified based on feedback given by the same users in the past.

The core objectives of the E-depot are managing, storing, and accessing digital government information in both the short and long term. Its technological architecture is based on the Open Archival Information System (OAIS) Reference Model [13]. Data can be imported file by file or in bulk. Bulk import can be done via the application itself as a SIP (Submission Information Package), via FTPS ( File Transfer Protocol Server) or via API (Application Programming Interface).

Besides monitoring retention periods and appraisal choices, as well as determining which information is public (more info below), the E-depot does a lot of technical actions that are separate from the Series Register:

  • File format validation

  • Virus scan

  • Generating preservation copies

  • Event logging of all the actions with an impact on the data

  • Extract attachments from e-mails so that the preservation functionalities of the E-depot can also be applied on these attachments

As the OAIS Reference Model prescribes, the data in the E-depot can be accessed via an API in other applications.

There is a clear link between the Series Register and the E-depot. In the E-depot, files and their documents are linked to a series that is published in the Series Register. For each new file, the series it belongs to must be specified at creation. (figure 3)

Figure 3 - A series must be selected when creating a new file in the digital repository E-depot. A window appears where the user has to select the series under which the new file falls. This is a drop-down list which appears in a red colour because this is a mandatory field.

Governing metadata from the chosen series will be inherited by the files as standard. These include:

The retention period chosen at the level of the series determines how long a file must be kept in the E-depot. After this period the appraisal choice is carried out. When choosing at serial level for permanent preservation, the files are permanently stored in the E-depot. In case of a choice for destruction, the files will be destroyed after the end of the administrative or legal retention period and only after approval by the owner of the series. When choosing a sample, a selection is saved on the basis of the sampling rules decided upon at the serial level and the other files are destroyed. Just as with destruction, sampling only occurs after approval from the customer.

As an example, if a file closed in 2010 is uploaded to the E-depot and is linked to a series that prescribes a retention period of 30 years and has the Appraisal choice to be destroyed, this means that the file has a scheduled destruction date in 2040 (2010 + 30). However, if the file has a 'permanent storage' appraisal, the file will not be assigned a scheduled destruction date, since in practice the file will never be destroyed.

Public access to the files and the documents

Based on the choice made here at the serial level, a citizen may or may not have access to the files in the public portal of the E-depot [14]. If it is decided not to make the files public, citizens can always request access, but the government managing the series can fall back on legislation from the Decree on Governance to refuse access to the files.

The information classification level

In contrast to the previous type of metadata, the information classification level has an influence on the right of access for employees of the government bodies which own the series. All employees have the right to consult series with information classification level 1 or 2. In the E-depot this means that they will be able to consult the files and documents of these series. For series with level 3 or 4, employees will only be able to consult the files and documents of the series if they are members of the security group(s) that are linked to such series.

Because a series can fall under information classification level 4, the storage of files in the digital repository has to meet a lot of technical requirements. Some examples are the full encryption of cloud storage, the implementation of privileged access management (PAM) and two-factor authentication [15], [16], [17]

Reusage of information

If metadata has been recorded at the serial level for the reuse of information, this is also passed on to the files and documents in the E-depot. This has an effect on the citizen who consults files in the public portal. If a government has decided to make a series public but prohibits the reuse of the information from the series, this means that a citizen will be able to consult the contents of the files and documents but will not be allowed to process this content in, for example, a news article.

It is important to know that the inheritance of metadata from the serial level to the file level is breakable in the E-depot. With regard to an individual file within a series, it is possible to opt for a longer retention period or to not make the content public.

Using series in SharePoint

Theoretical possibilities in SharePoint

Unlike the relationship between E-depot and the Series Register, there is no direct link between the Series Register and SharePoint, yet series can also play an important role here.

The most basic role would be to organize the structure of your SharePoint sites based on the series and the processes they belong to. As a result, a cluster of libraries will correspond to a process and one library or a cluster of document sets will correspond to one series. The big advantage is that the files in a library are managed in exactly the same way: on the basis of the series metadata.

The following metadata from a series can be implemented in SharePoint:

  • Name and description: the name of the library of document set corresponds with the name of a series.

  • Legal / administrative retention period: this metadata indicates when you can consider a file as closed and when it can therefore be transferred to a digital storage system. Of course, this is only possible if the digital repository can manage and execute appraisal choices.

  • Appraisal at the end of the retention period: if a library or document set based on the series needs to be preserved, you can place a unique URL in SharePoint that refers to the digital repository.

Implementation at the Agency for Home Affairs

In 2017 the Agency for Home Affairs started the project ‘Pimp your SharePoint’ [18]. This project is part of a larger operation to implement the series into daily operations and focuses on four major elements: Managing the storage, archiving and protection of digital information, and raising awareness on good management of digital information

All four elements could only be achieved with the help of the series but there were some roadblocks. The information was very dispersed across different platforms, and there was a lack of structure with information not being archived. Moreover, no appropriate measures were taken for sensitive information. Fortunately, there were series that could serve as a basis to solve the problems:

  • Managing the storage of digital information: SharePoint sites were structured based on the series. This allows files to be placed or created in the corresponding SharePoint site based on their series.

  • Managing the archiving of digital information: A vision was drawn up on the finality of SharePoint sites to clarify when data from a SharePoint site can be archived. In addition, a project methodology was drawn up for project sites. This ensures that archiving takes place at the correct time and makes it also clear under which series the agency, as a user of the E-depot, must archive information.

  • Managing the protection of digital information: if a series has a high information classification level, necessary security measures have been taken in the corresponding SharePoint sites to restrict access to the series' files.

  • Raising awareness on good management of digital information: The agency has a SharePoint site where employees can find information about the use of storage media, the SharePoint sites and all parts thereof, aftercare after the adjustment of a site and the responsibilities, the agreements regarding safety, and finally the further agreements and to-do's.

In order to maintain an overview, the agency's information managers have drawn up a configuration document in which all information under their management is listed. In most cases, this document contains a link to the corresponding series and the management metadata of the series according to which the information must be managed.

Although this project started in 2017, there is still work to be done to tie up the loose ends. Additionally, the multitude of site structures poses a continuous challenge and the lack of integration between SharePoint and other applications remains a barrier.

Using series in a file system

Theoretical possibilities in a file system

As is the case with SharePoint, there is no direct link between the Series Register and a file system such as Windows Explorer, yet series can also play an important role here.

The most basic role would be to organize the structure of your file system folders based on the series and the processes they belong to. Folders at a higher level correspond to a process. Folders at a lower level correspond to a series. The advantage being that the files in a folder are managed in exactly the same way: on the basis of the series metadata.

The following metadata from a series can be implemented in a file system:

  • Name and description: the name of a folder corresponds with the name of a series.

  • Legal and / administrative retention period: this metadata indicates when you can consider a file as closed and when it can therefore be transferred to a digital storage system. Of course, this is only possible if the digital repository can manage and execute appraisal choices.

  • Appraisal at the end of the retention period: if a file based on the series needs to be preserved, you can place a unique URL in in the folder that refers to the digital repository.

  • Information classification level: based on the information classification level, you can limit access to a folder to a number of employees.

Future implementation at the municipality of Wetteren

After creating and publishing the majority of their series, the municipality of Wetteren started to look into new ways to implement the use of series [19]. One new way will be to incorporate them into daily operations. In concrete terms, this means that they will structure their file system according to the series. In this way, one folder will contain files form one series, giving all files the same retention periods and appraisal choice. As a result, they will be able to monitor much more specifically when information should be permanently archived or destroyed.

Using series to raise awareness on preservation

Theoretical possibilities

The implementation of series in a digital repository, SharePoint, or a document system are interesting ways to take document management to a higher level in an organization. However, such implementations are pointless if employees are not involved. It is not the information manager who creates the files and works with them on a daily basis, but the average employee. Employees must therefore have a basic understanding of the series to know how long their files should be kept.

A solution could be to give all employees of the organization access rights to the Series Register, but this application is only aimed at users who create series. Additionally, this would mean that an employee would have to start working with a new application in addition to the many applications they already use.

Fortunately, the Series Register has an API (Application Programming Interface) which enables developers to programmatically interact with the Registers. In this way, other applications can request and use the metadata of series.

A possible application of this is to visualize the most important series’ metadata for an employee, namely the metadata about the retention period and the appraisal choice, in an application that the employee already knows. This way, an employee can quickly find out how long they should keep their files and whether these files can be destroyed after their retention period.

Implementation at De Watergroep

De Watergroep is the largest drinking water company in Flanders. To make their series accessible to all employees, two solutions were used: a theme site on SharePoint and the online mascot 'Spetter' [20].

The theme site consisted of three major elements: the information management policy, an overview of frequently asked questions, and the ability to consult the most important metadata of series. This third element was made possible by the Series Register API [21]. When an employee performs a search, the system searches all series that match that search. (figure 4) The employee can consult the following metadata of a series:

  • Title

  • Retention period

  • Information classification level

  • Appraisal choice with the motivation

Figure 4 - The metadata an employee from De Watergroep can consult. The tittle of the series appears at the very top with below (in order from top to bottom) the retention period, information classification level, appraisal choice and the motivation for the appraisal choice. These metadata can only be consulted.

This metadata provides the employee with sufficient information to manage their files correctly.

In addition to the theme site, employees can also call on the mascot 'Spetter'. (figure 5) This is an interactive searchbot with AI technology. If an employee writes that they are looking for retention periods or appraisal choices for certain files, the searchbot will automatically search the series and show the employee the series that best meets their question. The further development of this searchbot will include the ability to fill in request forms for new series if the series being searched for does not yet exist. In addition, the searchbot will also be used for day-to-day support regarding information management.

Figure 5 - AI Mascot 'Spetter'. Users can ask the chat robot questions about retention periods after which the chat robot displays the most relevant series.

Using series in other environments

the use cases described so far are not comprehensive. Some components of the series register allow series to be used in other applications in the future:

  • A series' unique and persistent ID and URL ensures that any reference to a series from any application will always point to the correct series.

  • The ability to export a series in CSV format lowers the threshold to start reusing the metadata.

  • The Series Register API allows the metadata of series to be sent to and reused in another application after mapping these metadata to data model of that application

Conclusion

A first introduction to the Series Register would make you think that the story ends after the selection committee approves a submitted series and it is published. However, the opposite is true! As the title already revealed, a published series is a tool with which the information manager can get started.

You can think of this a bit as a Swiss army knife with simple and sophisticated elements. The simplest applications concern, for example, reorganizing a document system based on the series or informing employees about the use of series. The more sophisticated applications involve setting up an API integration to display the metadata of series elsewhere or applying the metadata of series in digital repositories to manage the files.

Whichever application of series as a Swiss army knife you choose, it always boils down to the same thing: Using the series to preserve and manage records correctly. By establishing document management at a higher level, you can set management rules for a large volume of files without having to handle each file individually. The implementation in the E-depot also shows that in case of a file-level anomaly, metadata inheritance can be broken. In summary, the series does not have to be sacrosanct, but behaves like a shepherd leading his sheep on the right path.

Comments
0
comment
No comments here
Why not start the discussion?