Managing Data Quality. Tim King

the individual steps to be undertaken and, importantly, should ensure that the end-to-end process is effective in delivering the desired outcomes.

      Data, which include inputs to and outputs from the process, and flows through it.

      Software and hardware systems, which automate the process by storing and manipulating the data, although not every process will be automated by software.

      People, who are the ‘actors’ in the process, undertaking key process steps and ensuring suitable organisational outcomes.

      Despite data being a key enabler for any process, in many organisations there is a greater management focus on the technology elements, particularly when undertaking business change projects involving software. The software product is likely to be expensive, have a recognised name and be a core part of the project, therefore leading to much attention.

      In typical situations, however, the data that will be used to enable the technology to deliver the required outcomes are the data in one or more existing software systems. These data will need to be migrated to the new software tool, but the data migration process is typically a high-risk part of the overall project and, if not undertaken correctly, will actually degrade the quality of the data.

      If the quality of existing data is perceived to be poor then no matter how good a new software tool is, and how well it has been implemented, the outcomes of the system will be limited by the quality of the data. This poor quality data can mean that data migration is far more challenging and expensive, and may not even be feasible at all.

      Figure 1.1 The components of a business activity

      Managing Data Quality

      6

      Also, don’t forget that at some point in the future the ‘new’ software tool will be replaced by another tool. Where will the data come from for this even newer software? Well, it will be the data that you currently have (which in turn has been migrated from several different previous systems). This means that out of the four components of the business activity, the one that lasts the longest and will have a massive effect on outputs is the data.

      Data are an asset

      Data are being created at a faster rate than ever before (however conservatively you forecast future data growth) and data are now more important than they have ever been. As the world becomes a more data-driven place, smart businesses can gain competitive advantages by exploiting data more effectively. This vast data explosion brings newer, different challenges to businesses; it is one thing to store lots of data, but the benefits will only come if the data are of suitable quality and reach the right people at the right time in order to deliver better organisational outcomes. A mindset of treating data as an asset will help your organisation to achieve this.

      An asset is a resource with value that can deliver benefit to an organisation. Data, therefore, warrant being treated in the same way as a physical asset. Like physical assets, data:

      can have high value for your organisation;

      can be assessed for quality;

      can drive up business performance and safety by enabling better informed decisions;

      have legal or regulatory requirements to be managed effectively;

      have a life cycle – from conception, to capture, to operation and renewal;

      can increase business costs if not managed effectively (and therefore reduce efficiency and profitability).

      Many larger organisations, such as those in the utilities and transport sectors, are developing management systems that provide more effective and sustainable management of their assets and activities. Managing data requires a similar mindset.

      We have come across instances where an organisation has been using a spreadsheet-based performance dashboard. Concerns about the quality and integrity of the outputs from this triggered these organisations to spend significant money implementing a ‘best of breed’ analysis and dashboard tool to deliver performance dashboards. However, the data sources were not changed and thus, although the outputs looked far more impressive, the data quality was the same, leading to a false perception of the reliability of the performance indicators shown by the dashboard.

      The data asset

      7

      Unlike physical assets, data support strategic decision making; get this wrong and you will end up making incorrect, potentially expensive, decisions that could have long-term impact for the organisation. Also, unlike physical assets, when the data asset is used, it is not degraded, consumed or destroyed; in fact, the more data are used, then arguably the more value they could generate.

      Whatever sector your organisation operates in, there are benefits to be gained from treating data as an asset to your organisation. This means having a more balanced view about the importance of the data that drives organisational decisions and activities alongside the software and applications that use it.

      The phrase ‘treat data like an asset’ is used increasingly frequently. Assets, though, come in many forms. So, what type of asset is enterprise data?

      Some assets can be large, robust, fixed assets that, once built, will exist for centuries, such as a building or castle. Data are not robust like this and perhaps need to be considered more like a sandcastle, where the individual grains of sand represent items of data, and the configuration of sand that makes the sandcastle is the information that has value to the organisation. A sandcastle is fragile and can easily be degraded by wind and waves. Like a sandcastle, data and information are fragile assets that can easily be degraded by people, systems and processes. The reasons why data quality is difficult to manage are explored more in Chapter 2.

      Data risk losing their credibility if their condition is not monitored and the quality understood and nurtured. This might seem difficult to achieve, but we have written this book to show otherwise. In some cases, actual increases in data quality have not been recognised by organisations because outdated myths persist about the quality of data being poor.

      The data life cycle

      Data, like other assets in your organisation, have a life cycle. The benefits of good quality data will be delivered in cycles or distinct phases, from acquiring data all the way through to eventual archive and deletion.

      One view expressed by some data professionals is that data that are used increase in value, whereas data that are not used have little value and tend to degrade over time. This is not, however, always the case, depending on the context that the data relate to; for example, information describing how to decommission and dismantle a power station safely will not be used during the operational life of the power station, but it will be essential information at the end of its life.

      Static data can be critical reference data for other, more fluid, data. For example, the address of a building doesn’t change (unless the postcode is redefined), but is a critical data asset as it links where people work, what is produced at that building and so on. Therefore, if the reference data are not valued, it undermines everything else.

      Managing Data Quality

      8

      There are slightly different life cycles for general data and for documents, where much of the meaning is carried by unstructured content (i.e. free text and images). Other types of data will have variations on these two life cycles.

      The typical life cycle for general data consists of 11 stages, shown in Figure 1.2.

      Figure 1.2 A typical life cycle for general data