Reimagining public sector audit information

The story behind the development of the Auditor-General’s Report Information graph, my end-of-career project. 
KK Aw (hires)_2

Many people want to give something back to society at the end of their career. Rich folks may do philanthropic work through foundations and charities. Some take on difficult causes like fighting corruption, eradication of diseases, alleviation of poverty or addressing the other millennium development goals. Others become volunteers in various movements. We should laud these efforts as they address the gaps in the official programs.

There are many professionals at the retirement age, who may not be wealthy, but have a wealth of knowledge and expertise accumulated over the years. It would be beneficial if they can put their know-how to good use in addressing societal problems. This was the situation I faced a few years ago and my decision to do an end-of-career project.

My extraordinary background

My life journey has provided rich experiences across a variety of industries and activities. I started in the mining industry as an intern, operator, project engineer, project manager, and dredge designer.  As a project engineer, I did procurement for a World Bank financed project. When the tin mining industry of Malaysia collapsed in the mid -80s, I ventured into other industries and found myself in the world of computing. I developed a rail track rehabilitation program with a railway engineering consultant used in many KTM projects. I also developed a dimension-database system for a multinational energy cable manufacturer and the digital mapping system for the water reticulation network of a State water board.

In 1995, I became interested in networked information where everything can be related or interconnected. This is a better representation of the real world and is now the domain of graph computing and databases, terms popularized in the mid-2000s. Facebook and LinkedIn use social graphs. Google search uses their knowledge graphs. Amazon web services are based on graph databases. Applications developed using graph computing offers many possibilities and flexibility not available in other platforms. My application, based on graph computing, was the PIKOM-Computimes IT Product (software application ) for the year 1998. We also won the Best Booth Award of the “Innovation Center” in ITX 98. This application has since been developed into an information management framework with many novel features.

This information framework was implemented in an international plant genetics research center in Serdang.   It was also used for managing the flight safety audits of an airline, the legal opinions in a government agency,  a water information repository and an integrated water resource management system. I developed the ” issues based information system ” on top of this framework for a project to study the impact of climate change on water – related industries.

Each of these assignments was a learning experience where I gained important knowledge and skills. I have evolved into a specialist in managing strategic information from my mechanical engineering background.  

Identification of project

I needed to identify a workable project that will draw on my abilities while problematic for others. The project must address pressing societal issues for it to be worthwhile as an end-of-career project.  A friend mentioned in passing I should try to manage the Auditor General’s Report. When I looked into this, I noticed the wealth of information on the public sector captured in the Report. However, the information is spread over hundreds of documents and it is published for tabling in parliament.  I also noted that this information was not published in ways that would allow analysis of the information. That was in October 2013 and the start of my journey in developing my end-of-career project I called the “AG’s Report information graph ” (InfoGraph).

The Auditor General, assisted by over 1900 officers, has responsibility for the audit of about 1400 government departments and agencies, around 1700 government-linked companies and several hundreds of Federal and State projects. The Federal Constitution requires the AG to publish an annual report for tabling in Parliament and the State Assemblies. Each annual report has over 100 documents and about 200 to 250 activity audits per year.

The motto of the National Audit Department (NAD) is “ Quality Audits Enhance Accountability”. While the quality of the audits is important, achieving adequate accountability requires the active participation of the many stakeholders who make use of audit information.  The problem is that audit reports and related information are not published in a way that allows for the systematic use of this information to identify systemic and endemic challenges facing all levels of governments. The PAC ( Public Accounts Committee) and the mass media, so often, highlight the sensational audit findings whilst ignoring more deep rooted and systemic challenges associated with public accountability.  

Aims of InfographThe Aims of InfoGraph

When I set out to address this challenge, I wanted to develop an approach of drawing upon my career knowledge and know-how that would result in the AG’s Report and related data and information become more accessible, integrated, usable. My overall aspiration was to organize and publish audit information in new ways to help users make sense of this information and thus exercise their rights and responsibilities as citizens and public officials. I wanted this to become an initiative that stimulates a range of stakeholders to think about new types of accountability frameworks.  

Open Government Partnership (OGP)

Unknown to me at the time, my initiative is closely aligned to the objectives of what is called the Open Government Partnership (OGP); this project can be viewed as an open government partnership initiative.

OGP is a concept to promote transparency, empower citizens, fight corruption and harness new technologies to strengthen governance. “To harness new technologies to strengthen governance” is what I am trying to do.

The technologies used in InfoGraph include but not limited to integrating Internet and web technology, eXtensible Markup Language (XML) technology, electronic forms, graph computing, named entities extraction, image processing, list processing, and PDF technology. These technologies and features are part of our information framework.  

We developed this system without following the conventional rules in application development as we did not prepare detailed requirements of the system. Instead, we started with an overall aim and look for the opportunities to add value to the system. We build a prototype, populated the system with a few years of the annual AG ’s Report and subsequently let the public test the system. The system menu and facilities evolved as we learned more about public sector audits and received users feedbacks. This is an iterative process that Google Search and Facebook are also using.

Government procurement rules are unlikely to allow applications to be developed in this manner. The applications developed this way have an important common characteristic – they are owned by the developers and not by the clients. For this reason, this project needs to be an OGP initiative.

Technical challenges

Accessibility of audit information

The audit information on the same organization or audits on the same topics may be spread over several documents. In addition, some of these documents are huge and contain multiple audit results. It can be time-consuming to collate the information required for a research or analysis project.

The first step to making this more accessible is to decompose the documents (in PDF format) into smaller units of information such as chapters and sections. We also need to disaggregate useful resources like tables, charts, photographs, maps and so on.

With modern PDF tools, we can do this manually. For our purposes, we are in a production environment and we need to automate the processes as far as possible to increase productivity and reduce errors. The decomposed documents and disaggregated resources have to be named systematically and filed in the designated folders. We developed our own utility to facilitate the processes.

Integrated audit information

For citizens as users to make sense of any audit result, the audit information must be published in ways that allow for visibility and online navigation between a vast network of the public sector entities.

This involves creating linkages with both internally and with external information about the public sector and government-linked companies.

To identify the information to be integrated, we prepared a content model of the system.

AGR Content Model - 1The Content Model of InfoGraph

The audit results can be organized to the common metadata like the year, organization and so on. For further integration, we categorizing the audit results to the audit areas or topics and audit issues. The NAD does not have official lists of the audit areas and audit issues. As the two lists are indispensable in connecting the audit results, we have to glean them from the annual audit reports. We consider the two lists to be work-in-progress that will need to be extended and updated from time to time. At the time of writing this article, both lists have over 100 items.

To read the audit information and identify the audit areas and issues would be very tedious and error prone. It will also need high-quality content officers to process the documents. In recent years, research on “named entities extraction” are undertaken to help make sense of unstructured information. There are web services like http://www. by Thomson Reuters that can be used for extracting named entities. This service is not suitable for our purposes as the entities are US oriented, and the service is designed for news items.

We developed a utility to provide our own “named entities extraction” services. This utility has to be trained for it to work effectively.

Usable audit information

The litmus test for the project is usability. I will show the usefulness of the system through three usage scenarios.

Scenario 1 – Audit information of an organization

InfoGraph allows the user to find an organization using text search or by browsing the list of public entities. When viewing the organization information, the list of subsidiary organizations is also shown. The list of audit results for each organization inclusive of related audit results are also made available.  Without InfoGraph, finding this information would be excessively time consuming.

Scenario 2 – Government procurement

Procurement is one of the critical issues in the public sector where fraud and corruption can occur. The  NAD conducts special procurement audits. InfoGraph, provides the user an easy to see a list of these audits. By default, we group these procurement audits to the audit areas.

Procurement is further categorized into sub-issues. We can also cross-tabulate the procurement sub-issues against the audit areas. This summarizes the procurement sub-issues in the public sector where they are occurring, and the frequencies.

Scenario 3 – Government linked companies (GLC)

It is difficult to find information about the estimated 1700 government linked companies. From the 4+ years of the annual AG ’s Report processed, a list of 119 Federal government linked companies and 250 State government subsidiaries, has been compiled as part of InfoGraph.  For each of the GLCs listed, the number of audits is displayed and the user can access them through this list.

The user can also view a list of the related audits and cross-tabulate them against various issues.

List processing and cross-tabulation

The list processing and cross-tabulation facilities are novel features of the InfoGraph solution. Users can group and filter any list using the relationships captured in the system. The cross-tabulation facility introduced in InfoGraph adds another dimension in interactive information processing.

Towards accountability in the public sector

Accountability stakeholdersStakeholders and Mechanisms for Accountability

“Accountability is a golden concept that no one can be against.” In reality, the people involved will resist and from the recurrent issues raised by the NAD, our public sector is not adequately accountable.  Quality audits is important but the audit information must be leveraged by the stakeholders with the mechanisms available.

Making the audit information more accessible, integrated and usable could be the catalyst for these efforts to promote accountability. However, the big challenge is how to educate and activate the stakeholders so they can develop shared aspirations regarding accountability.

Current status of project

The beta test version of the system went online on 1st Nov 2015. Since then we have populated the system with four years of the annual AG ’s Report. The system is accessible to registered users and registration is free. To encourage registration, we give new registered users a few days of free full access and after which, they cannot download the PDF files. Paid subscribers will have full access.

The system has received over 30,000 page views (c. 25/03/2017) with many returning users.  There are about 250 registered users. Based on the page views, each of these registered users is averaging over 100 page views each. We achieved this by promotion through the social media and mailing list but with no paid advertisement. We considered this as our proof of concept.

Going forward

The system will be launch when we have at least five years of annual AG’s Report. We have to resolve issues in handling tables, charts, and photographs. Before the first release, we want to implement and test all the intended primary features.  When we solicit for subscribers, the system must be stable and the subscribers get value for their money.

So far, we have financed this project with internal funds by operating with a minimal crew for the development activities. We now need to move into production mode where we can process more of the existing annual AG’s Reports and be able to respond quickly and accurately to newly published reports and queries. This would require a team of about 10 staffs and will require monetary support. We would be thankful for sponsorships, grants, gifts or donations from individuals, foundations or corporations.

The next key activity is to carry out the outreach program to promote the system to the various target groups. We welcome civil society organizations and individuals with expertise in this area to join us in this effort.

April 2017


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s