Public and private roles in managing data (Part 3: Evolving roles)

Public and private roles in managing data (Part 3: Evolving roles)

Phuah Eng Chye (23 May 2020)

Historically, the private sector limited their role in managing personal data. Firms managed employee and customer data in relation to their business. Concerns were triggered when it was perceived firms used the data to coerce employees or when it resulted in discriminatory or monopolistic practices. But these risks were contained as data collection was limited.

Today, the situation is vastly different in an information society that is overflowing with data. The private sector has led the charge into technological surveillance, identity authentication, big data and AI. Data is collected from an array of communications, recording and sensing devices to build comprehensive databases. In particular, personal data is analysed to support marketing, advertising and transactions, facilitate a shift from purchase to subscription-based models and for managing employees.

Firms have widened their reach and role as they reposition data itself into a business. Data is no longer collected for single use but for general purpose. Every data point along a value chain could reveal a potential opportunity or risk. Companies can also trade data like commodities. AI is used to process the data and strategies are developed to monetise the value of personal data on a scaled basis.

In comparison, governments always had an extensive role in managing and collecting large amounts of data for administrative and enforcement purposes. However, with the exception of the intelligence and enforcement agencies, governments generally do not optimise the use of data largely due to legal, governance and skill constraints. Governments are subject to confidentiality restrictions and store data in silos which hampers data use and quality. As public agencies, they are restrained from selling data. The risk averse environment, combined with the lack of incentives, makes it difficult to harvest the data.

However, rising transparency is interacting with organisational disruption to reshape the public and private roles. Take the way ridesharing[1] has disrupted the taxi business. Ridesharing captures and uses significantly far more data than the traditional taxi operators. It aggregates information from customers, drivers and locations to provide information on availability, fares, routes and travel times. Data transparency has consequential effects on the organisation of the business. Traditional taxi companies are marginalised by the inflow of part-time drivers and personal cars to meet demand. Taxi regulation is undermined by demand-driven fare setting and the loss of capital value of a taxi license. Data gives the ridesharing and other platforms a massive competitive advantage over traditional competitors and the government. They have proven without doubt how data can be used to enhance demand and to organise supply. 

The evolving roles are also partly driven by the changing balance of power over data. In the past, governments held back from intervening as they did not want to impede innovation and growth. As platforms became dominant, private control over information on citizens is starting to exceed that of governments. This increases the power of the private sector and threatens the government’s hegemony over information. Governments are responding by reviewing legislation to impose strict privacy regulation and to curb platform “freedoms” through regulations to impose prohibitions and obligations or that result in legal consequences. In particular, there is growing attention to the competitive and security risks posed by allowing unfettered foreign access to domestic data.

This re-assertion of government authority is complicated by the blurring boundaries between the public and private sectors.  The relationship is becoming increasingly intertwined as the public and private sectors collaborate and play off each other’s strengths. Governments rely on private sector technology and expertise while the private sector relies on governments for contracts and subsidies.

Data-sharing arrangements sit at the center of the private-public relationship because data is useful to multiple parties. For example, the data they collect gives the ridesharing platforms a competitive advantage. But the same data is useful for planning, regulatory oversight, tax and other administrative purposes. Naturally, the ride sharing platforms are reluctant to give up their data. But data is a natural public good. Therefore, confusion can easily arise when various parties such as firms, individuals and governments attempt to assert their rights over the data.

Data regulation is still underdeveloped and is being explored in many countries. What is clear is that, in the future, governments are likely to assert their claims over data in one form or another. Regulators are likely to expand the scope of data submissions while some other cases, arrangements may be made to allow private firms to sell or manage data on behalf of the authorities.

There are many thorny regulatory issues that are likely to emerge. A European Political Strategy Centre note highlights that “rules on the allocation of liability play an important role in stimulating the development of data-driven business…A clear allocation of responsibility in case of damage (for example, for an erroneous use of data, wrong programming or flawed datasets) may help to inject certainty in markets where actors find it difficult to predict the risk entailed by their investment in data-driven activities…rules could follow a simple economic principle: liability should be allocated to the party that is best placed to minimise costs and litigation, to provide compensation and to ensure product safety. This would be the actor in the value chain – be that the user, the manufacturer of the machine or the programmer – that can most easily anticipate and manage the risk of failure in the data value creation process”.

The future role of the private sector will thus be shaped by regulatory reforms. In light of rising transparency and changing organisational structures, there is a need to clarify the role of technology firms and corporations. New regulations are needed to define to scope the legitimate uses of data, the context of data protection and usage and the obligations of firms deemed as data fiduciaries. The regulations should preferably be generic (principle-based) so that it is sufficiently flexible to cater to different zones[2] where some would be heavily regulated while, in others, it should be left to the private sector to find their own solutions. Protocols on connectivity and access can be set to govern the transfer of data among different zones.

The key aspect of the evolution is the remaking of the government role. It is time to end the ambivalence on the government’s role in managing data. While there is concern that governments could misuse personal data, the reality is that the threat of misuse is always there and won’t be stopped by privacy regulation. The real problem is that government’s use of data tends to be focused on achieving administration and enforcement goals. In this context, governments have generally been rather timid in relation to how they have approached the management of data. Reflecting this, government organisations have not undergone the type of information-induced restructuring that has occurred in private industries.

Massive organisational changes are thus pending before governments will be able to realise their potential in managing data for the public good. Governments need to be more proactive in raising the levels of transparency, widening the distribution of information distribution, enhancing governance and facilitating community self-organisation. This requires by a shift from centralised administration towards platforms and distributed processes. Centralised functions can be replaced with tools to assist communities to self-organise to achieve social objectives. Such a reorganisation will have large spill-over effects on tax and fiscal expenditures. Nonetheless, these reforms are critical as the traditional support systems are already breaking down.

Overall, the future role of governments should be shaped by a bold vision of how it could manage data in an information society while the private sector role would be redefined by how it could pursue opportunities in a changing regulatory landscape. A key aspect of the transition would be transforming the government of the people into becoming the government of the data.


European Political Strategy Centre (11 January 2017) “Enter the data economy: EU policies for a thriving data ecosystem”. EPSC Strategic Notes Issue 21.

Phuah Eng Chye (21 October 2017) “The sharing economy: Disruptive effects of ridesharing”.

Phuah Eng Chye (12 October 2019) “Information and organisation: Shades of surveillance”.

Phuah Eng Chye (26 October 2019) “Information and organisation: Cross border data flows and spying”.

Phuah Eng Chye (21 December 2019) “The debate on regulating surveillance”.

Phuah Eng Chye (4 January 2020) “The economics and regulation of privacy”.

Phuah Eng Chye (18 January 2020) “Big data and the future for privacy”.

Phuah Eng Chye (15 February 2020) “The costs of privacy regulation”

Phuah Eng Chye (29 February 2020) “The journey from privacy to transparency (and back again)”.

Phuah Eng Chye (14 March 2020) “Features of transparency”.

Phuah Eng Chye (28 March 2020) “The transparency paradigm”.

Phuah Eng Chye (11 April 2020) “Anonymity, opacity and zones”.

Phuah Eng Chye (25 April 2020) “Public and private roles in managing data (Part 1: Surveillance)”.

Phuah Eng Chye (9 May 2020) “Public and private roles in managing data (Part 2: Data sharing)”.

[1] “The sharing economy: Disruptive effects of ridesharing”.

[2] See “Anonymity, opacity and zones”.