Investigating private entities with open data

As part of OpenUp's Transparent Corporates project, we've got some tips on how to investigate private entities using open data.

In South Africa, and around the world, the use of open data in criminal investigations is quite new and although data has been around for centuries, even the mere existence of it in its current form is something of an enigma (although this is fast changing). When it comes to the fight against corruption, there is still a big discussion to be had around the use of data, including how to effectively communicate what it is used for.

About a year ago we turned some of our attention towards the private sector, to try and better understand the levels of transparency and accountability surrounding it. The data it produces can be essential in combating corruption around the world - just take the Panama Papers and Gupta Leaks as prime examples of this. But, the corporate sector has until now fallen behind in adopting open principles. Yes, more and more private entities are in fact becoming more transparent, but at the same time, many are not.

One would assume that finding the names of shareholders in a company would be fairly simple, but in South Africa (as in many places around the world), it really isn’t. Sure, when it comes to companies listed on the Johannesburg Stock Exchange, accessing that information is somewhat easier, but it will cost you a small (and ever-increasing) fee. At the same time, beneficial owners are easily able to hide behind nominees and quietly go about their business in the background, evading public inquisition into their activities. Registries generally only collect minimal information on the operations of private companies. Such secrecy allows them to evade government regulation and public scrutiny; it facilitates corruption, tax evasion and other corporate crimes.

But there are ways around this, they just take time and a lot of practice.

When it comes to corporate confidentiality there is no good reason why a corporate hierarchy should not be made public. In fact, refusal to make an entity’s information publicly available disadvantages those companies who want to be good corporate citizens.

So, where do we as journalists and researchers begin?

The best place to start when investigating private entities is to turn to public company records; that is, if a company hasn’t already put the information you’re looking for online. Usually, medium to large corporates will give the public access to information on their directors, shareholders, contact details, registration number and in some cases, company financials. All of this will usually be available in a company’s annual report. Today, it’s relatively easy to access these on a company’s website or by asking for a physical copy of it. Some of the main things to look out for in a company’s annual report include the business description, vision and mission statements, the director’s report, and corporate governance and compliance. These will give you insight into what a company values most and what it believes its failures and successes to be. You can use these to hold a company accountable according to its own standards, and you’ll be able to measure its progression over a few years.

Next, you should look to other types of public records, such as court documents (not yet available online here in South Africa), judgments on the Southern African Legal Information Institute and the Government Gazette. This last one is a treasure trove of information and when it comes to analysing corporate data, you should focus on tender bids and awards, company liquidation notices, liquor licence applications, and business and personal name changes. Often, you might not know exactly what you’re looking for in these public records, so before you start, you should compile a list of prominent names, ID numbers, and other stand-out numbers and amounts, addresses and contact details. This way, making those links at a later stage will be easier to do as you compile only the necessary bits of information.

Next you can and absolutely should do some actual reporting, and speak to several sources. Whether or not you’re a journalist by trade or training, it’s important to remember that in-person interviews are always first prize, but phone calls, emails and Instant Messenger are always options. In some cases, your source may want to remain anonymous and you need to respect that. When it comes to investigating private entities, you may need to turn to sources within the actual company, so before you do, think carefully about what type of questions you would ask them and how you would include that information in your reporting or research.

Of course there are also subscription databases. Here in South Africa, we have a few major ones such as the CIPC, WinDeed and LexisNexis, and there are variations of these all over the world that offer information on private entities. These are often used by journalists, but it takes a lot of practice to know how to conduct a good, efficient search. Listed companies will more than likely subscribe to these by submitting information on themselves as well as using it to source information on their competitors. While these are not necessarily a last resort, this is often low down on the list of tools to investigate private entities. Only pay for information if you cannot find it anywhere else and you absolutely need it.

Finally, there are free searches, which are incredibly useful for finding basic information on private companies, especially if you’re investigating more than one. There’s nothing worse than having to look all over the place for the same type of information; it’s useful to have the same standard of data available for various companies in one place. Open Corporates, Investigative Dashboard, Companies House and Who owns Whom are just a few good examples.

A living, breathing example

For the past few months, we’ve been working in a team of journalists and data analysts to investigate the existence, transnational links, architecture and impact of the global lottery industry. The project was done in partnership with eNCA, the Organized Crime and Corruption Reporting Project, the Columbia Journalism School and PennLive, and culminated in “Gaming the Lottery”, which is the South African aspect of the investigation, which also looked at lotteries in the UK, US, Australia, several other African countries and Switzerland. Our role was to analyse and report on the funding allocated to South African organisations, by the National Lottery Commission (NLC). The first thing we did was scrape 15 years worth of data from their annual reports; this dataset is now available as an interactive visualisation, and we’ve put out a call to the public to have a look and let us know if there’s anything else we should be investigating. Then we analysed the data, and what ensued was several months of looking for information on companies (listed and unlisted) and NGOs linked to the NLC’s funding activity. In one instance, we sought the business ownership history of a man who is at the centre of a very significant funding allocation, despite multiple of his business ventures having failed because of non-compliance. We also investigated the many companies registered to individuals within the sporting leadership community, as a large portion (about R5.7 billion) of the NLC’s funding has gone to sports and recreation.

We used almost all of the resources mentioned above, from the moment we started analysing the data to when it came time to start asking companies and NGOs difficult questions about large, or questionable sums of money. It was an expensive, challenging and important investigation, that is nowhere near from over. In fact, a company linked to the lottery sector has been implicated in tax avoidance and while there is no evidence (yet) to suggest its actions affected its operations in the South African lottery, local officials have said they will review the company’s tax records. This is why in-depth and lengthy investigations of private - and public - entities is so important. Were it not for this type of work, we wouldn’t know about the billions of dollars evaded in taxes by companies and individuals linked to the Panama Papers, nor would we be aware of the extent to which the Gupta family have captured so many aspects of South Africa.

Using these tools and resources, intertwined with the long and experiential process mentioned above, there are numerous civil society organisations contributing to investigations around private entities, here in South Africa. Remember that this type of information can be incredibly difficult to get your hands on, but there are ways around this, they just take time and a lot of practice.

Here is an easy-to-use list of resources for investigating private entities:

  1. Annual reports: Have a look at a company’s own website or ask for a physical copy at their offices.
  2. Court documents: Go to the Southern African Legal Information Institute ( for freely accessible and searchable judgments. Other court documents are not yet available online.
  3. Government Gazette: OpenUp has created a free-to-use and easily searchable database of gazettes (
  4. Companies and Intellectual Property Commission: A subscription-based registry of JSE-listed companies (
  5. WinDeed: Online search tool for personal, company and property information (
  6. LexisNexis: A platform for business research solutions and software for professionals in accounting, law and tax (
  7. Open Corporates: A resource which shares data on corporate entities as open data under the share-alike attribution Open Database Licence (
  8. Investigative Dashboard: A helpline for investigators; provides research assistance and a global database of company information (
  9. Companies House: Free company information, including registered office address, filing history, accounts, annual returns, officers, officers, charges and business activity (

Related links