News

The Different Varieties of Advisors & the Difference it Makes

January 10, 2022 by Robert Grossman

The fox knows many things, but the hedgehog knows one big thing. Archilochus, Greek poet, c. 680 – c. 645 BC

“The fox knows many things, but the hedgehog knows one big thing.” Archilochus, Greek poet, c. 680 – c. 645 BC
Question: “By the way, why did you make this classification [between hedgehogs and foxes]?” Isaiah Berlin: I never meant it very seriously. I meant it as a kind of enjoyable intellectual game, but it was taken seriously. Every classification throws light on something, this one was very simple.

For my January 2022 post, I want to return to return to the theme of last year’s January post about why a board should include someone with a good knowledge of analytic strategy. In this post, we will discuss different types of advisors about analysis and analytic strategy. It’s good to keep in mind the response of Isaiah Berlin points out when asked about his distinction between two types of thinkers (hedgehogs and foxes), “every classification throws light on something, [but] this one [is] very simple [1].”

In the January 2021, post I listed three reasons that a company should consider putting an expert on analytics on its board or one of its advisory boards:

Analytic and AI strategy is too important for a company not to have someone with board level experience in this area.
Board members with experience developing, deploying and operating complex analytic projects have critical experience using technology to innovate, not just operate.
An analytic perspective for a board member helps with evaluating cybersecurity and digital transformation, both critical topics for many boards.

A year later, these three reasons are as important as ever. In this post, I’ll review five different types of advisors in general, and how this applies to advisors familiar with analytic strategy. It is important to note that an individual is often a blend of two or more of these types.

Type 1. The Problem Solver

The first type of advisor is a problem solver, who can listen carefully, think through a problem, challenge or situation thoughtfully and carefully, and then crystallize the essential issues, trade-offs and choices, so others most familiar with the facts can make a better decision. This type of advisor has experience, judgement, and often gravitas, but any particular domain knowledge is usually less important than their overall ability to pinpoint the essential issues and frame the tradeoffs that must be thought through. For simplicity, I call this type of advisor a problem solver, but they usually simply crystallize and frame a problem so others can more easily work out a good solution. This type of advisor does not rely on knowledge of any particular technical domain, and for this reason, can be valuably contribute to technical discussions over a long period of time, even as the underlying technology changes.

Type 2. The Connector

The second type of advisor is the connector who seems to know almost everyone of importance in a particular industry, has a track record of being helpful with introductions, and is happy to connect you as required. There are different types of connectors. Some will do an introduction, which will guarantee at least one (courtesy) meeting. Others will work to introduce two individuals that can be mutually valuable to each other and will continue to connect from time to time even if there is no immediate benefit.

Another type of connector is someone who is a trusted member of a group that has a shared common experience and whose members have gone on to different successes but stayed in touch. Examples of these types of groups include: the PayPal Mafia and the Fairchildren [2] that are the descendants of the Traitorous Eight.

Type 3. The Technical Expert

The third type is the technical expert, who has a deep understanding of a technical field, understands the economics and business value that the technical field, and has broad experience of the common problems and pitfalls and how to avoid them. This type of expert can save an enormous amount of time and costs for a company. Some challenges include that this type of expert may not always be the best at arguing his or her point of view and others, with less knowledge and experience, may be more pervasive. Another problem is that as technology changes and business and competitive environments change, the a particular advisor’s expertise may become less and less relevant.

A great example of an influential advisor who was Type 3 is John Tukey, one of the original data scientists [3], who came up with the idea of the fast Fourier transform (FFT) in a meeting of President Kennedy’s Scientific Advisory Committee in 1963 [4]. The discussion topic was the ability to verify a proposed United States/Soviet Union nuclear test ban. One idea was to analyze seismological signals obtained from off-shore seismometers that could be positioned to verify compliance with a test ban treaty. The seismometers would produce a lot of data and the FFT was one way to speed up the processing of the data. For an interesting history of the origins of the FFT and its acceptance by the scientific community, see [3]. For a broader overview of the FFT and its relevance to seismology, see [5].

Type 4. The Management Expert

The fourth type of advisor is the management expert, who can bring management expertise to a problem or opportunity. This is especially valuable when an outside perspective is needed. Sometimes these advisors may be put on a board because they are particularly familiar with certain management issues, such as corporate strategy, marketing, acquisitions, information strategy, change management, government sales, etc.

An archetype of a Type 4 advisor is Peter Drucker, who coined the term “knowledge worker” in the 1950’s and argued that knowledge workers were an important source of competitive advantage long before computers, information and digital transformation became commonplace [6, 7]. As another example, he argued that “purposeful abandonment” was a critical part of any business strategy. This is the argument that management focus requires a very purposeful pruning of business units and business activities that are no longer worthwhile spending resources to pursue, thus freeing up resources for more strategic activities. Although these observations sound quite familiar now, they were much less familiar when he first argued for them.

Type 5. The Trusted, Independent & Discreet Advisor

The fifth type of advisor is not required, and is not always present, but is the trusted and discreet advisor who is independent and often has a long relationship with those in the company or organization being advised.

An interesting example of the trusted and discreet advisor was Harry Hopkins, who was President Franklin Delano Roosevelt’s advisor and was influential in shaping the relationship between the United States and the United Kingdom in World War II.

Again, it is important to point out that an advisor is often a blend of two or more of these types.

Relevancy

Advisors seem to age at different rates. As long as they remain sharp and their expertise is still relevant to the company, the capabilities provided by the problem solver and the management expert remain relevant and useful. On the other hand, the value of the network of some connectors may become less relevant over time. Other connectors seem to have the almost magical ability to be personally acquainted with exactly the right individuals at the right time.

Finally, there are important differences in technical experts. Some have a deep and profound knowledge of certain technologies, such as data or analytics, and are comfortable using the current technical parlance and buzzwords For example, a technical expert of a certain age will move from framing technical discussions in the language of data mining (1990’s), to predictive analytics (2000’s), to data science (2010’s), to AI (2020’s). Others seem to stick with a particular framing of the problem, and although their advice is probably still relevant, it can seem dated and not relevant.

References

[1] Jahanbegloo, Ramin. 2007. Conversations with Isaiah Berlin. Halban. Kindle Edition. Location 2742.

[2] Laws, David. 2016. Fairchild, Fairchildren, and the Family Tree of Silicon Valley, December 20, 2016, retrieved from computerhistory.org on January 2, 2022.

[3] Donoho, David. 2017. 50 years of data science. Journal of Computational and Graphical Statistics, 26(4), pp.745-766.

[4] Cooley, James W. 1987. How the FFT gained acceptance. In Proceedings of the ACM conference on History of scientific and numeric computation, pp. 97-100.

[5] Rockmore, Daniel N. 2000. The FFT: an algorithm the whole family can use. Computing in Science & Engineering, 2(1), pp.60-64.

[6] Drucker, Peter F. 1959. Landmarks of Tomorrow. Harper, New York.

[7] Drucker, Peter F. 1992. The new society of organizations. Harvard Business Review 70(5) pp. 95-104.

100 Millisecond Auctions, Your Privacy and Regulatory Risk

December 14, 2021 by Robert Grossman

The Regulatory Risks Posed by GDPR When Using Bidstream Data in Real Time Bidding (RTB)

You can download Mobilewalla and Ubermedia's data directories from the evidence we sent the DPC 13+ months ago here https://t.co/3ccOMbNyNC. Note: this is clearly the IAB standard. pic.twitter.com/aSHP3NzQgE
— Johnny Ryan (@johnnyryan) November 18, 2021

Figure 1. A tweet from Johnny Ryan from the Irish Council for Civil Liberties (ICCL) showing some of the data that is available in a TC consent string. The ICCL is involved in litigation under the GDPR with those using bidstream data for targeting.

In real time bidding (RTB), there is a 100 ms auction in which different advertisers bid against each other to place an ad on a webpage or other space offered by a publisher. This is why you sometimes have the experience when browsing that the web jumps a bit as you start to read the page. The jump is because the web page content can shift after the ad loads.

The data exchanged in this auction is called bidstream data and includes location information, information about the application and system you are using, and standardized codes developed by the Interactive Advertising Bureau (IAB). The information about the application and system you are using is enough for online device fingerprinting. The specificity in the IAB codes can be a bit shocking the first time you look at them. Here are some examples of IAB codes from the IAB Taxonomy:

ID 281 Interest | Businesses and Finance | Bankruptcy

There has been concerns about the leakage of privacy information in bidstream data for sometime and the inadequacy of the pop-up consents that are suppose to give users some control.

Providing Consent for Bidstream Tracking Data

For those website that follow the European General Data Protection Regulation (GDPR), users are presented with a pop-up banner that asks for consent for the collection of tracking information. In general, websites use this information for both internal purposes and for targeted advertising, such as supplying information for bidstream auctions.

In 2018, IAM developed a “consent framework” called the Transparency and Consent Framework to standardize how advertisers and publishers could collect and store the required consent information. The information is passed around in a format standardized by the IAB called the TC consent string.

Changes in the Regulatory Framework for Using Bidstream Data

The Irish Council for Civil Liberties (ICCL) has brought a case under the GDPR claiming the the IAB Transparency and Consent Framework and its use with TC String in real time bidding does not provide adequate consent [1]. The actual 174 page court filing [2] provides a very detailed explanation of the bidstream data that is exposed and why it should be considered a data breach under GDPR. For a good overview of the ICCL case and its implications, see [3].

Upcoming Regulatory Changes and Your Analytic Strategy

As a result of the ICCL court case, there maybe changes in the regulations for using bidstream data and one of the challenges from an analytic strategy point of view is to have a plan to update your own strategy in anticipation of future regulatory changes.

Figure 2. The process of how advertisers (the Brand) sell ads to publishers. An advertiser can use a demand side platform to work with different networks of publishers and a publisher can use a supply side platform to work with different networks of advertisers. Advertisers buying space for advertising and publishers selling space for advertising are matched by the ad exchange and use the bidstream data to determine the auction prices. The figure is from Wikipedia (CC-By).

Problems with Bidstream Data

The exchange of bidstream data involves the advertiser, the publisher, the ad exchange, and, usually, a demand side and supply side platform. Advertisers buying space for advertising and publishers selling space for advertising are matched by the ad exchange. An advertiser can use a demand side platform to work with different networks of publishers and a publisher can use a supply side platform to work with different networks of advertisers. See Figure 2. With this many different entities involved in the auction, it is not surprising that there are often problems with bidstream data.

Inaccurate information. Given the amount of information exchanged with RTB and the monetary value involved, there is a significant level of inaccurate information being exchanged and a significant level of activity where the bidstream data is being used for purposes for which it was not intended and for which it was not consented. An example would be surveilling bidstream data and processing it using device fingerprints to create profiles of the users bidding.

Misuse of bidstream data. This is an example of adversarial analytics in the original sense (not the more modern GAN sense) in which there are two parties building analytic models that are at cross purposes. Here the advertisers and published exchanging bidstream information for real time auctions and the adversaries who are monitoring the information to build profiles of users that can be monetized.

Challenges with changing it. Most websites list a long list of advertising partners that they exchange bidstream data with it (often hundreds) and leave it to the user to update or correct the information for each partner. This is almost never practical.

The Emerging Era of Post Third Party Cookies

We are moving to an era in which advertising will not be able to use third party cookies, but rather rely on other types of information for matching advertisers and publishers. From an analytic strategy point of view, it is critical to make sure that your analytic frameworks still provide the value you require when third party cookies are not supported. More generally, platforms will have to update so as not to rely on third party cookies.

Regulatory changes almost always take years, giving your organization time to put in new place new analytic frameworks. The challenge though is usually not a technical one of transitioning to a new regulatory framework but rather figuring out how to extract the necessary business value in the new regulatory framework to provide the foundation for a successful analytic business.

References

[1] Irish Council for Civil Liberties, ICCL lawsuit takes aim at Google, Facebook, Amazon, Twitter and the entire online advertising industry, 15 June 2021, retrieved from ICCL.ie on December 6, 2021.

[2] ICCL complaint filed against IAB Technology Lab, Inc. and others in Hamburg District Court, May 15, 2021 (machine translated from the original German).

[3] Daniel Cooper, How a civil rights group is holding Europe’s online ad industry to account, Engadget, December 3, 2021. Retrieved from Engadget on Dec 14, 2021.

The Economics of Inferencing: Managing the Long Tail of Data

November 15, 2021 by Robert Grossman

The different steps in the inferencing process and the costs involved.

When developing an analytic strategy, it is important to have a good understanding of the economics of inferencing.

Our conceptional model is that we: 1) collect, purchase or otherwise acquire the data, 2) clean, curate and process the data so that it can be used as the inputs to models and to train models, 3) build the model; 4) deploy the model, and 5) operate the model, using the data as model inputs to to produce outputs of interest. See Figure 1.

To simplify the analysis, we will call the outputs of the model “inferences,” and assign a monetary value to each inference.

At the highest level, the economics of inferencing is driven by two equations: The first equation is the value provided by the inferences. This is the simplest equation and since inferences usually have a fixed value or a value determined by the subsequent actions (such as the value of the sale or value of the sale and subsequent sales over the next month). The value is simply the number of inferences x the value of each inference. This the right side of Figure 1.

The second equation is the cost of the inference. This is the sum of five components. See Table 1 and Figure 1. There is the cost to acquire the data, to clean and process the data, build the model, to deploy the model and to operate the model.

The challenge is that the cost to clean and process the data can have a long tail. In other words, to clean 50% of the data is often fairly easy, to clean 75% much hard, to clean 90% much, much harder, and, so, with the costs to clean the last few percent getting higher and higher.

There is an important difference between how for profit-companies offering a product or service and not-for-profit companies that are mission driven can manage the escalating cost associated with the long tail of data curation. For-profit companies can cut the tail at a sensible operating point, while not-for-profit or mission-driven organizations often have to process more of the tail in oder to satisfy their mission. Here are some example of why the long tail of data curation is often important for mission driven organizations:

When cleaning up scientific data for machine learning a high model coverage threshold is important, since an important discovery may be in the long tail
When identifying threats in national defense, a high model coverage threshold is important since missing just one threat can have devastating consequences
When identifying fires and floods, covering the long tail is critical, since early detection of a fire or flood can result in significant savings of property and sometimes savings in lives.

costs of inferencing = 
   cost to acquire the data                    large & variable
   + cost to clean & curate the data           large & variable
   + cost to build the model                   fixed cost
   + cost to deploy the model                  fixed cost
   + cost to operate the model                 variable, but modest
Table 1. Costs of inferencing

For more about the costs of acquiring and curating data, see Chapter 6 of Developing an AI Strategy: a Primer.

Analytic Strategies for Not-for-Profit Organizations: Chasing Down Root Causes

October 11, 2021 by Robert Grossman

Lessons from the Rockefeller Sanitary Commission in 1902

Figure 1. The incident levels of some diseases are concentrated in certain geographic regions. This post is about the incidence of Hookworm diseases in the Southern states in 1902. The figure above shows the incident level of hypertensive heart disease (ICD-9 402) for the period 2003-2010 is higher in certain Southern US states. Source [1].

Peter Drucker’s General Advice About Strategy for a NFP

Peter Drucker wrote in Managing the Nonprofit Organization:

There’s an old saying that good intentions don’t move mountains, bulldozers too. In nonprofit management, the mission and the plan – if that is all there is – are good intentions. Strategies are the bulldozers.

They convert what you want to do into accomplishment. They are particularly important in nonprofit organizations. Strategies lead you to work for results. They convert what you want to do into accomplishment. They also tell you what you need to have by the way of resources and people to get the results.

Peter Drucker, Managing the Nonprofit Organization [2].

Lazy Workers in Southern States – The View in 1902

The quote above from Peter Drucker is inspirational, but not particularly helpful for those broadly familiar with strategy and analytics. In this post, we look at a data driven strategy developed by John D. Rockefeller and those around him to improve living conditions and worker productivity in southern US states at the turn of the 20th Century. We follow [3] and [4].

The story begins with the US Public Health scientist Charles Stiles who identified diseases caused by hookworms (Necator americanus) as prevalent in Southern states, especially on farms and plantations in sandy areas. Hookworms are parasites, and those infected with them were anemic and had difficulty working [3]. To outsiders, workers infected with hookworms appeared to be lazy.

Stiles lectured on his investigations at a 1902 Sanitary Conference meeting in Washington and his findings were reported in newspapers, such as the New York Sun, which used the headline “Germ of Laziness Found [3].”

European physicians by the mid 1870’s linked the parasitic worms to human symptoms of severe anemia, pallor, and weakness [3]. Hookworms could enter through the souls of your feet, infect your body, were passed in your stools, returned to the ground, and could then infect others.

Rockefeller Sanitary Commission

In 1909, Stiles attracted the attention of John D. Rockefeller, who was by then a philanthropist, and Rockefeller provided $1 million (equivalent to about $27 million in 2021) to establish the Rockefeller Sanitary Commission for the Eradication of Hookworm Disease. Rockefeller hired Stiles and project administrators [3].

The conditions in the Southern US were conducive to the spread of hookworm disease. As summarized in [3]: “Primitive living standards increased Black and White farm families’ vulnerability to disease. The risk of exposure to hookworm infection was great as well; John A. Ferrell noted that in the Southern rural sections, ‘open privies and, far too often, no privies at all, are used, [so that] millions upon millions of [hookworm] eggs are scattered over the earth, and develop into minute, infecting worms ready to attack.’ … The climate favored wide regional geographic distribution of hookworm as the land, marked by warmth, moisture, and aerated sandy or loamy soils, allowed hookworm larvae to burrow for protection from the sun, perhaps for years. Hookworm also was endemic in the mining towns of North Carolina, Tennessee, Kentucky, and West Virginia, where the parasite found protection in mines from extremes of temperature and dryness [3].

The Rockefeller Sanitary Commission [RSC] developed and executed a three prong strategy over the years 1909-1914:

estimate hookworm prevalence in the American South;
provide treatment;
and eradicate the disease.

The RSC surveyed populations in 11 Southern states and found that about 40% of the population surveyed were infected with hookworms. Hookworms are still a problem today, with 500 million to 750 million individuals around the world estimated to be infected.

Reducing the prevalence of hookworm required understanding how it was spread:

Human transmission occurred as newly hatched hookworm larvae from eggs in contaminated soils entered hosts via direct, physical contact through a foot or hand. After penetrating the skin, they traveled to the heart and lungs and then to the small intestine, where they attached to capillaries to feed, remaining from 2 to 13 years. Females released thousands of eggs per day (9000–25000) into the intestine, which, after expulsion from the human body into inadequately designed privies or moist dirt, hatched in the soil and waited to enter human hosts, thus completing the cycle [3].

A campaign to educate people about the disease and how it was spread and to install latrines and toilets dramatically reduced the prevalence of the disease [3].

Three Lessons for Data Driven Not for Profits

The RSC was remarkably effective and there are many lessons that can be learned from it. Here we focus on just three.

Lesson 1. Use data to identify the problem and inform a solution. The RSC first surveyed the southern states to understand the problem and how best to deploy their resources. A large part of the effort was to collect appropriate data in order to develop an appropriate solution. Note that they did not rely on third party data from others, but funded efforts to collect the data that was needed.

Lesson 2. Make sure you understand the root cause of the problem so that appropriate actions can be taken. Here the actions was not just treating the disease, but even more important was understanding that the root cause of the disease was poor sanitary conditions that enabled the parasite to spread, and encouraging the use of latrines and toilets to stop the spread of the disease.

Lesson 3. Look at the problem holistically and deploy your funds to attack the problem as a whole. The RSC split their $1M of funding between i) research into the problem, ii) actions to treat patients and improve living conditions, and iii) education to change people’s behavior.

References

[1] Patterson, Maria T., and Robert L. Grossman. “Detecting spatial patterns of disease in large collections of electronic medical records using neighbor-based bootstrapping.” Big data 5, no. 3 (2017): 213-224.

[2] Drucker, Peter. Managing the non-profit organization. Routledge, 2012. Chapter 2.

[3] Elman, Cheryl, Robert A. McGuire, and Barbara Wittman. “Extending public health: the Rockefeller Sanitary Commission and hookworm in the American south.” American journal of public health 104, no. 1 (2014): 47-58.

[4] Bleakley, Hoyt. “Disease and development: evidence from hookworm eradication in the American South.” The quarterly journal of economics 122, no. 1 (2007): 73-117.