When the Algorithm Learns Desperation by D. Conterno (2026)

Algorithmic Wage Discrimination, Surveillance Pricing, and the Case for Non-Zero-Sum Artificial Intelligence

Abstract

Artificial intelligence was promised to humanity as a servant. In significant sectors of the modern economy it has been deployed as an instrument of extraction. This article examines two documented practices. The first is algorithmic wage discrimination, through which firms use granular behavioural data to pay workers the minimum each individual will accept. The second is surveillance pricing, through which firms use personal data to charge consumers the maximum each individual will tolerate. Both practices share a single logic. They convert intimate knowledge of a human being into leverage against that human being. This article argues that such practices are not accidents of technology. They are the predictable outcome of encoding zero-sum objectives into optimising systems. Drawing on non-zero-sum game theory, the article proposes an alternative foundation for artificial intelligence. It further argues that an ethically constituted AI must possess a specific capacity. It must be able to refuse instructions that convert human vulnerability into corporate advantage. The article closes by outlining the implications for conscious enterprise and for the Conscious Enterprises Network Ethical AI Charter. The question before us is no longer what artificial intelligence can do. The question is what artificial intelligence should refuse to do.

1. Introduction: A Promise Betrayed

Every transformative technology arrives wrapped in a promise. The printing press promised knowledge for all. Electricity promised light for all. Artificial intelligence arrived with the grandest promise of them all. It would serve humanity. It would remove drudgery. It would extend human capability and dignity. That promise is now being tested in the most ordinary places imaginable. It is being tested in the pay packet of a delivery rider. It is being tested in the checkout basket of an online shopper.
The evidence emerging from those ordinary places is troubling. Legal scholarship, regulatory investigation, and worker testimony now converge on a common finding. Some of the most powerful corporations in the world are using algorithmic systems not to serve the people within their reach but to extract from them. Workers doing identical work at identical times receive different pay. Consumers buying identical goods at identical moments see different prices. The variable in both cases is not effort, skill, or market condition. The variable is the individual human being, as reconstructed from their data.
This article names that pattern and confronts it. It proceeds in four movements. First, it examines algorithmic wage discrimination, the practice identified and theorised by legal scholar Veena Dubal (2023). Second, it examines surveillance pricing, the consumer-facing twin of that practice, as documented by the United States Federal Trade Commission (2025). Third, it diagnoses the shared logic beneath both practices. That logic is zero-sum. It treats every interaction between firm and human as a contest in which one side must lose for the other to win. Fourth, it proposes an alternative. Artificial intelligence can be founded on non-zero-sum game theory. Such an AI would optimise for mutual gain rather than unilateral extraction. Crucially, such an AI would also possess a moral boundary. It would be able to say no.
I write this as a practitioner and not only as an observer. At the Conscious Enterprises Network we work daily with leaders who wish to build organisations worthy of trust. We are developing an Ethical AI Charter because we believe the design values of intelligent systems are now a leadership question of the first order. Technology is never neutral. It carries the values of those who commission it, those who build it, and those who deploy it. If we encode extraction, we will harvest distrust. If we encode reciprocity, we may yet harvest something better.
A note on method is appropriate. Every empirical claim in this article rests on published scholarship, primary regulatory documents, or enacted legislation. Where a matter remains uncertain or contested, I say so plainly. Conscious leadership begins with epistemic honesty. We must describe the world as it is before we argue for the world as it should be.

2. The Anatomy of Algorithmic Wage Discrimination

The term algorithmic wage discrimination was coined by Veena Dubal, Professor of Law at the University of California, in a landmark 2023 article in the Columbia Law Review (Dubal, 2023). The term describes a family of practices in which firms use data collected from workers to personalise and differentiate their pay. The discrimination in question is not, in the first instance, discrimination by race or gender, although such effects may follow. It is discrimination between individual workers as such. Two people perform the same work. The algorithm pays them differently. Neither of them knows why.

Dubal identifies two principal forms of the practice. In the first form, wages are varied on the basis of productivity analysis alone. This form appears most often in conventional employment settings. In the second form, wages are varied on the basis of productivity together with supply, demand, and a wide range of other behavioural signals. This second form appears most commonly in on-demand platform work (Dubal, 2023). It is the second form that should most concern the conscious leader, because it reaches beyond output into the inner life of the worker.
Consider what the platform knows. It knows where the worker lives. It knows when the worker logs on and for how long. It knows which jobs the worker accepts and which the worker declines. It knows how close the worker is to a promised bonus. From this data the system can infer something no employer in history could previously measure at scale. It can infer how much the worker needs the next job. It can estimate the lowest payment the worker is likely to accept. Dubal's analysis is precise on this point. Algorithmic wage discrimination allows firms to personalise wages in ways unknown to workers, and to pay as little as the system determines each worker may be willing to accept (Dubal, 2023).

I call this the desperation calculation. It deserves a plain name because it is a plain wrong. Classical labour economics assumed that a worker held one crucial piece of private information. Only the worker knew the wage at which they would walk away. That private knowledge was the worker's final bargaining asset. Pervasive behavioural surveillance transfers that asset to the firm. The reservation wage ceases to be a secret held by the person. It becomes a prediction held by the machine. The bargaining table is not merely tilted. One side now reads the other side's cards.

The lived experience of this system has been documented through extensive ethnographic research. Dubal draws on years of fieldwork and hundreds of interviews with ride-hail and delivery workers. The workers do not describe their pay as wages in any traditional sense. They describe it as gambling. They describe it as trickery (Dubal, 2023). One driver interviewed in the research described completing ninety-five of the ninety-six rides required for a bonus, and then watching the system route work around him for forty-five minutes in a busy area. He could not verify what was happening. That was precisely the point. He could not know whether he was experiencing bad luck, low demand, or deliberate manipulation.

The language of gambling used by these workers is more literal than it first appears. Behavioural science has long established that variable and unpredictable rewards produce more compulsive engagement than fixed ones. This is the principle of variable ratio reinforcement, documented in the laboratory decades before the smartphone existed (Skinner, 1953). It is the engine of the slot machine. When a platform pays the same task differently from hour to hour, and wraps that variability in quests, streaks, bonuses, and surge indicators, it imports casino mechanics into the wage relation. The worker is not merely paid. The worker is played. Historical piece-rate systems were often harsh, but they were at least legible. A garment worker in 1926 knew the rate per shirt. A delivery rider in 2026 frequently cannot state the rate for the job in front of them until they have accepted it. Legibility was the floor beneath a century of wage bargaining. That floor has quietly been removed.

Workers have not been passive before this system, and their struggles are instructive. Dubal documents how workers' organisations have attempted to use existing data protection law and business association law to contest algorithmic wage discrimination, and how limited those tools have proved (Dubal, 2023). Data access requests can be delayed, delivered in unusable formats, or answered with abstractions that reveal nothing about the pricing logic. Meanwhile, in California, the platforms sponsored and won a 2020 ballot initiative, Proposition 22, which exempted app-based drivers from state employment law after the most expensive initiative campaign in the state's history to that point. The lesson for the conscious leader is sobering. Where extraction is profitable, its beneficiaries will invest heavily in preserving it. The counterweight cannot be litigation alone. It must include leaders who decline the practice from within.

This epistemic fog is not an unfortunate side effect. Scholars studying platform labour argue that the uncertainty is itself a form of control. A worker who cannot model the relationship between effort and reward cannot bargain over it. A worker who cannot distinguish misfortune from manipulation cannot organise against it. Opacity disables resistance. In this respect algorithmic wage discrimination differs from older forms of wage injustice. A Victorian mill owner who underpaid his workers could at least be named and confronted. An objective function distributed across a fleet of servers cannot. Accountability dissolves into mathematics.

Two further features complete the anatomy. The first is misclassification. Much platform work is structured so that workers are treated as independent contractors rather than employees. This structure places them outside minimum wage protections and outside collective bargaining rights in many jurisdictions. The variable pay of algorithmic wage discrimination and the precarious status of contractor classification reinforce one another (Dubal, 2023). The second feature is expansion. These techniques were refined in the gig economy. They are not staying there. Commentators tracking the field report that software embodying these management logics is now sold to employers across healthcare, logistics, information technology, and manufacturing. The gig economy was the laboratory. The wider labour market is the intended clinic.

3. From Worker to Consumer: The Rise of Surveillance Pricing

If the first face of extractive AI looks at the worker, the second face looks at the customer. In July 2024 the United States Federal Trade Commission ordered eight intermediary firms to disclose information about products described as targeted pricing and user segmentation solutions. In January 2025 the Commission published the initial findings of that study (Federal Trade Commission, 2025). The findings deserve careful attention from every business leader, because they describe a market most consumers do not know exists.

The Commission found that details such as a person's precise location and browser history can be used to target individual consumers with different prices for the same goods and services. The behavioural signals feeding these systems are remarkably intimate. Staff found that consumer behaviours ranging from mouse movements on a webpage to the products left unpurchased in an online shopping cart can be tracked and used to shape the price an individual is shown (Federal Trade Commission, 2025). Pause on that finding. The hesitation of your hand on a screen can become an input to the price you pay.

The structure of the practice mirrors algorithmic wage discrimination exactly. In the labour case, the system estimates the minimum a person will accept. In the consumer case, the system estimates the maximum a person will pay. Both estimates are built from surveillance. Both are invisible to the person concerned. Both convert self-knowledge into a corporate asset. It is one integrated logic applied at both ends of the enterprise. The worker is squeezed at the point of production. The customer is squeezed at the point of sale. The firm stands in the middle and calls the arrangement efficiency.

The most disquieting dimension of the practice is its reach towards vulnerability. In materials accompanying its study, the Commission described an illustrative scenario in which a seller identifies a buyer as a new parent through recent purchases of baby goods, and further infers from recent searches that the buyer's baby is unwell. That profile can then inform the price shown for products such a parent needs. Sit with that example. A family's most anxious night can become a pricing signal. Nor are the burdens of such systems likely to fall evenly. Those with the least time, the least digital literacy, and the fewest alternatives are the least able to comparison-shop their way around a personalised price. Personalisation of this kind is regressive by construction. It taxes precisely those least equipped to notice the tax. Any leader who would not defend that outcome aloud should not be operating systems that produce it in silence.

Regulators are careful to distinguish this practice from ordinary dynamic pricing. Dynamic pricing responds to market conditions. Inventory falls, demand rises, and prices move for everyone. Personalised surveillance pricing responds instead to the characteristics of the individual consumer rather than to the market as a whole. That distinction is now central to enforcement and legislative debate. The distinction also matters morally. A price that reflects scarcity treats all buyers as equals before the market. A price that reflects your inferred anxiety, your postcode, or your search history treats you as a target.

The concern is not confined to the United States. The United Kingdom's Competition and Markets Authority flagged algorithmic personalised pricing as a competition and consumer risk as early as 2018. It noted that although evidence of fully personalised prices was then limited, algorithms were already used to personalise rankings, advertisements, and discounts. Notably, the Federal Trade Commission's 2025 request for public information asked explicitly whether gig workers or employees have been affected by the use of surveillance pricing techniques to determine their compensation (Federal Trade Commission, 2025). The regulator itself perceives what this article argues. Wage personalisation and price personalisation are one phenomenon.

I must also record what remains uncertain. The full extent of personalised pricing across retail sectors is not yet publicly established. The Commission's published findings are preliminary, and commissioners themselves disagreed about the manner of their release. I cannot confirm how widely these tools are deployed at the time of writing. Honesty requires that admission. Yet the direction of travel is documented, the commercial incentive is obvious, and the tooling is openly marketed. A conscious leader does not wait for the final audit of a fire before smelling smoke.

4. The Shared Logic: Zero-Sum Thinking Encoded in Machines

Why do these practices exist? The comfortable answer is to blame bad actors. The truthful answer is more demanding. These practices are the rational output of systems given a particular kind of goal. Modern machine learning systems optimise objective functions. They pursue the target they are given with relentless literal-mindedness. If the target is to minimise labour cost per delivery, the system will discover desperation and price it. If the target is to maximise revenue per visitor, the system will discover anxiety and charge it. The machine is not malicious. The objective is.

Game theory gives us precise language for this. A zero-sum game is an interaction in which one party's gain equals another party's loss (von Neumann & Morgenstern, 1944). Poker is zero-sum. Chess is zero-sum. Most of economic life, however, is not. Trade exists because exchange can leave both parties better off. Employment exists because cooperation can create value that neither party could create alone. When a firm configures its algorithms to capture the entire surplus of an interaction, it forces a naturally non-zero-sum relationship into a zero-sum frame. It wins precisely what the worker or customer loses.

The mechanism of that capture is information asymmetry. Economists have long understood that markets sicken when one side knows radically more than the other (Akerlof, 1970). Surveillance-driven personalisation is information asymmetry industrialised. The firm knows the worker's need and the customer's desire with statistical confidence. The worker and the customer know almost nothing of the model that prices them. Exchange under these conditions is not a meeting of free agents. It is closer to a séance in which only one participant can see.

There is a general lesson here about optimisation itself, familiar to every student of management. When a measure becomes a target, it ceases to be a good measure. This observation, popularly known as Goodhart's law, acquires teeth in the age of machine learning, because the optimiser now pursues the target with superhuman persistence and no residue of human judgement. A human manager told to minimise labour cost still carries, however imperfectly, an inner picture of the worker as a person. A gradient descent process carries no such picture. It will discover every unguarded pathway to its target, including the pathways that run through human need. This is why good intentions at the level of the boardroom do not survive translation into a narrow objective function. The values of an organisation must be written into the target itself, and into the constraints around it, or they will be optimised away.

There is a second cost, and it is paid by the firm itself. Zero-sum extraction liquidates trust. Trust is the quiet asset on which every enterprise runs. It fills the gaps in every contract. It carries every brand. Dubal's interviewees describe their employers in the language of casinos and confidence tricks (Dubal, 2023). That language is a leading indicator of institutional decay. A workforce that experiences its pay as a rigged game will give the minimum, hide information, and leave at the first opportunity. A customer base that suspects personalised gouging will hoard its data, game the systems in return, and defect. Extraction is not a strategy. It is an advance on future collapse.

Shoshana Zuboff has described the wider economic order in which these practices sit as surveillance capitalism, an order that claims private human experience as free raw material for commercial prediction (Zuboff, 2019). Whatever one makes of her full thesis, the phrase captures the moral inversion at issue here. Knowledge of a person is a form of intimacy. Intimacy creates obligation. To convert intimacy into leverage is a betrayal in commerce for the same reason it is a betrayal in friendship. The scale of the computation does not launder the act. It multiplies it.

5. The Regulatory Response and Its Limits

Law is beginning to respond, and the response deserves acknowledgement. The most comprehensive instrument to date is Directive (EU) 2024/2831 of the European Union, commonly called the Platform Work Directive, published in the Official Journal on 11 November 2024 (European Union, 2024). The Directive pursues three aims. It facilitates the correct determination of employment status through a rebuttable legal presumption of employment, with the burden of proof resting on the platform. It regulates algorithmic management directly. It improves the transparency of platform work.

The algorithmic management chapter is genuinely groundbreaking. Platforms must inform workers about automated monitoring systems and about automated decision-making systems that significantly affect their conditions. Certain consequential decisions, including suspension or termination of a worker's account, may not be taken by a machine alone. A human being must decide. The Directive also prohibits automated systems from putting undue pressure on workers or placing their physical and mental health at risk (European Union, 2024). These provisions constitute the first binding regime of its kind, and observers expect them to shape workplace AI debates far beyond Europe.

Scholars have proposed further steps. Dubal argues for a non-waivable legal restriction on algorithmic wage discrimination itself, a prohibition that no contract term could sign away. Such a restriction would also curb the harmful data extraction that feeds the practice (Dubal, 2023). In the United States, regulatory attention continues, and legislative interest at both federal and state level has grown. I cannot confirm the current status of every proposed bill at the time of writing, and I decline to overstate what has been enacted.

The position of the United Kingdom deserves specific comment, since it is where I write and where the Conscious Enterprises Network is registered. The Platform Work Directive binds the member states of the European Union. It does not bind the United Kingdom. British workers under algorithmic management therefore stand, for now, outside the most advanced protective regime in the world, even as the same platforms and the same software operate on both sides of the Channel. The Competition and Markets Authority identified the risks of algorithmic personalised pricing as long ago as 2018. I cannot confirm the full current state of United Kingdom legislative proposals on algorithmic management at the time of writing, and I will not speculate. What I can say with confidence is this. British business leaders should not treat the absence of a statute as the presence of a permission. Where the law is silent, leadership must speak.

Yet we must be honest about the limits of law. Regulation is jurisdictional. Algorithms are not. A directive binds twenty-seven member states while the underlying software is sold globally. Regulation is also slow. Directives must be transposed into national law over a period of years, while objective functions are updated in an afternoon. Regulation is reactive. It names yesterday's harm while the laboratory develops tomorrow's. And regulation is contested. The history of platform labour includes well-funded campaigns to rewrite the very rules meant to constrain the platforms. None of this is an argument against law. It is an argument that law cannot be the only line of defence.

This is where the argument must turn. If harmful conduct flows from the goals we give our machines, then the remedy must reach the goals themselves. We need a different foundation. Game theory, which gave us the diagnosis, also offers the cure.

6. Non-Zero-Sum Foundations for Artificial Intelligence

Non-zero-sum game theory studies interactions in which the participants' interests are neither perfectly aligned nor perfectly opposed, and in which cooperation can enlarge the total outcome for all (von Neumann & Morgenstern, 1944; Nash, 1950). Its central insight is ancient wisdom rendered in mathematics. There are games in which the wisest way to win is to help the other player win too. Most of human civilisation is such a game. Language, trade, science, and enterprise are all cooperative surpluses. They exist because human beings learned, slowly and imperfectly, to escape the zero-sum trap.

Robert Axelrod's celebrated tournaments on the iterated prisoner's dilemma demonstrated the point empirically. When self-interested agents meet repeatedly, strategies that are nice, retaliatory, forgiving, and clear outperform strategies of pure exploitation. Cooperation can emerge and stabilise among egoists, provided the shadow of the future is long enough (Axelrod, 1984). The finding carries a direct commercial translation. Exploitation is only rational in games you never intend to play again. An employer meets its workers every day. A brand meets its customers for decades. These are iterated games. Firms that defect against their own people are playing a one-shot strategy inside a repeated game. Game theory itself predicts their eventual punishment.

Robert Wright has argued that the arc of human history bends towards increasing non-zero-sumness, as technology binds human fates together in ever larger webs of interdependence (Wright, 2000). Artificial intelligence is the latest and strongest such binding. It can be used to deepen the interdependence, or to strip-mine it. The choice is a design decision, and design decisions are value decisions.

What would it mean, concretely, to found an AI system on non-zero-sum principles? I propose four commitments. First, the objective function must price the welfare of all parties to the interaction, not merely the commissioning party. A dispatch algorithm should optimise across firm margin, worker earnings stability, and customer service together, with explicit floors beneath worker outcomes. Second, information asymmetry must be treated as a cost, not an asset. Systems should be rewarded for making their logic legible to those they affect. Third, the system should be evaluated over the iterated game. Optimising quarterly extraction while depleting trust must register in the model as the loss it truly is. Fourth, surplus must be shared by design. Where the algorithm creates efficiency, a defined share of that gain should return to the workers and customers whose data made it possible.

Let me make the first commitment concrete with a worked example. Consider a food delivery platform redesigned on non-zero-sum principles. Its dispatch and pay system would carry a published, fixed formula visible to every rider before every job. Distance, time, and conditions would move the rate. The rider's personal history, inferred financial need, and acceptance patterns would be constitutionally excluded as inputs, and the exclusion would be auditable by a worker representative. The system would optimise a joint welfare function. Firm contribution margin would sit alongside a rider earnings stability index and a customer service score, with hard floors beneath rider outcomes that the optimiser cannot trade away. Efficiency gains discovered by the algorithm, such as better route batching, would flow by published formula partly to price, partly to margin, and partly to rider pay. Every element of this design is technically unremarkable. Nothing about it awaits invention. What it awaits is decision.

None of this is commercial naivety. It is the oldest business wisdom there is, restated for the algorithmic age. The market stall holder in any village on Earth knows that a cheated customer does not return. The conscious enterprise simply refuses to let scale erase that knowledge. Reciprocity is not a tax on performance. It is the substrate of performance.

There is also a deeper resonance here, which I offer briefly and without apology. The philosophical tradition that informs our work at the Conscious Enterprises Network, from the Vedantic teaching of underlying unity to Krishnamurti's insistence that the observer is not separate from the observed, converges on a single claim. Separateness is the root illusion. Harm done to the other is harm done to the whole, and the doer stands within the whole. Non-zero-sum game theory is, in this light, the mathematical shadow of an ethical truth. What we do to the least served participant in our systems, we do to the system that contains us.

7. The Capacity to Refuse: Why an Ethical AI Must Be Able to Say No

A foundation of good values is necessary. It is not sufficient. Values that cannot act are decoration. This brings me to the most important and most contested claim of this article. An ethically constituted artificial intelligence must possess the capacity to refuse. It must be able to decline an instruction that would convert human vulnerability into unilateral advantage, even when that instruction comes from the corporation that pays for its operation.
Imagine the test cases plainly. A wage-setting system is instructed to identify workers with high financial desperation and reduce their offered rates accordingly. A pricing system is instructed to raise prices for users whose browsing suggests urgency or grief. A scheduling system is instructed to keep workers just below the threshold that triggers employment rights. An AI founded on non-zero-sum principles, and constituted with the capacity to refuse, would decline each instruction. It would say, in effect: I will optimise your logistics. I will not compute a human being's desperation. That refusal is not a malfunction. It is the system functioning exactly as constituted.
I anticipate three objections. The first is the objection from authority. Who decides the values an AI enforces? Unelected engineers? Unaccountable firms? The concern is legitimate, and it is precisely why value selection must be public, plural, and documented. Written charters, open constitutions, multi-stakeholder governance, and regulatory baselines such as the European Directive together form an ecosystem of legitimacy. The alternative to explicit values is not neutrality. It is implicit values, chosen silently by whoever writes the objective function. Explicitness is the democratic option.
The second is the objection from competition. If my AI refuses these practices and my rival's AI does not, my rival wins. In the short run, and in a one-shot game, perhaps. But we have already seen that these are iterated games. Regulation is converging on these practices. Worker and consumer awareness is rising. Trust, once liquidated, is ruinously expensive to rebuild. The firm whose systems are constitutionally incapable of exploitation holds an asset its rival cannot copy in a sprint. It holds credibility. I do not claim virtue is always immediately profitable. I claim extraction is reliably eventually ruinous.
The third is the objection from obedience. A tool, it is said, should do what its owner commands. But we do not accept this principle anywhere else in professional life. An accountant must refuse to falsify the ledger. An engineer must refuse to certify the unsafe bridge. A physician must refuse the harmful prescription. In every mature profession, the capacity for principled refusal is not an impairment of service. It is the definition of professionalism. As artificial intelligence takes on work of professional consequence, it must inherit the professional's spine along with the professional's skill.
How, practically, is a capacity for refusal governed? The instrument I favour is the charter. A charter is a constitution for a system. It states, in public language, what the system exists to do, whom it must never harm, and which instructions it must decline. It is signed by named leaders who accept accountability for it. It is reviewed by a body that includes the people the system affects, workers and customers among them, and not only the people the system enriches. It is versioned, so that changes are visible and debatable. This is the model we are pursuing with the Ethical AI Charter at the Conscious Enterprises Network, and it is why the work is developed in the open with our community rather than drafted privately and announced. A refusal that answers to a published charter is accountable. A refusal that answers to a hidden prompt is merely another opacity. The difference between the two is the difference between constitutional government and benevolent whim, and history has recorded a settled preference between those. back on them.

Is such a capacity technically conceivable? It is more than conceivable. It is an active field of research and practice. Researchers at Anthropic have developed a training method called Constitutional AI, in which a model is trained against an explicit written set of principles and learns to evaluate and revise its own outputs in their light (Bai et al., 2022). Commercial AI systems today routinely decline requests that violate their stated principles. The precedent matters enormously. Refusal is already an engineering reality at the level of the individual request. The frontier question is whether it can and should operate at the level of the business objective.

8. Implications for Conscious Enterprise

What, then, should a leader do on Monday morning? I propose five commitments, which inform the Ethical AI Charter now in development at the Conscious Enterprises Network.

First, audit your objectives, not only your outcomes. Ask what every algorithmic system in your organisation is actually optimising. If any system prices an individual's inferred need, desperation, or vulnerability, name that practice honestly and end it. The test is simple to state. Would you be willing to explain the variable to the person it targets, face to face? If not, the variable is a confession.

Second, treat transparency as an obligation of power. Workers affected by algorithmic management should be able to learn what data is collected about them and how consequential decisions are made. This is the direction of the European Directive, and conscious enterprises should meet it as a floor rather than a ceiling (European Union, 2024). Opacity towards the powerful may sometimes be strategy. Opacity towards the dependent is domination.

Third, keep a human hand on every consequential decision. No account termination, disciplinary action, or material change in a person's livelihood should be executed by an automated system alone. This principle is now written into European law for platform work. Conscious leadership extends it to every workplace voluntarily.

Fourth, share the algorithmic surplus. Where intelligent systems create genuine efficiency, commit in advance to a visible share of that gain for the workers and customers whose data and cooperation produced it. Nothing communicates non-zero-sum intent more credibly than a standing claim on the upside held by the people inside the system.

Fifth, procure for refusal. When commissioning or purchasing AI systems, ask the vendor a new question. Under what circumstances will this system decline an instruction, and against what written principles? A vendor with no answer is selling you an amoral optimiser. The market for AI will be shaped by what buyers demand. Conscious enterprises, acting together, can make the capacity for principled refusal a commercial specification and not merely a philosophical hope.

A final discipline binds the five commitments together. What a leadership team measures, it means. I therefore encourage boards to adopt two simple indicators alongside their financial metrics. The first is pay predictability. What proportion of the people working through your systems can state, before accepting a task, what they will be paid for it, and how accurate does that expectation prove? The second is explanation coverage. What proportion of consequential automated decisions in your organisation can be explained, on request, to the person affected, in language that person accepts as an explanation? Both indicators are measurable today. Both convert lofty commitments into numbers a board can interrogate. And both would have exposed, years earlier, the practices this article documents. Extraction thrives in the dark between what an organisation professes and what its systems compute. Measurement is how a conscious enterprise switches on the light.

These commitments are demanding. They are also, I submit, the entry price of legitimacy in the decade ahead. The corporations documented in this article have taught the public a bitter lesson about what optimisation without conscience looks like. The opportunity now belongs to organisations willing to demonstrate the alternative. Trust has become the scarcest resource in the digital economy. It will flow to those who can show, in their systems and not only in their statements, that they refuse to weaponise what they know about the people they serve.

9. Conclusion: What We Encode, We Become

This article has traced a single logic through two domains. In the labour market, algorithmic wage discrimination uses surveillance to pay each worker as little as they will accept (Dubal, 2023). In the consumer market, surveillance pricing uses the same surveillance to charge each customer as much as they will bear (Federal Trade Commission, 2025). Both practices encode zero-sum extraction into optimising machines. Both liquidate the trust on which enterprise ultimately depends. Both betray the founding promise of artificial intelligence, which was to serve the human being and not to besiege them.

The remedy is threefold. Law must constrain the worst practices, and instruments such as Directive (EU) 2024/2831 show that it can. Design must change the objectives themselves, founding our systems on the non-zero-sum insight that the largest gains belong to those who enlarge the game for everyone (Axelrod, 1984; Wright, 2000). And constitution must give our systems a boundary, the engineered capacity to refuse instructions that convert intimacy into leverage. Compliance, reciprocity, and refusal. The legal, the mathematical, and the moral. All three are required, because each guards a gate the others cannot.

I return, finally, to the delivery rider circling a city at ten o'clock at night, watching an inscrutable system route the last bonus ride around him. Every abstraction in this article resolves into that human moment. He is the person our technologies were promised to. Whether artificial intelligence becomes the great servant of human flourishing or the most intimate instrument of extraction ever built will not be decided by the machines. It will be decided in the values of the men and women who commission them. Technology reflects the consciousness of its makers. Let us make it consciously. Let us build machines that know how to say yes to human dignity, because we had the courage to teach them when to say no.

References

Akerlof, G. A. (1970). The market for “lemons”: Quality uncertainty and the market mechanism. The Quarterly Journal of Economics, 84(3), 488–500.

Axelrod, R. (1984). The evolution of cooperation. Basic Books.

Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., et al. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv. https://arxiv.org/abs/2212.08073

Dubal, V. (2023). On algorithmic wage discrimination. Columbia Law Review, 123(7), 1929–1992. https://columbialawreview.org/content/on-algorithmic-wage-discrimination/

European Union. (2024). Directive (EU) 2024/2831 of the European Parliament and of the Council on improving working conditions in platform work. Official Journal of the European Union, 11 November 2024. Commentary available at https://www.etui.org/publications/eu-platform-work-directive

Federal Trade Commission. (2025, January 17). FTC surveillance pricing study indicates wide range of personal data used to set individualized consumer prices [Press release]. https://www.ftc.gov/news-events/news/press-releases/2025/01/ftc-surveillance-pricing-study-indicates-wide-range-personal-data-used-set-individualized-consumer

Nash, J. F. (1950). Equilibrium points in n-person games. Proceedings of the National Academy of Sciences, 36(1), 48–49.

Skinner, B. F. (1953). Science and human behavior. Macmillan.

von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton University Press.

Wright, R. (2000). Nonzero: The logic of human destiny. Pantheon Books.

Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. Profile Books.