Social Protection Rights in the Age of Algorithmic Decision-Making

The UNU Global AI Network Insight Blog Series is launched to celebrate the Network’s two-year anniversary. In this series, members of the UNU Global AI Network share diverse insights at the intersection of artificial intelligence and sustainable development. Bringing together perspectives across regions and disciplines, the series explores both the opportunities and challenges of shaping AI for all. 

The views expressed in published articles of this series are solely those of the author(s) and do not necessarily reflect the positions of the United Nations University, UNU Macau, the United Nations, or affiliated entities. Publication does not imply endorsement or institutional approval.

Authors are responsible for the accuracy and originality of their contributions. UNU reserves the right to edit, decline, or remove submissions in line with the purpose and values of the series.

For inquiries, please contact: AINetwork@unu.edu

Social protection delivery mechanisms translate legal rights into effective access to benefits and services. The gap between what the law promises and what people actually receive is shaped by institutions, financing and administrative capacity. Today, automated systems are increasingly informing and governing the determination and delivery of entitlements, shaping policy deliberately and sometimes inadvertently (ISSA, 2025).

This article distinguishes three types of automation: rules-based systems, which apply a fixed, transparent legal formula with no prediction involved; predictive scoring, which uses statistical regression to estimate outcomes from historical data through a legible formula a human can inspect (e.g. proxy means tests); and machine learning (ML) models, which learn rules from data, often gaining accuracy by detecting patterns no person specified, at the cost of explainability, since the logic exists only as learned weights. "Algorithmic decision-making" is used as an umbrella term. 

From a rights perspective, States remain obligated to respect, protect and fulfil the right to social protection through enforceable legal frameworks, non-discrimination, dignity, transparency, accountability and the right to appeal.

The Myth of Precision Targeting and the Challenge of Contestability

Everyone needs support at certain stages of the lifecycle, yet many people remain uncovered, leaving social assistance a critical role in reducing the extent and depth of poverty. In low- and middle-income countries, poverty targeting often relies on proxy means testing (PMT). However, PMTs may incur exclusion errors exceeding 70 percent (Kidd & Athias, 2020). ML techniques promise to reduce such errors: research in Togo, Indonesia, and elsewhere shows improved accuracy over classical PMT (Aiken et al., 2022; Wobcke & Mariyah, 2023). 

However, as an essentially probabilistic technique, it is highly inadequate to call the use of ML “precision targeting”. Whilst statistical match may improve with ML techniques, the main reservations of PMTs still apply to ML: outdated data, movements of individuals and households above or below the poverty line can oscillate wildly, and significant numbers may hover just above the threshold (Kidd and Athias, 2020). 

Most importantly, ML methods compound PMT's transparency problems. Most PMT systems rest on explicit regression formulas whose results can be replicated — but households rarely know the formulas or find them too complex to contest on the merits; Colombia's Sisbén, whose income-capacity parameters were confidential, is emblematic (López, 2020; Jiménez, 2021). ML goes further: the underlying inference parameters cannot technically be observed at all, eroding the transparency, rules-stability and appeal rights presumed under ILO Convention No. 102.

The significant impact on people's livelihoods, their position of dependency, and the difficulty of contesting such decisions are the reasons why the EU AI Act (Annex III, point 5(a)) classifies AI systems used by public authorities to evaluate eligibility for public assistance benefits as high-risk. Classifying eligibility systems correctly is therefore the first act of accountability: a legislated formula, a statistical model and an ML system each fail differently and must be contested differently — through disclosure of formulas, error rates, bias audits of training data, or outright calls for legal reviews of the rules or prohibition.

Dignity Concerns Arising from Automated Controls

Social protection has always balanced provision with control. Social assistance should treat people as rights-holders, not monitored suspects. 

The Netherlands' Systeem Risico Indicatie (SyRI) cross-referenced up to 17 government databases to generate fraud-risk scores for residents of low-income neighborhoods, without informing them or providing any means of contestation. In 2020, a Dutch court found the scheme incompatible with Article 8 of the European Convention on Human Rights: its opacity, and its concentration on poor areas made the interference disproportionate and unforeseeable (Rachovitsa & Johann, 2022). 

Where institutional culture treats claimants as presumptively fraudulent, algorithmic risk scoring operationalizes suspicion at scale. The alternative is to flag errors in exclusion and discrimination patterns and support consistent staff decisions (Cho & Kim, 2024).

Risks of Predictive Automation in Assessing and Calculating Entitlements

Determining benefit levels and paying them rapidly and correctly remains a rights-determining act the State must answer for. Rules-based simulators provide verifiable clarity: Portugal's calculators on the Segurança Social Direta portal let citizens compute in advance, from the same legal formula the administration applies, exactly what right-holders are entitled to. 

Efficiency drives, however, can erode adequacy. Australia's NDIS reform matches assessments to standardized benefit packages; staff and advocates warn this reduces human judgment and misses needs that do not fit standard categories (Every Australian Counts, 2025). Because prediction is probabilistic, error exists by design — landing as payments delayed, reduced or denied for those most in need. Since the amount and timeliness of support are essential to the adequacy of the right to social security, predictive techniques may inform entitlement decisions but cannot be permitted to constitute those decisions.

Collective Solidarity Versus Individual Risk Rating

Solidarity in health financing means creating a shared pool covering actual need, financed by ability to contribute rather than prior risk scores. 

Rules-based data matching already enables progressive financing in some countries: Rwanda links insurance membership to its Ubudehe socio-economic database to set household contributions; Korea's National Health Insurance Service computes contributions in near real time across more than 30 interoperable databases (Mathauer, Kutzin & Meessen, 2024). 

By contrast, a growing platform-economy trend uses AI to price commercial insurance for gig workers individually, transaction by transaction, rather than pooling them into stable collective schemes (Behrendt, Nguyen & Rani, 2019). The convenience of such products risks entrenching fragmented coverage that shifts risk back onto individuals, dissolving contributions to national solidarity-based schemes (ILO, ISSA & OECD, 2023). Where solidarity is mandated by law, algorithmic techniques serve collective schemes; where market incentives dominate, technology aligns to those.

Profiling Can Misrepresent the Needs of Vulnerable People

Digital systems can tailor support to real circumstances and correct bureaucratic biases favoring those with literacy and connections. But unprincipled design stratifies rather than personalizes. 

Serbia's Social Card registry, launched in 2022, excluded an estimated 22,000–27,000 people — disproportionately Roma and persons with disabilities — often on stale or misclassified data: cars long sold for scrap, subsistence recycling income treated as earnings, or a funeral donation flagged as income. Its semi-automated design simultaneously curtailed social workers' discretion to verify circumstances through field visits — removing precisely the human judgment that could have corrected the data's misrepresentation of poverty as actually lived (Amnesty International, 2023).

Conclusion: Moving Toward Meaningful Human Judgement and Governance

The same capabilities that enable States to extend protection more accurately and proactively can also allow them to enforce control more rigidly and insulate decisions from accountability. The balance between the two counter approaches is often tilted because automation tends to be driven by efficiency gains under conditions of austerity in the public sector.

The determining factor is not the technology but the policy logic it operationalizes. Where claimants are treated as rights-holders, algorithmic tools reduce exclusion and personalize support; where they are treated as risk objects, the same tools encode bias at scale and displace the contextual judgment that complex lives require.

Whilst it is critical to ensure human oversight, it is equally important to emphasize the need to reinforce meaningful human judgement. Human oversight can easily become a rubber stamp. A caseworker may technically remain “in the loop”, but may not have the authority, time, information or institutional incentives to challenge the model’s recommendation.

Finally, it is important to understand and clearly distinguish the different algorithmic decision-making systems to demand the proper governance and accountability rules: publication of variables, weights, logic and decision rules where possible; assess impacts on adequacy, equality and dignity before deployment and as said, requiring meaningful human judgement for rights-affecting decisions.

The author gratefully acknowledges the valuable guidance and insightful comments provided by Shahra Razavi, Director of the Universal Social Protection Department; Veronika Wodsak, Social Protection Specialist; and Liyuan Xiao, Digital Social Protection Expert, International Labour Office, Geneva.

References

Aiken, E., Bellue, S., Blumenstock, J. E., Karlan, D., & Udry, C. (2022). Machine learning and phone data can improve targeting of humanitarian aid. Nature, 603(7903), 864–870.

Amnesty International (2023). Trapped by Automation: Poverty and Discrimination in Serbia's Welfare State. EUR 70/7407/2023.

Behrendt, C., Nguyen, Q. A., & Rani, U. (2019). Social protection systems and the future of work: Ensuring social security for digital platform workers. International Social Security Review, 72(3), 17–41.

Cho, W., & Kim, J. (2024). Why and how does a machine learning algorithm coexist with alternative methods? The case of the social welfare blind spot identification system. MIS Quarterly, 48(4).

Every Australian Counts (2025). Warning over computer-generated NDIS plans and erosion of independent oversight.

ILO, ISSA, & OECD (2023). Providing Adequate and Sustainable Social Protection for Workers in the Gig and Platform Economy. Geneva: ILO.

ISSA (2025). Applications de l'intelligence artificielle dans la sécurité sociale: Éléments factuels et éclairages tirés de TechByte. Geneva: ISSA.

Jiménez, M. B. (2021). Chosen by a secret algorithm: Colombia's top-down pandemic payments. Transformer States series, CHRGJ, NYU School of Law, 14 December 2021.

Kidd, S., & Athias, D. (2020). Hit and Miss: An Assessment of Targeting Effectiveness in Social Protection. Development Pathways.

López, J. (2020). Experimentando con la pobreza: El Sisbén y los proyectos de analítica de datos en Colombia. Fundación Karisma.

Mathauer, I., Kutzin, J., & Meessen, B. (2024). Exploring the effects of digital technologies in health financing for universal health coverage. Oxford Open Digital Health, 2(1).

Rachovitsa, A., & Johann, N. (2022). The human rights implications of the use of AI in the digital welfare state: Lessons learned from the Dutch SyRI case. Human Rights Law Review, 22(2).

Wobcke, W., & Mariyah, S. (2023). Machine Learning Approaches to Proxy Means Testing: Evidence from Indonesia's DTKS.