We present an aggregation theory for the Social Welfare Function (SWF) that enables its empirical estimation. Agents have heterogeneous perceptions of how the government should value the welfare of other agents, which results in so-called Individual Welfare Functions (IWFs) that are shaped by the individuals‘ life experiences. The SWF is constructed as an aggregation of IWFs, weighted by the political weights of each group of agents. We then develop an estimation strategy to identify the SWF and IWFs based on the observed levels of capital tax, consumption tax, labor tax, and public debt. This strategy extends the inverse optimal approach to a general equilibrium heterogeneous-agent model. The application of our methodology to France and the United States shows that France‘s SWF is more Egalitarian and places greater emphasis on low-income individuals than the United States‘ SWF, which is more Libertarian and assigns greater weight to high-income individuals. Finally, we simulate the fiscal system of the United States under the assumption that it adopts the French SWF. Our findings indicate that the SWF significantly shapes the observed fiscal system and equilibrium inequality.