In some cases I think it may come down to whether we make an explicit agreement vs doing things informally. If I let a friend live with me because I have a spare room, and then they decide to spend weekends fixing up the house out of gratitude, I think that's not taxed. But if we explicitly agree that I will waive rent in exchange to 12hr/week of house repairs that probably is taxed. Penalizing people for being explicit in their interactions with each other seems harmful, though, as clarity in expectations can be a good way to reduce conflict.

Other things seem even messier. What if I take a job that pays less because I value the experience I'm gaining. What if it's for the fun? Prestige? Social impact? Or imagine I live in the US and want to donate to a charity that's tax-deductible in another country, and you vice-versa, so we each agree to donate to the other's charity. Or I agree to give up eating pigs in exchange to your giving up eating eggs. Is any of this taxable? Should it be?

Mostly I'm just pretty confused. I think it's possible that lots of things are technically taxable but most people don't report them and the IRS doesn't really expect them to be reported?

[1] Barter in the US is taxed as if we had each performed our services for "fair market value" in money. So if $50 would have been a fair price for the ride and for the babysitting, we each report an additional $50 of income.