Looks like in 1961, it was an entire country of fair weather fans who watched the World Series but didn't actually attend games.
1961: 2 teams averaged over 20k a game attendance, 6 teams under 10k.
Cardinals averaged 10,965
Red Sox averaged 10,373
So no one went to regular season games, there were no games besides the World Series on TV, but it was the golden age of baseball? Thanks I'll take 2014 when even pud markets get 18k a game, every team is on TV, and you don't have to wait to open up baseball card packs to know a player's stats.