For one, we may not have an accurate view of how common terrestrial planets are. Out of the estimated 100 billion planets in the Milky Way, we've only seen 1000 of them--just a tiny, infinitesimal sample of the total. Plus, smaller, Earth-sized planets are harder to detect, so who knows how many we're missing. And we've only really zoomed in on the planets in one small piece of the sky, within just one galaxy. It's like trying to guess the composition of a big, complex jigsaw puzzle when you only have one or two pieces in your hand.