Game of Thrones fans have been waiting for almost a decade to discover who will claim the Iron Throne. There have been many viable candidates, some still living and many dead, but the most recent episode introduced an unanticipated contender: The lowly brick. (Editor’s warning: GoT spoilers and fascinating historical information about a seemingly innocuous building material ahead.)

As King’s Landing and its crowning ruby, the Red Keep, burned under trial-by-dragon, bricks were falling everywhere—often with deadly results. Fans took their red-hot anger to Twitter, transforming mediocre stock photos of construction materials into angsty memes.

The complete unraveling of eight seasons worth of character-building was certainly hard to stomach. But now that the episode’s over, it’s time to acknowledge a painful truth. We should always have seen the brick coming.

Bricks are so omnipresent as to be invisible, but they’re the sturdy block upon which the modern world has been built. The Great Wall of China, which has withstood thousands of years of war and weather, is part brick and part sticky rice mortar. The Roman empire’s preferred building block was red or white kiln-fired sand. The city of Chicago, rebuilt after a devastating 1871 fire, rose taller than any other city on earth (for a time, anyway), thanks to the skyscraping combination of fire-proof brick and high-tensile steel. And if that isn’t enough, ask yourself: Where would Instagram or your favorite coffee shop even be without the ascendancy of the exposed brick wall?

As the BBC explains in its series “50 Things That Made the Modern Economy,” the humble rectangle has never been a static object, instead evolving constantly across its 10,000 year history. The earliest strategy was the simplest: hand-shaped mud bricks left out to solidify in the sun. Then came the first molds, allowing for quick and tidy mass production. But the most important—and most expensive—innovation was baking bricks. Firing the sandy slabs in a kiln around 2,000 degrees Fahrenheit created a more durable building material. It also opened up new markets for bricklayers, who could now raise little red houses and big masonry bridges in colder climes, where the sun wasn’t always a reliable cook.

They may all appear similar to the untrained eye, but bricks have always been an individualized product. Today there are tens of thousands of varieties on the market—and in the hands of avid brick collectors (and brick thieves). In Milwaukee, Wisconsin, many old buildings feature “Cream City” bricks, an unusually light yellow color block thanks to local clay. In the Windy City, where hundreds of old buildings have been demolished, a reuse market for certified “Old Chicago” bricks have formed. Down in St. Louis, historic bricks can be tied back to one of the 50-or-so brick manufacturers that operated there in the mid-19th century. Wherever you live, your city’s bricks tell a story.

And the real-life-Legos continue to change. Eco-conscious companies like StoneCycling are carving out their own niche, fashioning bricks from recycled waste like old windows and porcelain from trashed toilets. Scientists are testing bricks made from facsimile Martian soil, in the hopes of preparing future builders for the demands of off-Earth construction. And who will those builders be? Masonry, one of the oldest professions on this planet, is increasingly outsourced to robots. One machine, called the Semi-Automated Mason, or SAM, can lay bricks at least three times faster than humans.

Taking bricks for granted has never served us well. Like all infrastructure, they require maintenance in order to function year after year, decade after decade. “Old brick buildings are one of the deadliest structures in an earthquake,” according to the Los Angeles Times. While individual bricks are exceedingly durable, the paste that holds them together is easily shaken loose. From Southern California to Seattle, historic buildings are at risk of collapse in the next seismic event, especially those that have not been retrofitted.

Even without a 9.0-trigger, crumbling brick facades can cause serious damage: Search “falling bricks kill people,” and you’ll find plenty of preventable tragedies, like the death of 2-year-old Greta Greene, who was struck by falling debris in New York City after the building’s owners and municipal inspectors reportedly ignored signs of damage. Such events, while smaller than a bridge collapse or an apartment building fire, are a reminder of just how much work maintaining our shared infrastructure really takes.

Most of us don’t dream about masonry or think about construction materials on our walk to work. But taking a page from avid brick collectors or accessibility activists fighting for better paving might do us all some good. The brick wasn’t the monarch Game of Thrones devotees wanted, but perhaps it’s the monarch we deserve.