It's simple. This is the year of our lord 2015. You can talk about a rebellious spirit and "heritage" all you want. But there is more than enough material out there to tell you exactly what that flag means to black people. And there's more than enough material to tell you that it only resurfaced precisely because of a racist reaction to the Civil Rights movement. And no, it's not the same thing as the US flag. Plenty of horrendous things have been done in the name of Ol' Glory, but it wasn't unearthed specifically as a way to terrorize an entire race that was trying to fight for the civil rights owed to them.
So if you willfully fly that flag in 2015, you can protest how much you aren't a racist, but it is damn near the equivalent of "I have black friends tho." If you're flying it, you're well aware of the horrors it represents, and you're (at the very least, subconsciously) a racist bad person. So don't try and insult our intelligence. Either take it down or rough ridin' own your racism so all of your neighbors can know just how much of a crap human being you are.