It's your story, so it is your choice. It is hard to draw such a line, since strictly speaking all fanfiction (especially in this particular fandom) is an alternative take on the canon.
I've noticed a habit for some at least to divide Canon and AU by the setting - feudal or modern, that's one way to draw the line, I guess, but there are many ways to set the story in the feudal era and still make it AU.
My personal opinion is that you should change the universe to AU
if you feel you have deviated too much from a canon setting or altered the canon storyline excessively. For me it would be enough of a reason to change into AU if the cultural influences you have incorporated are not historically accurate. But that's just me.
You should do as you feel best. Think of which feels more right to you, and which universe allows you the best premise to develop your story.