Well, looks like you found a relationship you can finally sink your teeth into...
I jest, but just a bit. There is a certain allure regarding "forbidden fruit", and I think you need to do some soul searching to see if that is what's going on here.
Then there is just the proximity of him--I mean, when he's there, he's right there, right? So there is a certain familiarity that you might misconstrue as the hormones rage and pillow fights ensue--do I guess right?
My suggestion is that you just give it time.
EVERYBODY has feelings of attraction that may be deemed "inappropriate" at times. It's perfectly natural. There is a difference between "feeling attraction", however, and acting on that attraction. The fact that you posted this here suggests to me that you already realize there will be serious consequences of such a daliance. I promise you, if it's drama you want, life will offer you plenty.
Slow down, chill out, and by all means, go look around out there in the big wide world.
There's plenty of forbidden fruit elsewhere.
If not, shrug--I hear Arkansas is pretty!
Much luck to you.