Do you think my husbands behavior is something to be worried about?

This obviously needs a lot of explaining. We live next to this woman, who he has known his entire life. He isn't related to her, but he claims that she is like his "little sister". But, their behavior concerns me. First off, after knowing her for over a year, she confirms to me that she used to love him and she hated me at first. It's been seven years, but she STILL brings this up. I am now married to him. They text and call each other frequently and she always says she loves him before hanging up/ending texting him. He has a BIG past of lying to me- about internet browsing history, porn watching and everytime he has swore on our relationship that he wasnt and even one time on our unborn child that he wasnt (at that time we were trying). I feel like there's something going on between them and when I try to ask him about it he gets defensive and swears that I'm being ridiculous. It's not just there language, its there "playful manner". They will kidfully punch or touch each other, and even today when she was wearing booty shorts he spanks her leg, right below her butt. I feel like I don't have enough evidence that they are, but I just wanted to see how others perceived the relationship. I try to be a wife that isnt harporing on him all the time and I dont want him to feel like I dont trust him, but the truth is I don't because he continues to lie to me. Any advice would be greatly appreciated.