I know I've been telling people that abusers are attracted to people who have been abused before, as if they can sense it but sometimes that's not it, although that's some of it. Abusers will choose what they think is a likely target, and try different women to see what their response is. From what I've heard and read, they'll sometimes start off by testing small boundaries first. He will make a note of the type of women who allowed the boundary crossing, (or didn't leave quickly enough) and target that "type" next time. They're shopping!
He will make a note of the kinds of women he tried to abuse, but couldn't, and not approach those women, because he considers them too much work. He has every intention of being abusive from the jump, but will test the waters first, so to speak, to see if a woman will allow it, or leave him. If she lets him cross her boundaries, he will stay and escalate. If she leaves, he'll move on to test the next woman.
It's got nothing to do with what you're doing. You're not doing anything but existing, while they are "shopping" for their next victim.