Ben Affleck Thinks His Family Is 'Charmed' by His Love for Dunkin': 'It's What I Tell Myself'

Ben Affleck is doing his best to pass on his love for Dunkin'.

The actor, 50, whose family includes wife Jennifer Lopez, and kids Violet Anne, 17, Seraphina Rose, 14, and Samuel, 10, from his marriage to ex Jennifer Garner, tells PEOPLE in this week's issue that his children are amused by his passion for all things coffee.

"I think they're charmed by it. I like to think so. I don't know if it's true, but, just as with so many other things, it's what I tell myself," he said on set of his Dunkin' Super Bowl commercial. "I think it's tolerated, which is the most you can hope for as the parent of adolescents, I found."

The Air director went back to his home state of Massachusetts to film the big game commercial. In it he plays a Dunkin' employee and serves unsuspecting customers — and Lopez, 53 — their beverages and treats in a hidden camera-like spot.

Because "there aren't as many" Dunkin' locations in Los Angeles where his family lives now, Affleck said his three kids look at their dad's love for the coffee company as something unique.

"I think they think of it as a slightly eccentric habit of their dad's, that like, there's this one coffee shop that I really stick with and go to," he said with a laugh, "but they know that it's associated with sort of where I grew up and Boston."

(The first Dunkin' store to ever open was located in Quincy, Mass.)

Affleck told PEOPLE that getting to moonlight as an employee was enjoyable. "I think people already sort of think that I work for Dunkin'," he joked.

"In Boston, it was such a big deal," he said. "I think I became kind of associated with it, and this sounded like a fun opportunity to kind of play with that association, and a chance to come back here to shoot it at home in Boston."